Experimental Design
In the 2008 paper, we identify demand curves for consumer credit by randomizing both the interest rate offered to each of more than 50,000 past clients on a direct mail solicitation, and the maturity of an example loan shown on the offer letter. First, the Lender randomized the interest rate offered in “pre-qualified,” limited-time offers that were mailed to 58,168 former clients with good repayment histories. Most of the offers were at relatively low rates. Clients eligible for maturities longer than four months also received a randomized example of either a four-, six-, or twelve-month loan. Clients wishing to borrow at the offer rate then went to a branch to apply, per the Lender’s standard operations. Final credit approval (i.e., the Lender’s decision on whether to offer a loan after updating the client’s information) and maximum loan size and maturity supplied were orthogonal to the experimental interest rate by construction.
The sample frame consisted of all individuals from 86 predominantly urban branches who had borrowed from the Lender within the past 24 months, were in good standing, and did not currently have a loan from the Lender as of 30 days prior to the mailer. The experiment was implemented in three mailer “waves” of mailer/start dates that grouped branches geographically, for logistical reasons. We pilot-tested in three branches during July 2003 (wave 1), and then expanded the experiment to the remaining 83 branches in two additional waves that started with mailers sent in September 2003 (wave 2) and October 2003 (wave 3). The offer rate randomization was stratified by the client’s pre-approved risk category because risk determined the loan price under standard operations. The standard schedule for four-month loans was: low-risk = 7.75 percent per month; medium-risk = 9.75 percent; high-risk = 11.75 percent. The randomization program established a target distribution of interest rates for four-month loans in each risk category and then randomly assigned each individual to a rate based on the target distribution for her category. Rates varied from 3.25 percent per month to 14.75 percent per month. At the Lender’s request, 96 percent of the offers were at lower-than-standard rates, with an average discount of 3.1 percentage points on the monthly rate (the average rate on prior loans was 11.0 percent). Slightly more than 1 percent of the offers were at a higher-than-standard rate (with a 1.9 percentage point increase on average), and the remaining offers were at the standard rate. At the time of the randomization, we verified that the assigned rates were uncorrelated with other known information, such as credit report score.
A subset of borrowers in waves two and three received mailers containing a randomized maturity suggestion as well. The suggestion took the form of a nonbinding “example” loan showing one of the Lender’s most common maturities (four, six, or twelve months), where the length of the maturity was randomly assigned. This randomization was orthogonal to the interest rate randomization. All letters clearly stated that other loan sizes and maturities were available. The example loan size presented was not randomized; it was the client’s last loan size. Only low- and medium-risk borrowers were eligible to receive the suggestion randomization, since high-risk borrowers could not obtain maturities greater than four months under the Lender’s standard operations. Of low- and medium-risk clients (of whom 493 borrowed), 3,096 received a suggestion (51 percent four-month, 25 percent six-month, 24 percent twelve-month). Loan officers were instructed to ignore any example loan(s) featured in the letter. In both training and ongoing monitoring, the Lender’s management and the research team stressed to branch personnel that the mailers were for marketing and pricing purposes only, and should not have any impact on the loan officer’s underwriting of the loan application.
Each mailer contained a deadline, ranging from two to six weeks, by which the client had to respond in order to be eligible for the offer rate. The Lender routinely mailed teasers to former borrowers but had never promoted specific interest rate offers before this experiment. A total of 1,358 mailers were returned to the Lender by the postal service and 3,000 contained atypical (i.e., nondecreasing) relationships between loan maturity and price, leaving us with a sample frame of 53,810 offers for analysis of demand elasticities. Clients accepted the offer by entering a branch office and filling out an application in person with a loan officer. Applicants did not need to bring the mailer with them to get the offer rate, since each randomly assigned rate was hard-coded into the Lender’s computer systems by client account number. Data collected by the Lender suggests that many clients read their letter, but this must be interpreted cautiously given that letter-reading is unverifiable. Strong demand responses to randomly assigned marketing content treatments contained in the direct mail solicitations provide additional evidence that many did read their. Loan applications were taken and assessed per the Lender’s standard underwriting procedures. Specifically, loan officers: (a) updated observable information and decided whether to offer any loan based on their updated risk assessment; (b) decided the maximum loan size to offer the accepted applicants; and (c) decided the longest loan maturity to offer the accepted applicants. Each decision was made “blind” to the experimental rates, with strict operational controls (including software developed in consultation with the research team), ensuring that loan officers instead used the Lender’s standard rates in any debt service calculations. This rule was designed to prevent loan supply from adjusting endogenously to a lower rate (due to debt service ratios) and thereby complicating estimation of loan size demand elasticities. A total of 4,540 clients (out of 53,810) in our sample frame applied for a loan at the offered interest rate (i.e., before the deadline on the letter), an 8.4 percent application rate. Of these, 86 percent, or 3,887, were approved for a loan. Following the loan officer’s assessment, approved clients chose an allowable loan size and maturity. All clients who were approved ended up taking a loan. This is not surprising, given that the typical application process takes only 45 minutes and everyone in our sample had borrowed from the Lender before.
In the 2010 paper, we drew our sample frame from the universe over 3,000 “new” applicants who had no prior borrowing from the Lender and applied at any of 8 branches between September 21 and November 20, 2004. Our sample frame was comprised of “marginal” applicants: new, rejected, but potentially creditworthy. Specifically, applicants were eligible for the loan randomization if they were rejected under the Lender’s normal underwriting criteria but not deemed egregiously uncreditworthy by a loan officer. 787 applicants met these criteria. The Lender implemented the experiment in four steps. First, loan officers evaluated each of about 3,000 new applicants using the Lender’s standard underwriting process and three additional steps. The experiment forced loan officers to take the first additional step of dividing the “reject” category into two bins. “Marginal” rejects would be eligible for treatment; “egregious” rejects would not be assigned a loan under any circumstances. Egregious rejects were identified subjectively, based on extremely poor credit history, over-indebtedness, suspected fraud, lack of contactability, or legal problems. During our study period loan officers approved 1,405 new applications based on the standard underwriting criteria. 705 applications were deemed egregious rejects, leaving us with a sample frame of 787 marginally rejected applicants for the experiment. Second, special “randomizer” software encouraged loan officers to reconsider randomly selected marginal rejects. Loan officers inputted basic information (name, credit history, maximum feasible loan size if approved, and reason for rejection) on each of the 787+705 = 1,492 rejected applications into the randomizer. The randomizer then used the inputted information to treat applications with probabilities that were conditional on the credit score and loan officer assessment. The treatment was simply a message on the computer screen that the application had been “approved” (control applicants remained “rejected”). The 705 egregious applications had zero probability of being treated. The 787 marginal applicants were divided into two groups based on their credit score. Those with better credit scores were treated with probability 0.50, and those with worse credit scores were treated with probability 0.25 (all analysis controls for this condition of the randomization). In total, 325 applicants were assigned to the treatment group, leaving 462 in the control group.
Last, the branch manager made the final credit decision and announced it to the applicant. The applicant was not privy to the loan officer’s initial decision, the existence of the software, or the introduction of a randomized step in the decision-making process. Accepted applicants were offered an interest rate, loan size, and maturity per the Lender’s standard underwriting criteria. Recall that nearly all received the standard contract for first-time borrowers: a 4-month maturity at 200% APR. Loan repayment was monitored and enforced according to normal operations. Branch manager compensation was based in part on loan performance, and the experiment did not change incentive pay. Following the experiment, we hired a firm to survey applicants in the treatment and control groups. The purpose of the survey was to measure behavior and outcomes that might be affected by access to credit. Surveyors completed 626 surveys, for an 80% response rate.