|
Field
Primary Outcomes (End Points)
|
Before
I will consider three main outcomes. The outcomes variables and their various definitions are stated below:
1. Borrower selection: An indicator variable that equals one if the borrower is selected to receive a loan
2a. Borrower default probability: The definitions of default will follow Field et al. (2013). The time frames will be adjusted to account for the short tenure of the loans. Formally, default will be defined as an indicator variable that equals one if the borrower does not repay the full loan within:
(i) 2 weeks of the due date
(ii) 4 weeks of the due date
(iii) 8 weeks of the due date
2b. Borrower default amount: The loan amount outstanding for clients who had not repaid within 8 weeks of the due date.
3. Loan net profit: Total amount of profit earned or lost on the loan. For instance, if a borrower repays PKR 1,050 for PKR 1,000 loan, and this payment is on time, then net profit equals PKR 50. If the borrower defaults on the loan (where default is defined as not paying the full amount due within 8 weeks of the due date), then net profit is PKR -1,000.
|
After
I will consider three main variables. The outcomes variables and their various definitions are stated below:
1. Borrower selection: An indicator variable that equals one if the loan application is approved, zero otherwise.
2. Borrower default probability: Default is defined as an indicator variable that equals one if any part of the loan amount (including interest and relevant late fees) is overdue:
(i) For more than 8 days after the due date.
(ii) For more than 30 days after the due date.
(iii) For more than 60 days after the due date.
(iv) For more than 365 days after the due date.
3. Loan margin: Total accounting and economic margin on the loan. For illustrative purposes, consider a PKR 1,000 loan with PKR 50 in charged interest. The margins are calculated as follows:
(i) Accounting margin: Total margin earned or lost on the loan. If a borrower repays the loan by the due date, then accounting margin equals PKR 50. If the borrower completely defaults on the loan (where default is defined as having the full amount overdue for more than 365 days after the due date), then accounting margin is PKR -1,000.
(ii) Economic margin: Total margin earned or lost on the loan, taking into account forgone interest. If a borrower repays the loan and interest by the due date, then economic margin equals accounting margin at PKR 50. However, in case of complete default, the economic margin is the accounting margin plus forgone interest on the defaulted principal. Forgone interest is calculated as the product of: Defaulted principal x average APR on similar principal amounts earned by E x average repayment probability.
|
|
Field
Experimental Design (Public)
|
Before
Loan applications received by E will be randomly assigned to one of four treament groups.
In the first and second groups, the exact same applicant information will be provided to a human or fed into the algorithm, respectively. In the third and fourth treatment groups, I will feed additional information usually only observed by humans and algorithms, respectively, in order to ascertain the relative comparative advantages of the two.
|
After
Loan applications received by E will be randomly assigned to one of four treatment groups.
In the first and second groups, the exact same applicant information will be provided to a human or fed into the algorithm, respectively. In the third and fourth treatment groups, I will feed additional information usually only observed by humans and algorithms, respectively.
|
|
Field
Intervention (Hidden)
|
Before
E's current business model uses a machine learning algorithm trained on applicant characteristics as well as unconventional smartphone data in order to make a credit decision within 15 minutes of an application being made. The lending process works as follows: On the company's proprietary mobile application, an applicant fills out a questionnaire that asks for standard information on employment status, income, etc. (hereafter referred to as "hard" or ``limited" information). In addition to these hard data points, E also has access to "soft" information from other, past borrowers' smartphone data. Soft information includes metrics such as social media usage, the time of day individuals make phone calls, etc. In order to make a new lending decision, E feeds in both hard and soft data into a machine learning algorithm that creates a user specific credit score, which is then used to determine credit approval, loan size, and interest rate.
Once approved for a loan, an applicant must provide additional information (hereafter referred to as "identifying" information), including name, age, gender, a picture and their location in order to satisfy E's Know Your Client (KYC) requirements. Importantly, the applicant agrees that E can access her own smartphone data if she accepts the loan. Thus, through this iterative process, the algorithm uses soft data from past borrowers - who share observable characteristics with a current applicant - in order to inform new lending decisions.
We have formed a research partnership with a leading microfinance bank in Pakistan, hereafter referred to as H, in order to recruit loan officers for the proposed experiment. These loan officers will spend two days with the research team, where they will evaluate loan applications that E has already received and approved in the past. Given that these loans have already been administered, I can observe their repayment outcomes ex-ante. The officers will have access to all the information that was entered by an applicant on E's app when they applied for a loan. Officers will not be informed of the fact that loan decisions have already been made. Instead, they will be told that their help is being solicited to screen digital loan applications, and that their approval or rejection of a loan is "real", in that funds will be disbursed if they choose to approve an application. All credit decisions will be made on a custom-made web interface.
There might be concern that I am only able to evaluate those loans that were approved by E. However, when E was testing its product, it randomly approved around 60\%-65\% of loan applications that would \textit{otherwise} have been rejected by its algorithm. This unique feature allows me to observe default outcomes across the full support of the applicant pool rather than for only those applicants that were approved by the algorithm.
|
After
E's current business model uses a machine learning algorithm trained on applicant characteristics as well as unconventional smartphone data in order to make a credit decision within 15 minutes of an application being made. The lending process works as follows: On the company's proprietary mobile application, an applicant fills out a questionnaire that asks for standard information on employment status, income, etc. (hereafter referred to as "hard" or "limited" information). In addition to these hard data points, E also has access to "soft" information from other, past borrowers' smartphone data. Soft information includes metrics such as social media usage, the time of day individuals make phone calls, etc. In order to make a new lending decision, E feeds in both hard and soft data into a machine learning algorithm that creates a user specific credit score, which is then used to determine credit approval, loan size, and interest rate.
Once approved for a loan, an applicant must provide additional information (hereafter referred to as "identifying" information), including name, age, gender, a picture and their location in order to satisfy E's Know Your Client (KYC) requirements. Importantly, the applicant agrees that E can access her own smartphone data if she accepts the loan. Thus, through this iterative process, the algorithm uses soft data from past borrowers - who share observable characteristics with a current applicant - in order to inform new lending decisions.
We have formed a research partnership with a leading microfinance bank in Pakistan, hereafter referred to as H, in order to recruit loan officers for the proposed experiment. These loan officers will spend two days with the research team, where they will evaluate loan applications that E has already received and approved in the past. Given that these loans have already been administered, I can observe their repayment outcomes ex-ante. The officers will have access to all the information that was entered by an applicant on E's app when they applied for a loan. Officers will not be informed of the fact that loan decisions have already been made. Instead, they will be told that their help is being solicited to screen digital loan applications, and that their approval or rejection of a loan is "real", in that funds will be disbursed if they choose to approve an application. All credit decisions will be made on a custom-made web interface.
There might be concern that I am only able to evaluate those loans that were approved by E. However, when E was testing its product, it randomly approved around 60%-65% of loan applications that would otherwise have been rejected by its algorithm. This unique feature allows me to observe default outcomes across the full support of the applicant pool rather than for only those applicants that were approved by the algorithm.
|