Estimating the Causal Effect of Matching on Fundraising Velocity

In Part A of the experiment, a subset of loans posted on a microfinance website will be selected for the experiment, and a randomly selected portion of these will be guaranteed to be provided with additional matching funds.
In Part B of the experiment, participants in a survey will be shown a randomly selected set of loans and will be asked to provide predictions and subjective ratings about these loans.
We will then assess whether the survey responses from Part B can be used to predict the experimental results from Part A.

2019-10-22

2020-02-10

The key object of prediction is the rate at which loans are funded during an initial period after posting. Our primary measure of this rate will be the inverse hyperbolic sine of the funding velocity over the first day.
We will assess how well we measure treatment effects using a variety of metrics (such as bias, mean-squared prediction error, and calibration, as in the working paper (Bernheim, Bjorkegren, Naecker, & Rangel, 2013)).

The period considers for each loan will be the first 24 hours or entire fundraising period (whichever is shorter). Fundraising velocity is the number of (non-matching organization) dollars raised per day during that period.

We may also assess performance across these measures:
Inverse hyperbolic sine of funding velocity, first three days
Square root of funding velocity, first day
Square root of funding velocity, first three days
Funding velocity, first day
Funding velocity, first three days
Fraction of all funds raised on the site received by this loan, during the first day
Fraction of all funds raised on the site received by this loan, during the first three days

These metrics represent different approaches to lowering the noise in the measure of funding velocity.

N/A

In Part A, we consider a target population of loans valued at $1,000 or less. After the start date, as loan listings go live on the microfinance website, loans within this target population will be selected into the experimental sample, and randomly assigned to either unmatched or matched treatments.
Loans will fall into four groups:
Loans that we assign to be unmatched, which are not matched by other matching organizations: control compliers
Loans that we assign to be unmatched, but which are still matched by other matching organizations: control always takers
Loans that we assign to be matched, which would not have been matched by other matching organizations: treated compliers
Loans that we assign to be matched, which are also matched by other matching organizations: treated always takers
For loans that are assigned to be matched, we will provide matching funds to ensure that the loan is listed as matched on the microfinance website for the first 24 hours of posting. However, our matching funds will be listed as lowest priority, so that if the loan meets the criteria of another matching organization, it will instead be matched by them. Thus, we can observe whether treated loans are compliers (funded by randomized match), versus those that would be always takers (funded by other matching organizations).
Starting on the intervention start date, we will include loans into the experimental sample until the matching account is depleted. The date of depletion defines the end date of the intervention. The probability of matching may differ by sector: we may include a sector that is matched at a low rate to assess cannibalization.


Part B represents a separate data collection effort. Participants will be selected from Amazon Mechanical Turk and shown a number of the loans from the experimental sample.
Part B will include loans in the control group (control compliers and control always takers).
Loans will be displayed in two versions: (1) exactly as they appeared in the matching experiment, or (2) edited to add/remove the matching funds indicator. Mechanical Turk participants will provide survey responses about each (real or counterfactual) loan, such as:
How likely would a typical person that has interest in microlending be to lend $25 to this individual?
Agreement with statement:
Making this loan would make you feel good about yourself
Others would approve of someone who contributed to this loan
This business is likely to become self sustaining
This loan would benefit the community, not just the borrower(s)
This person would get by without the loan
This project is a worthy endeavor
I trust this person
I'd like to tell friends about this loan
I'd want others to know that I was a part of this loan
Prediction of fundraising velocity
Fund allocation exercise
These responses will be used to predict actual fundraising velocity. These predictions will then be used to estimate the treatment effect of matching on fundraising velocity. This estimate will then be validated by comparing to the previously run experiment.

Randomization of loans done using the ID numbers of each loan: loans with an identifier ending in zero (e.g., 10, 20, 30) will be assigned to be matched; loans with other identifiers will be assigned to be unmatched.
Participants on Mechanical Turk will be shown a random set of loans selected by computer from the set of loans in previously run experiment.

A loan posted on our partner microfinance website.

No

1

Depends on depletion of matching funds and amount of other matching activity. We are aiming for 2000 loans total, and 500 in the complier subset. 500 Mechanical Turk Participants

Depends on depletion of matching funds and amount of other matching activity. We are aiming for 250 loans in matched and unmatched treatment.

Power calculations using historical lending data show that with the observations afforded for the intended funding amount, the MDE is 7% of the mean at conventional significance and power levels.

Analysis Plan