x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment
Last registered on April 12, 2017

Pre-Trial

Trial Information
General Information
Title
The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment
RCT ID
AEARCTR-0001849
Initial registration date
April 11, 2017
Last updated
April 12, 2017 3:59 PM EDT
Location(s)
Primary Investigator
Affiliation
Northwestern University
Other Primary Investigator(s)
PI Affiliation
Clemson University
Additional Trial Information
Status
Completed
Start date
2007-06-01
End date
2014-04-01
Secondary IDs
Abstract
We test how donors respond to new information about a charity's effectiveness. Freedom from Hunger implemented a test of its direct marketing solicitations, varying letters by whether they include a discussion of their program's impact as measured by scientific research. The base script, used for both treatment and control, included a standard qualitative story about an individual beneficiary. Adding scientific impact information has no effect on average likelihood of giving or average gift amount. However, we find important heterogeneity: large prior donors are both more likely to give and also give more, whereas small prior donors are less likely to give. This patter is consistent with two different types of donors: warm glow donors who respond negatively to analytical effectiveness information, and altruism donors who respond positively to such information.
External Link(s)
Registration Citation
Citation
Karlan, Dean and Daniel Wood. 2017. "The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment." AEA RCT Registry. April 12. https://doi.org/10.1257/rct.1849-1.0.
Former Citation
Karlan, Dean and Daniel Wood. 2017. "The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment." AEA RCT Registry. April 12. https://www.socialscienceregistry.org/trials/1849/history/16475.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
The researchers conducted a randomized controlled trial to test how charity donors respond to information about a charity's effectiveness. The RCT was conducted in two waves in June 2007 and October 2008. Partnering with Freedom from Hunger, the researchers randomly assigned the recipients of a direct-mail solicitation campaign to receive a donation renewal letter which featured an emotional appeal (control group), or a donation renewal letter which included the same emotional appeal as well as evidence from a randomized trial on the charity's positive impact (treatment group).

The researchers developed a model in which charitable donors have two motivations for donating to charity: altruism, whereby the donor gains utility by social welfare gain, and warm-glow, whereby the donor gains utility from the act of donating. In this model, donation size can serve as a proxy for distinguishing altruistic donors from warm-glow donors. In addition, altruistic donors should respond favorably to information about a charity's effectiveness, while warm-glow donors may find the emotional appeal of donating lessened by the additional information.

The researchers found no effects on average likelihood to donate or on amount donated. However, the average effect masked heterogeneous effects on large donors (>$100) versus small donors (<$100). Large donors exposed to treatment were both more likely to donate and donated more, whereas small donors exposed to treatment were less likely to donate and donated less on average.
Intervention Start Date
2007-06-01
Intervention End Date
2008-10-31
Primary Outcomes
Primary Outcomes (end points)
Making any donation (within 5 months of receiving the mailer); Donation amount (within 5 months of receiving the mailer)
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
In the first wave, mail solicitations were sent to 16,889 individuals who had donated to FFH in the previous three years. Recipients were randomized into a control group of 11,258 and a treatment group of 5,631. The control group was divided into two subgroups of 5,628 and 5,630. Randomization was stratified by the most recent donation year and whether the last donation was more or less than $100. The control group consisting of 5,628 recipients received a standard donation renewal letter, while the control group consisting of 5,630 recipients received a standard renewal letter with an additional insert describing the emotional story of a participant in one of FFH's programs in Bolivia. The treatment group received the standard donation renewal letter and the same insert, except the final paragraph was replaced with one detailing the positive impact of the program as demonstrated by "rigorous scientific methodologies".

In the second wave, mail solicitations were sent to 17,784 individuals who had donated to FFH in the previous three years. Recipients were randomized into a control group of 5,960 and two treatment groups of 5,903 and 5,921. Randomization was stratified by whether the last donation was more or less than $100, treatment assignment in the first wave, and whether the recipient had donated in 2008. The control group received the standard donation renewal letter with a personal story and emotional appeal. The treatment group of 5,903 received the same renewal letter, except additional paragraphs were added which described the positive impact of the programs. The treatment group of 5,921 received the same renewal letter, but the information about the positive impacts of the programs explicitly cited Yale-affiliated researchers.
Experimental Design Details
Randomization Method
Randomization done in office by computer
Randomization Unit
First wave: randomized at individual level, stratified by prior giving amount and recency of donation
Second wave: randomized at individual level, stratified by prior giving amount, whether they donated the previous year, and treatment status in first wave
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
First wave: 16,889 individuals
Second wave: 17,784 individuals
Sample size: planned number of observations
First wave: 16,889 individuals Second wave: 17,784 individuals
Sample size (or number of clusters) by treatment arms
First wave: 5,631 individuals in treatment group; 11,258 individuals in control group
Second wave: 5,903 individuals in first treatment group; 5,921 individuals in second treatment group; 5,960 individuals in control group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Study has received IRB approval. Details not available.
IRB Approval Date
Details not available
IRB Approval Number
Details not available
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
Yes
Intervention Completion Date
October 31, 2008, 12:00 AM +00:00
Is data collection complete?
Yes
Data Collection Completion Date
March 31, 2009, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
First wave: 16, 889 individuals
Second wave: 17,784 individuals
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
First wave: 16, 889 individuals
Second wave: 17,784 individuals
Final Sample Size (or Number of Clusters) by Treatment Arms
First wave: 5,631 individuals in treatment group; 11,258 individuals in control group Second wave: 5,903 individuals in first treatment group; 5,921 individuals in second treatment group; 5,960 individuals in control group
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers
Abstract
We test how donors respond to new information about a charity's effectiveness. Freedom from Hunger implemented a test of its direct marketing solicitations, varying letters by whether they include a discussion of their program's impact as measured by scientific research. The base script, used for both treatment and control, included a standard qualitative story about an individual beneficiary. Adding scientific impact information has no effect on whether someone donates, or how much, in the full sample. However, we find that amongst recent prior donors (those we posit more likely to open the mail and thus notice the treatment), large prior donors increase the likelihood of giving in response to information on aid effectiveness, whereas small prior donors decrease their giving. We motivate the analysis and experiment with a theoretical model that highlights two predictions. First, larger gift amounts, holding education and income constant, is a proxy for altruism giving (as it is associated with giving more to fewer charities) versus warm glow giving (giving less to more charities). Second, those motivated by altruism will respond positively to appeals based on evidence, whereas those motivated by warm glow may respond negatively to appeals based on evidence as it turns off the emotional trigger for giving, or highlights uncertainty in aid effectiveness.
Citation
Karlan, Dean, and Daniel H. Wood. "The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment." Working Paper, June 2015.