The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment

The researchers conducted a randomized controlled trial to test how charity donors respond to information about a charity's effectiveness. The RCT was conducted in two waves in June 2007 and October 2008. Partnering with Freedom from Hunger, the researchers randomly assigned the recipients of a direct-mail solicitation campaign to receive a donation renewal letter which featured an emotional appeal (control group), or a donation renewal letter which included the same emotional appeal as well as evidence from a randomized trial on the charity's positive impact (treatment group).

The researchers developed a model in which charitable donors have two motivations for donating to charity: altruism, whereby the donor gains utility by social welfare gain, and warm-glow, whereby the donor gains utility from the act of donating. In this model, donation size can serve as a proxy for distinguishing altruistic donors from warm-glow donors. In addition, altruistic donors should respond favorably to information about a charity's effectiveness, while warm-glow donors may find the emotional appeal of donating lessened by the additional information.

The researchers found no effects on average likelihood to donate or on amount donated. However, the average effect masked heterogeneous effects on large donors (>$100) versus small donors (<$100). Large donors exposed to treatment were both more likely to donate and donated more, whereas small donors exposed to treatment were less likely to donate and donated less on average.

2007-06-01

2008-10-31

Making any donation (within 5 months of receiving the mailer); Donation amount (within 5 months of receiving the mailer)

In the first wave, mail solicitations were sent to 16,889 individuals who had donated to FFH in the previous three years. Recipients were randomized into a control group of 11,258 and a treatment group of 5,631. The control group was divided into two subgroups of 5,628 and 5,630. Randomization was stratified by the most recent donation year and whether the last donation was more or less than $100. The control group consisting of 5,628 recipients received a standard donation renewal letter, while the control group consisting of 5,630 recipients received a standard renewal letter with an additional insert describing the emotional story of a participant in one of FFH's programs in Bolivia. The treatment group received the standard donation renewal letter and the same insert, except the final paragraph was replaced with one detailing the positive impact of the program as demonstrated by "rigorous scientific methodologies".

In the second wave, mail solicitations were sent to 17,784 individuals who had donated to FFH in the previous three years. Recipients were randomized into a control group of 5,960 and two treatment groups of 5,903 and 5,921. Randomization was stratified by whether the last donation was more or less than $100, treatment assignment in the first wave, and whether the recipient had donated in 2008. The control group received the standard donation renewal letter with a personal story and emotional appeal. The treatment group of 5,903 received the same renewal letter, except additional paragraphs were added which described the positive impact of the programs. The treatment group of 5,921 received the same renewal letter, but the information about the positive impacts of the programs explicitly cited Yale-affiliated researchers.

Randomization done in office by computer

First wave: randomized at individual level, stratified by prior giving amount and recency of donation
Second wave: randomized at individual level, stratified by prior giving amount, whether they donated the previous year, and treatment status in first wave

No

First wave: 16,889 individuals
Second wave: 17,784 individuals

First wave: 16,889 individuals Second wave: 17,784 individuals

First wave: 5,631 individuals in treatment group; 11,258 individuals in control group
Second wave: 5,903 individuals in first treatment group; 5,921 individuals in second treatment group; 5,960 individuals in control group

We test how donors respond to new information about a charity's effectiveness. Freedom from Hunger implemented a test of its direct marketing solicitations, varying letters by whether they include a discussion of their program's impact as measured by scientific research. The base script, used for both treatment and control, included a standard qualitative story about an individual beneficiary. Adding scientific impact information has no effect on whether someone donates, or how much, in the full sample. However, we find that amongst recent prior donors (those we posit more likely to open the mail and thus notice the treatment), large prior donors increase the likelihood of giving in response to information on aid effectiveness, whereas small prior donors decrease their giving. We motivate the analysis and experiment with a theoretical model that highlights two predictions. First, larger gift amounts, holding education and income constant, is a proxy for altruism giving (as it is associated with giving more to fewer charities) versus warm glow giving (giving less to more charities). Second, those motivated by altruism will respond positively to appeals based on evidence, whereas those motivated by warm glow may respond negatively to appeals based on evidence as it turns off the emotional trigger for giving, or highlights uncertainty in aid effectiveness.

Karlan, Dean, and Daniel H. Wood. "The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment." Working Paper, June 2015.

http://www.nber.org/papers/w20047