x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Optimal Targeting in Fundraising: A Causal Machine-Learning Approach
Last registered on April 22, 2021

Pre-Trial

Trial Information
General Information
Title
Optimal Targeting in Fundraising: A Causal Machine-Learning Approach
RCT ID
AEARCTR-0007581
Initial registration date
April 21, 2021
Last updated
April 22, 2021 6:22 AM EDT
Location(s)
Region
Primary Investigator
Affiliation
Johannes Kepler University Linz
Other Primary Investigator(s)
PI Affiliation
CREST-ENSAE, Institut Polytechnique Paris and CESifo
PI Affiliation
University of Erlangen-Nuremberg
PI Affiliation
Deutsche Bundesbank
Additional Trial Information
Status
Completed
Start date
2014-03-01
End date
2016-03-01
Secondary IDs
Abstract
Ineffective fundraising lowers the resources charities can use for goods provision. We combine a field experiment and a causal machine-learning approach to increase a charity's fundraising effectiveness. The approach optimally targets fundraising to individuals whose expected donations exceed solicitation costs. Among past donors, optimal targeting substantially increases donations (net of fundraising costs) relative to benchmarks that target everybody or no one. Instead, individuals who were previously asked but never donated should not be targeted. Further, the charity requires only publicly available geospatial information to realize the gains from targeting. We conclude that charities not engaging in optimal targeting waste resources.
External Link(s)
Registration Citation
Citation
Cagala , Tobias et al. 2021. "Optimal Targeting in Fundraising: A Causal Machine-Learning Approach." AEA RCT Registry. April 22. https://doi.org/10.1257/rct.7581-1.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
We demonstrate the potential of machine-learning-based optimal targeting using a common fundraising instrument as an example: small unconditional gifts that accompany a solicitation letter. We teamed up with a charity that sends out solicitation letters to its donor base once a year. In this setting, we randomly assigned almost 20,000 potential donors to a gift treatment (in which individuals received the solicitation letter with a gift) and a control group (in which they received only the letter).

The goal of optimal targeting lies in directing a fundraising instrument (a gift in our example) to a subset of individuals such that the fundraising campaign's expected profits are maximized (i.e., the additional expected net donations collected by the campaign). Hence, our main outcome variable are net donations.

Yet, who should charities target for that purpose? By definition, the optimal targeting rule states that the profit-maximizing targets are the so-called net donors: individuals whose expected additional donation is higher than the marginal fundraising cost. In an ideal world, the charity would know each individual's donation with and without the fundraising instrument (i.e., the gift) and could, thus, determine the set of net donors. In reality, however, this set is unknown to the charity. The reason is that donations under both conditions (gift vs. no gift) are unknown before the campaign. Moreover, even after the campaign, the charity could still only learn a donor's behavior under her assigned condition. These complications motivate our approach to identify optimal targets for fundraising activities.

Our approach to optimal targeting relies on two ingredients. First, it exploits random variation in the assignment of an unconditional gift at the donor level. Second, as previously highlighted, our approach relies on machine-learning algorithms. Particularly, we let an algorithm learn the relationship between the individuals' observable characteristics and their expected donation behaviors in the experiment's treatment and control groups (i.e., with and without the gift). From this relationship, we then estimate the (out-of-sample) set of predicted net donors who should be targeted. This procedure establishes what we label the estimated optimal targeting rule. Importantly, although we cannot trace out the causal effect of the characteristics that drive the heterogeneity, the policy-relevant increase in net donations achieved by the targeting rule is causally identified.

More specifically, our paper draws on the following machine-learning methods. We implement the optimal-policy-learning algorithm of Athey and Wager (2021), which extends the empirical welfare-maximization approach of Kitagawa and Tetenov (2018) by machine learning. In our main specifications, we then estimate optimal targeting rules with the Exact Policy-Learning Tree of Zhou et al. (2018).

We feed our algorithm with information from various sources and of different types, including socioeconomic characteristics, past donation data, and geospatial information. The geospatial information consists of publicly available information from Google Maps on economic and cultural facilities close to the potential donor's place of residence.
Intervention Start Date
2014-03-01
Intervention End Date
2016-03-01
Primary Outcomes
Primary Outcomes (end points)
The goal of optimal targeting lies in directing a fundraising instrument (a gift in our example) to a subset of individuals such that the fundraising campaign's expected profits are maximized (i.e., the additional expected net donations collected by the campaign). Hence, our main outcome variable is net donations in the first year after the intervention.

We feed our algorithm with information from various sources and of different types, including socioeconomic characteristics, past donation data, and geospatial information. The geospatial information consists of publicly available information from Google Maps on economic and cultural facilities close to the potential donor's place of residence.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Our secondary outcomes are:
Donation probability in the first and first two years after the intervention
Net donations in the first two years after the intervention
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
In 2014, we implemented a natural field experiment in collaboration with a fundraiser of the Catholic Church that operates in a German urban area. For decades, this fundraiser has organized a large-scale, annual fundraising campaign: Once a year, it has mailed solicitation letters to all resident church members, irrespective of previous donations. This fund drive aims to finance local church-related projects, such as the renovation of clergy houses, parish centers, or churches. Our experiment exploited this campaign by (a) experimentally altering how the fundraiser contacted potential donors in 2014 and (b) analyzing individuals' behavior in 2014 and 2015.

Control group. Individuals in our experiment's control group received the standard solicitation letter, the contents of which remained unchanged from the pre-experiment years. Particularly, the letter highlighted the fundraiser's cause and asked recipients for a donation. To lower transaction costs, the fundraiser distributed the solicitation letter together with a remittance slip, prefilled with the fundraiser's bank account and the donor's name. In the pre-experiment years, potential donors received identical transaction forms.

Treatment group. Our design of the gift treatment closely follows Falk (2007). In 2014, individuals in this treatment received the solicitation letter together with an unconditional gift. The gift consisted of three envelopes paired with different folded cards. Further, we added one sentence to the solicitation letter, stating that the fundraiser „would like to provide the included folded cards as a gift.“ The total per-unit cost for mailing the control-group solicitation letter amounted to 0.43 euro (printing plus postage). In the gift treatment, the per-unit cost increased by 1.16 euro (postcards and envelopes: 0.47 euro; boxing and additional postage: 0.69 euro). Notably, our treatment consisted of a one-time intervention. From 2015 onward, all individuals in the sample received a solicitation letter very similar to the one distributed in the pre-experiment years.

Our setting serves as a suitable testing ground for machine-learning-based optimal targeting. First, as the fundraiser did not employ any targeting strategies before the experiment, the setting offers a clean environment to study our machine-learning approach's potential. Second, it provides rich data that not only allow us to estimate powerful targeting rules, but also enable us to test which type of data are especially beneficial for machine-learning-based optimal targeting (see the following description of the data). Third, because the fundraiser contacts all church members exhaustively, the setting offers the possibility to study the cold and warm lists separately. We, hence, can not only examine the optimal targeting of gifts among past donors, but also explore whom to target in the process of acquiring new donors. Fourth, because we were able to gather data for two postexperiment years, the setting allows us to study if the estimated optimal targeting rule increases total donations or simply pulls forward donations from 2015 to 2014. Fifth, because religious giving dominates the charitable giving landscape, targeting is particularly relevant in this context.
Experimental Design Details
In 2014, we implemented a natural field experiment in collaboration with a fundraiser of the Catholic Church that operates in a German urban area. For decades, this fundraiser has organized a large-scale, annual fundraising campaign: Once a year, it has mailed solicitation letters to all resident church members, irrespective of previous donations. This fund drive aims to finance local church-related projects, such as the renovation of clergy houses, parish centers, or churches. Our experiment exploited this campaign by (a) experimentally altering how the fundraiser contacted potential donors in 2014 and (b) analyzing individuals' behavior in 2014 and 2015.

Control group. Individuals in our experiment's control group received the standard solicitation letter, the contents of which remained unchanged from the pre-experiment years. Particularly, the letter highlighted the fundraiser's cause and asked recipients for a donation. To lower transaction costs, the fundraiser distributed the solicitation letter together with a remittance slip, prefilled with the fundraiser's bank account and the donor's name. In the pre-experiment years, potential donors received identical transaction forms.

Treatment group. Our design of the gift treatment closely follows Falk (2007). In 2014, individuals in this treatment received the solicitation letter together with an unconditional gift. The gift consisted of three envelopes paired with different folded cards. Further, we added one sentence to the solicitation letter, stating that the fundraiser „would like to provide the included folded cards as a gift.“ The total per-unit cost for mailing the control-group solicitation letter amounted to 0.43 euro (printing plus postage). In the gift treatment, the per-unit cost increased by 1.16 euro (postcards and envelopes: 0.47 euro; boxing and additional postage: 0.69 euro). Notably, our treatment consisted of a one-time intervention. From 2015 onward, all individuals in the sample received a solicitation letter very similar to the one distributed in the pre-experiment years.

Our setting serves as a suitable testing ground for machine-learning-based optimal targeting. First, as the fundraiser did not employ any targeting strategies before the experiment, the setting offers a clean environment to study our machine-learning approach's potential. Second, it provides rich data that not only allow us to estimate powerful targeting rules, but also enable us to test which type of data are especially beneficial for machine-learning-based optimal targeting (see the following description of the data). Third, because the fundraiser contacts all church members exhaustively, the setting offers the possibility to study the cold and warm lists separately. We, hence, can not only examine the optimal targeting of gifts among past donors, but also explore whom to target in the process of acquiring new donors. Fourth, because we were able to gather data for two postexperiment years, the setting allows us to study if the estimated optimal targeting rule increases total donations or simply pulls forward donations from 2015 to 2014. Fifth, because religious giving dominates the charitable giving landscape, targeting is particularly relevant in this context.
Randomization Method
Stratified randomization done in office by a computer. The strata were defined based on list (warm vs. cold), gender, household type indicators, quintiles of individuals’ predicted baseline willingness to give, and quintiles of age. To construct a proxy for the baseline willingness to give in the treatment year, we first regressed an indicator variable for giving in the year before the experiment on indicator variables for further lags of the giving indicator. We then use dthe estimated model to predict the probability of giving in the treatment year (out of sample).
Randomization Unit
Individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
0
Sample size: planned number of observations
19779
Sample size (or number of clusters) by treatment arms
Treatment group: 3463
Control group: 16316
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Main outcome: donations in first year after the experiment Power calculations: based on 2013 data (year before the experiment) Mean donation in 2013 (warm+cold list): 2.450162 euro SD in 2013: 13.8711 Minimum detectable effect (treatment versus control; power: 0.8 ): 0.7271 euro (0,0524 SDs)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
Yes
Intervention Completion Date
March 01, 2016, 12:00 AM +00:00
Is data collection complete?
Yes
Data Collection Completion Date
Final Sample Size: Number of Clusters (Unit of Randomization)
19779
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
19779
Final Sample Size (or Number of Clusters) by Treatment Arms
Treatment group: 3463 Control group: 16316
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
Abstract
Ineffective fundraising lowers the resources charities can use for goods provision. We combine a field experiment and a causal machine-learning approach to increase a charity's fundraising effectiveness. The approach optimally targets fundraising to individuals whose expected donations exceed solicitation costs. Among past donors, optimal targeting substantially increases donations (net of fundraising costs) relative to benchmarks that target everybody or no one. Instead, individuals who were previously asked but never donated should not be targeted. Further, the charity requires only publicly available geospatial information to realize the gains from targeting. We conclude that charities not engaging in optimal targeting waste resources.
Citation
Cagala et al. (2021) Optimal Targeting in Fundraising: A Causal Machine-Learning Approach. Mimeo.
REPORTS & OTHER MATERIALS