Optimal Targeting in Fundraising: A Causal Machine-Learning Approach

We demonstrate the potential of machine-learning-based optimal targeting using a common fundraising instrument as an example: small unconditional gifts that accompany a solicitation letter. We teamed up with a charity that sends out solicitation letters to its donor base once a year. In this setting, we randomly assigned almost 20,000 potential donors to a gift treatment (in which individuals received the solicitation letter with a gift) and a control group (in which they received only the letter).

The goal of optimal targeting lies in directing a fundraising instrument (a gift in our example) to a subset of individuals such that the fundraising campaign's expected profits are maximized (i.e., the additional expected net donations collected by the campaign). Hence, our main outcome variable are net donations.

Yet, who should charities target for that purpose? By definition, the optimal targeting rule states that the profit-maximizing targets are the so-called net donors: individuals whose expected additional donation is higher than the marginal fundraising cost. In an ideal world, the charity would know each individual's donation with and without the fundraising instrument (i.e., the gift) and could, thus, determine the set of net donors. In reality, however, this set is unknown to the charity. The reason is that donations under both conditions (gift vs. no gift) are unknown before the campaign. Moreover, even after the campaign, the charity could still only learn a donor's behavior under her assigned condition. These complications motivate our approach to identify optimal targets for fundraising activities.

Our approach to optimal targeting relies on two ingredients. First, it exploits random variation in the assignment of an unconditional gift at the donor level. Second, as previously highlighted, our approach relies on machine-learning algorithms. Particularly, we let an algorithm learn the relationship between the individuals' observable characteristics and their expected donation behaviors in the experiment's treatment and control groups (i.e., with and without the gift). From this relationship, we then estimate the (out-of-sample) set of predicted net donors who should be targeted. This procedure establishes what we label the estimated optimal targeting rule. Importantly, although we cannot trace out the causal effect of the characteristics that drive the heterogeneity, the policy-relevant increase in net donations achieved by the targeting rule is causally identified.

More specifically, our paper draws on the following machine-learning methods. We implement the optimal-policy-learning algorithm of Athey and Wager (2021), which extends the empirical welfare-maximization approach of Kitagawa and Tetenov (2018) by machine learning. In our main specifications, we then estimate optimal targeting rules with the Exact Policy-Learning Tree of Zhou et al. (2018).

We feed our algorithm with information from various sources and of different types, including socioeconomic characteristics, past donation data, and geospatial information. The geospatial information consists of publicly available information from Google Maps on economic and cultural facilities close to the potential donor's place of residence.

2014-03-01

2016-03-01

The goal of optimal targeting lies in directing a fundraising instrument (a gift in our example) to a subset of individuals such that the fundraising campaign's expected profits are maximized (i.e., the additional expected net donations collected by the campaign). Hence, our main outcome variable is net donations in the first year after the intervention.

We feed our algorithm with information from various sources and of different types, including socioeconomic characteristics, past donation data, and geospatial information. The geospatial information consists of publicly available information from Google Maps on economic and cultural facilities close to the potential donor's place of residence.

Our secondary outcomes are:
Donation probability in the first and first two years after the intervention
Net donations in the first two years after the intervention

In 2014, we implemented a natural field experiment in collaboration with a fundraiser of the Catholic Church that operates in a German urban area. For decades, this fundraiser has organized a large-scale, annual fundraising campaign: Once a year, it has mailed solicitation letters to all resident church members, irrespective of previous donations. This fund drive aims to finance local church-related projects, such as the renovation of clergy houses, parish centers, or churches. Our experiment exploited this campaign by (a) experimentally altering how the fundraiser contacted potential donors in 2014 and (b) analyzing individuals' behavior in 2014 and 2015.

Control group. Individuals in our experiment's control group received the standard solicitation letter, the contents of which remained unchanged from the pre-experiment years. Particularly, the letter highlighted the fundraiser's cause and asked recipients for a donation. To lower transaction costs, the fundraiser distributed the solicitation letter together with a remittance slip, prefilled with the fundraiser's bank account and the donor's name. In the pre-experiment years, potential donors received identical transaction forms.

Treatment group. Our design of the gift treatment closely follows Falk (2007). In 2014, individuals in this treatment received the solicitation letter together with an unconditional gift. The gift consisted of three envelopes paired with different folded cards. Further, we added one sentence to the solicitation letter, stating that the fundraiser „would like to provide the included folded cards as a gift.“ The total per-unit cost for mailing the control-group solicitation letter amounted to 0.43 euro (printing plus postage). In the gift treatment, the per-unit cost increased by 1.16 euro (postcards and envelopes: 0.47 euro; boxing and additional postage: 0.69 euro). Notably, our treatment consisted of a one-time intervention. From 2015 onward, all individuals in the sample received a solicitation letter very similar to the one distributed in the pre-experiment years.

Our setting serves as a suitable testing ground for machine-learning-based optimal targeting. First, as the fundraiser did not employ any targeting strategies before the experiment, the setting offers a clean environment to study our machine-learning approach's potential. Second, it provides rich data that not only allow us to estimate powerful targeting rules, but also enable us to test which type of data are especially beneficial for machine-learning-based optimal targeting (see the following description of the data). Third, because the fundraiser contacts all church members exhaustively, the setting offers the possibility to study the cold and warm lists separately. We, hence, can not only examine the optimal targeting of gifts among past donors, but also explore whom to target in the process of acquiring new donors. Fourth, because we were able to gather data for two postexperiment years, the setting allows us to study if the estimated optimal targeting rule increases total donations or simply pulls forward donations from 2015 to 2014. Fifth, because religious giving dominates the charitable giving landscape, targeting is particularly relevant in this context.

In 2014, we implemented a natural field experiment in collaboration with a fundraiser of the Catholic Church that operates in a German urban area. For decades, this fundraiser has organized a large-scale, annual fundraising campaign: Once a year, it has mailed solicitation letters to all resident church members, irrespective of previous donations. This fund drive aims to finance local church-related projects, such as the renovation of clergy houses, parish centers, or churches. Our experiment exploited this campaign by (a) experimentally altering how the fundraiser contacted potential donors in 2014 and (b) analyzing individuals' behavior in 2014 and 2015.

Control group. Individuals in our experiment's control group received the standard solicitation letter, the contents of which remained unchanged from the pre-experiment years. Particularly, the letter highlighted the fundraiser's cause and asked recipients for a donation. To lower transaction costs, the fundraiser distributed the solicitation letter together with a remittance slip, prefilled with the fundraiser's bank account and the donor's name. In the pre-experiment years, potential donors received identical transaction forms.

Treatment group. Our design of the gift treatment closely follows Falk (2007). In 2014, individuals in this treatment received the solicitation letter together with an unconditional gift. The gift consisted of three envelopes paired with different folded cards. Further, we added one sentence to the solicitation letter, stating that the fundraiser „would like to provide the included folded cards as a gift.“ The total per-unit cost for mailing the control-group solicitation letter amounted to 0.43 euro (printing plus postage). In the gift treatment, the per-unit cost increased by 1.16 euro (postcards and envelopes: 0.47 euro; boxing and additional postage: 0.69 euro). Notably, our treatment consisted of a one-time intervention. From 2015 onward, all individuals in the sample received a solicitation letter very similar to the one distributed in the pre-experiment years.

Our setting serves as a suitable testing ground for machine-learning-based optimal targeting. First, as the fundraiser did not employ any targeting strategies before the experiment, the setting offers a clean environment to study our machine-learning approach's potential. Second, it provides rich data that not only allow us to estimate powerful targeting rules, but also enable us to test which type of data are especially beneficial for machine-learning-based optimal targeting (see the following description of the data). Third, because the fundraiser contacts all church members exhaustively, the setting offers the possibility to study the cold and warm lists separately. We, hence, can not only examine the optimal targeting of gifts among past donors, but also explore whom to target in the process of acquiring new donors. Fourth, because we were able to gather data for two postexperiment years, the setting allows us to study if the estimated optimal targeting rule increases total donations or simply pulls forward donations from 2015 to 2014. Fifth, because religious giving dominates the charitable giving landscape, targeting is particularly relevant in this context. Sixth, we can assess the estimated rule’s external validity by exploiting data from a follow-up experiment that took place in 2015 using different participants. Specifically, in 2015, we randomly allocate a new sample of 3,616 warm-list individuals to the control group and the gift treatment. Using the targeting rule estimated based on the 2014 sample, we assess whether the charity would benefit from applying the rule one year later to a different sample of individuals.

Stratified randomization done in office by a computer. The strata were defined based on list (warm vs. cold), gender, household type indicators, quintiles of individuals’ predicted baseline willingness to give, and quintiles of age. To construct a proxy for the baseline willingness to give in the treatment year, we first regressed an indicator variable for giving in the year before the experiment on indicator variables for further lags of the giving indicator. We then use dthe estimated model to predict the probability of giving in the treatment year (out of sample).

Individual

No

0

19779

Treatment group: 3463
Control group: 16316

Main outcome: donations in first year after the experiment Power calculations: based on 2013 data (year before the experiment) Mean donation in 2013 (warm+cold list): 2.450162 euro SD in 2013: 13.8711 Minimum detectable effect (treatment versus control; power: 0.8 ): 0.7271 euro (0,0524 SDs)

Ineffective fundraising lowers the resources charities can use for goods provision. We combine a field experiment and a causal machine-learning approach to increase a charity's fundraising effectiveness. The approach optimally targets fundraising to individuals whose expected donations exceed solicitation costs. Among past donors, optimal targeting substantially increases donations (net of fundraising costs) relative to benchmarks that target everybody or no one. Instead, individuals who were previously asked but never donated should not be targeted. Further, the charity requires only publicly available geospatial information to realize the gains from targeting. We conclude that charities not engaging in optimal targeting waste resources.

Cagala et al. (2021) Optimal Targeting in Fundraising: A Causal Machine-Learning Approach. Mimeo.

https://arxiv.org/pdf/2103.10251.pdf