Experimental Design Details
In 2014, we implemented a natural field experiment in collaboration with a fundraiser of the Catholic Church that operates in a German urban area. For decades, this fundraiser has organized a large-scale, annual fundraising campaign: Once a year, it has mailed solicitation letters to all resident church members, irrespective of previous donations. This fund drive aims to finance local church-related projects, such as the renovation of clergy houses, parish centers, or churches. Our experiment exploited this campaign by (a) experimentally altering how the fundraiser contacted potential donors in 2014 and (b) analyzing individuals' behavior in 2014 and 2015.
Control group. Individuals in our experiment's control group received the standard solicitation letter, the contents of which remained unchanged from the pre-experiment years. Particularly, the letter highlighted the fundraiser's cause and asked recipients for a donation. To lower transaction costs, the fundraiser distributed the solicitation letter together with a remittance slip, prefilled with the fundraiser's bank account and the donor's name. In the pre-experiment years, potential donors received identical transaction forms.
Treatment group. Our design of the gift treatment closely follows Falk (2007). In 2014, individuals in this treatment received the solicitation letter together with an unconditional gift. The gift consisted of three envelopes paired with different folded cards. Further, we added one sentence to the solicitation letter, stating that the fundraiser „would like to provide the included folded cards as a gift.“ The total per-unit cost for mailing the control-group solicitation letter amounted to 0.43 euro (printing plus postage). In the gift treatment, the per-unit cost increased by 1.16 euro (postcards and envelopes: 0.47 euro; boxing and additional postage: 0.69 euro). Notably, our treatment consisted of a one-time intervention. From 2015 onward, all individuals in the sample received a solicitation letter very similar to the one distributed in the pre-experiment years.
Our setting serves as a suitable testing ground for machine-learning-based optimal targeting. First, as the fundraiser did not employ any targeting strategies before the experiment, the setting offers a clean environment to study our machine-learning approach's potential. Second, it provides rich data that not only allow us to estimate powerful targeting rules, but also enable us to test which type of data are especially beneficial for machine-learning-based optimal targeting (see the following description of the data). Third, because the fundraiser contacts all church members exhaustively, the setting offers the possibility to study the cold and warm lists separately. We, hence, can not only examine the optimal targeting of gifts among past donors, but also explore whom to target in the process of acquiring new donors. Fourth, because we were able to gather data for two postexperiment years, the setting allows us to study if the estimated optimal targeting rule increases total donations or simply pulls forward donations from 2015 to 2014. Fifth, because religious giving dominates the charitable giving landscape, targeting is particularly relevant in this context. Sixth, we can assess the estimated rule’s external validity by exploiting data from a follow-up experiment that took place in 2015 using different participants. Specifically, in 2015, we randomly allocate a new sample of 3,616 warm-list individuals to the control group and the gift treatment. Using the targeting rule estimated based on the 2014 sample, we assess whether the charity would benefit from applying the rule one year later to a different sample of individuals.