Experimental Design Details
EXPERIMENTAL DESIGN
Regular donors of the charitable organisation receive a donation upgrade request via postal mailing. All mailings are identical except from (1) the precise wording of the how information about how many others have increased their donations in the past already and (2) the choice set at the bottom of each letter. We implement six different treatments using a 3x2 factorial design.
Social Information:
Subjects in T2 & T3 and T5 & T6 receive information about how many other regular donors have upgraded their donation sizes in the past. Subjects in T1 and T4 receive the same mailing, but without any information about others’ behavior.
Static vs. dynamic norm:
Subjects in T3 and T6 receive information about the dynamic norm, namely how the number of regular donors who have upgraded their donation size in the past has developed over time. Subjects in T2 and T5 receive information about the static norm, namely how many other regular donors have already upgraded at a certain point in time.
Choice Set:
While half of the subjects (T4, T5, T6) receive a choice set with relatively low options (3, 5, 8) the other half of the subjects (T1, T2, T3) can choose between relative large numbers (8, 10, 13).
PLANNED ANALYSES
We analyze our experimental data by means of OLS regressions (accounting for the randomization procedure explained under VI):
• The main dependent variables are given by the key variables of interest
• The main independent variables are dummies for assignment to treatments
• To account for potential imbalances between treatment groups, we control for
o date of subjects’ first donation to the charity,
o regular donation size,
o city size,
o gender,
o whether subjects received an additional e-mail,
o and acquisition channel.
(For all non-binary control variables, we also consider log)
• All of these variables are also used for heterogeneity analyses in which interaction terms of each variable with each of the treatment dummies are coefficients of interest
• Regarding the analysis of overall treatment effects including long-term effects: Because we have multiple observations per subject, we cluster standard errors at the subject level.
MAIN HYPOTHESES
1. The effect of social information
Take as dependent variables (i) number of donations, (ii) average donation sizes, and (iii) total donations raised. Regress each of these on a Info dummy. We hypothesize that in all three cases the effects are positive and significant.
2. The effect of the dynamic norm
Take as dependent variables (i) number of donations, (ii) average donation sizes, (iii) total donations raised. Regress each of these on a dynamic norm dummy. We hypothesize that in all three cases the effects are positive and significant. We further hypothesize that the effects are larger in size compared to the effects under 1.
3. The effect of the choice set
Take as dependent variables (i) number of donations, (ii) average donation sizes, (iii) total donations raised. Regress each of these on the Choice Set high (Choice Set low) dummy interacted with a dummy for high donor (low donor). We hypothesize that in all three cases the effects are positive and significant.
******************************************************************************
SECOND SET OF EXPERIMENTS
General Information
- Mailing date: May 2nd letters sent out - currently (May 7th) collection in progress and no data received
- Total N = 52.367 in two additional experiments
- Focus: Effects of the Ask String
Experiment 1
N = 30.366 („Panel“) - only participants who took part in the previous experiment
o All participants received a low (8-5-3) or a high (13-10-8) Ask String in the previous experiment in 2021
o N in 2021 = 33.686 --> Attrition: 3.320
Treatments in Experiment 1 in 2022; again low vs high:
p1: 8-5-3
p2: 13-10-8
Experiment 2
- N: 22.000 („New“) - only participants who did not take part in the previous experiment
Treatments in Experiment 2 - Order of suggested numbers and location of a round number (10/15):
n1: 7-10-14
n2: 14-10-7
n3: 7-11-15
n4: 15-11-7
Online reminder via eMail in both experiments:
Online Ask String: 10-15-20
Default is 15
Online vs. offline upgrade traceable on individual level
Randomization method:
Experiment 1: random block assignment, stratified by 1) “Ask String last year” and 2) “Upgrade last year: yes/no”
p1: 15.184
p2: 15.182
Experiment 2: complete random assignment with 4 treatment arms
n1: 5.498
n2: 5.499
n3: 5.501
n4: 5.502
Main Hypotheses:
Panel:
1. High Choice Set (vs. low) - sign. higher amounts (also conditional on giving)
2. Upgrade 2021 (vs. no Upgrade 2021) - higher probability Upgrade 2022
3. Subsample: Small Choice Set 2021 + Upgrade; Small Choice Set 2022 (vs. high) - higher probability Upgrade 2022
4. Pooled 2021 and 2022 - Low/Low Choice Set (vs. Low/High, High/Low, and High/High) - higher upgrade incidence (and higher total sum)
New:
1. Descending Ask String (vs. ascending) - higher upgrades (cond. on giving), higher sums, but lower upgrade incidence
2. Compromise Effect weakened by “5er” (n3 & n4)
3. Steeper string (7-10-14 vs. 8-10-13) - increased compromise effect
Other hypotheses:
1. Subsample: High Choice Set 2022 - High previous donation - higher probability Upgrade 2022
2. Subsample: High Choice Set 2022 - Acquisition via F2F - lower probability and sums 2022
3. Online upgrades (vs. offline upgrades) - higher sums and more choices of the default