Back to History Current Version

Charitable giving trials linked to ESSExLab recruiting and omnibus: i. Crowding out (does one contribution/appeal come at the expense of another?) ii. Do people commit more before a win (conditionally on winning) or after a win? iii. Does the opportunity to donate boost reported happiness (and vice/versa). Associated instrument: Beliefs and attitudes towards starting salaries. [Registered later (but before 30 July implementation): Further trials via the Prolific platform (30 July, 1 August 2017)]

Last registered on August 18, 2017

Pre-Trial

Trial Information

General Information

Title
Charitable giving trials linked to ESSExLab recruiting and omnibus: i. Crowding out (does one contribution/appeal come at the expense of another?) ii. Do people commit more before a win (conditionally on winning) or after a win? iii. Does the opportunity to donate boost reported happiness (and vice/versa). Associated instrument: Beliefs and attitudes towards starting salaries. [Registered later (but before 30 July implementation): Further trials via the Prolific platform (30 July, 1 August 2017)]
RCT ID
AEARCTR-0002180
Initial registration date
May 05, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 05, 2017, 7:34 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
August 18, 2017, 6:06 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
University of Exeter

Other Primary Investigator(s)

PI Affiliation
University of Southampton
PI Affiliation
University of Dusseldorf

Additional Trial Information

Status
On going
Start date
2017-05-04
End date
2017-09-01
Secondary IDs
SC-07480 (British Academy:
Abstract
This is a field experiment in the context of a recruitment drive for the ESSExLab (University of Essex, Colchester). A sample of local participants, drawn from a marketing database, are being paid to join the ESSExLab database as participants, and again to take the Omnibus survey. A sample of already-registered ESSExLab participants (henceforth "Student subjects", although a small share of these are university staff and local residents) are also being paid to take the Omnibus, and again to take an Employability survey. Within these contexts we are running experiments into charitable giving following our previous research, and related research by other authors.

See in particular https://davidreinstein.wordpress.com/research-and-publications/

- "Giving and Probability" (Kellner, Reinstein, Riener); Giveifyouwin.org (popular summary)
- "Substitution Among Charitable Contributions: An Experimental Study.” (Reinstein)

The employability study contains another embedded experiment about gender-differences in perceptions of starting salaries.


Further details are given within the hidden section.

The motivation for the substitution experiment and the broad design are also presented in
"Full_proposal_adapting_LOI_SPI_Reinstein_et_al with timeline.pdf", our 2013 grant application, and again in our BA grant application "Experiments on Political Ideology for BA grant - word version.pdf".

---

[Registered later (but before these were implemented on 30 July - 1 August 2017): Further trials of "Giving and Probability" via the Prolific platform on UK nonstudents; compensating for low turnout of Essex nonstudents]


Registration Citation

Citation
Kellner, Christian, David Reinstein and Gerhard Riener. 2017. "Charitable giving trials linked to ESSExLab recruiting and omnibus: i. Crowding out (does one contribution/appeal come at the expense of another?) ii. Do people commit more before a win (conditionally on winning) or after a win? iii. Does the opportunity to donate boost reported happiness (and vice/versa). Associated instrument: Beliefs and attitudes towards starting salaries. [Registered later (but before 30 July implementation): Further trials via the Prolific platform (30 July, 1 August 2017)]." AEA RCT Registry. August 18. https://doi.org/10.1257/rct.2180-4.0
Former Citation
Kellner, Christian, David Reinstein and Gerhard Riener. 2017. "Charitable giving trials linked to ESSExLab recruiting and omnibus: i. Crowding out (does one contribution/appeal come at the expense of another?) ii. Do people commit more before a win (conditionally on winning) or after a win? iii. Does the opportunity to donate boost reported happiness (and vice/versa). Associated instrument: Beliefs and attitudes towards starting salaries. [Registered later (but before 30 July implementation): Further trials via the Prolific platform (30 July, 1 August 2017)]." AEA RCT Registry. August 18. https://www.socialscienceregistry.org/trials/2180/history/20550
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This is a field experiment in the context of a recruitment drive for the ESSExLab (University of Essex, Colchester). A sample of local participants, drawn from a marketing database, are being paid to join the ESSExLab database as participants, and again to take the Omnibus survey. A sample of students (current participants) are also being paid to take the Omnibus, and again to take an employability survey. Within these contexts we are running experiments into charitable giving following our previous research, and related research by other authors.

See in particular https://davidreinstein.wordpress.com/research-and-publications/

- "Giving and Probability" (Kellner, Reinstein, Riener); Giveifyouwin.org (popular summary)
- "Substitution Among Charitable Contributions: An Experimental Study.” (Reinstein)

The employability study contains another embedded experiment about gender-differences in perceptions of starting salaries.

Further details are given within the hidden section.

30 Jul 2017: Because of a low response rate to the 'nonstudent' part of the initially planned trial, we have added an additional set of trials for the Giving and Probability and happiness components, involving 320 additional participants recruited on Prolific Academic. This trial will be run on 30 Jul 2017 (except for 20 pilot observations). I am registering this component immediately in advance of running it (160 participants on 30 Jul 2017 and 160 more on 31 July or 1 August). The timing and all details of the trial (and my trial account) on the Prolific site and the Qualtrics site (which includes the time and geocode of each response) can be made available in case of a serious need to verify the integrity of this study and its data.

____

Followup 18 Aug 2017: Prolific run on 30 July (140 participants) and 1 August as planned (160 obs) as planned; plus 20 obs in "Pilot runs" on 29 July.
Intervention Start Date
2017-05-04
Intervention End Date
2017-08-01

Primary Outcomes

Primary Outcomes (end points)
(These variables pertain to each of the parts of the study)

- Charitable donations and commitments to donate; incidence, amounts, and expected values of donation amounts (where individuals make commitments in uncertain situations)

- Choice of charity (a very secondary concern)

- Reported happiness

- Hypothetical responses to scenarios involving starting salaries.

These measures and our proposed analysis are further described below in several sections.
Primary Outcomes (explanation)
These measures and our proposed analysis are further described below in several sections.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The parts of our design which we would like to reveal at this point have been described above under 'Intervention (Public)'.
Experimental Design Details

Note: Most of the design elements can be seen in the included Qualtrics instruments. Key details are below.


##Substitution (and happiness)


Reinstein (2006) ran multi-stage lab experiments to measure and test how one appeal/ask (and direct donation responses) affects later and simultaneous donation choices. We extend this, focusing on a small set of treatments, and relying mainly on *between-subject* variation in an isolated decision, avoiding contrast and experimenter demand effects.


Our basic design involves a participant’s decisions, at one or two points in time (henceforth "phases"), to divide an endowment between herself and one or more charities. Between participants, we vary


1. Whether the participant is asked to make a donation from her earnings in both the first and the second phase (separated by days or weeks), or only in the second phase.

2. Whether the charities in each phase are (typically seen as) similar or very distinct.

- Charities were chosen based on prominence in the UK and the potential to easily divide into disjoint similarity classes. We excluded charities with very similar names.

- We confirmed the similarity classes in a separate survey (via Prolific Academic on 4 March 2017) of a demographically-similar group (N=104). Similarity was measured using both unincentivized and beauty-contest elicitation. See: similaritysurveymaterial.zip.

3. The time gap between the first and the second time the participant is *invited to* participate in each phase.

###Phase 1

The context and presentation can be seen in the instruments attached. These differ slightly: for Nonstudents this is paired with a reward for signing up for the ESSExLab pool; for students ... with a reward for completing the Omnibus.

A participant is randomized into one of three ask treatments:

1. No ask
2. Asked to donate to Oxfam
3. Asked to donate to the British Heart Foundation

Sample text:

> Before we explain how to claim your reward… We are giving you the opportunity to donate from your reward to Oxfam. For every pound you donate, we will add an extra 25p. Please click on the image below for further information about this charity (link will open in a new tab). From your £10 Amazon gift certificate WOULD you be willing to donate to Oxfam? If you donate, your donation will be automatically deducted from your reward and passed on to this charity, plus an additional 25% from our own funds. Donations will be made within 7 days and receipts will be kept at the ESSExLab office. We will not pass your personal information on to the charity. Please enter the amount you would like to donate, if anything, in the box below. (Enter a whole number between 0 and 10; you do not need to enter the £ sign.)

Notes: The respondent must enter *some* number (possibly 0). We are using this 'mandated decision' in all treatments (all of the treatments mentioned in this file) because we believe it is likely to increase the baseline incidence of giving, allowing for more powerful statistical measures of the impact of the treatments. We are only allowing integer responses to aid our administration; in previous similar trials non-integer responses are rare anyways.

**Happiness**: As a secondary treatment (both here and in the second phase), we ask each participant to rate their happiness on a seven-point Likert scale ranging from "Extremely unhappy" to "Extremely happy".

In each case this question follows the neighbor questions. However, for treatments 2-3 we vary whether this is asked before or after the donation screen, exactly balancing across treatments (i.e., administering this orthogonally to the charity ask treatments).

Those who make a donation in phase 1 are thanked (within the survey) for making this specific donation.

###Phase 2

The context can be seen in the survey/experiment instruments attached. Again, the contexts are slightly different between the Nonstudent and Student samples. For the Nonstudents this is paired with a reward for completing the Omnibus; for Students this is paired with a reward for completing an Employability survey.

Each participant is randomized into one of two charitable ask treatments. We balance this randomization by the phase-1 treatment, so that the empirical probability of being assigned to a phase-2 treatment is exactly equal for each phase-1 treatment. (This is done in Qualtrics by assigning an embedded data variable, and running a separate randomizer for each value of this variable.)

1. Asked to donate to Save the Children
2. Asked to donate to Cancer Research UK

Happiness: this treatment is administered as in Phase 1.

##Giving and probability:

###1/2 chance of winning

These treatments will be administered to the 401-600th Nonstudent responders to the initial email inviting them to sign up to the ESSExLab pool.
... and to 2/3 of the first 600 students who respond to the Omnibus.

These participants are told:

> If you complete this survey, you have a 50% chance of winning a £10 Amazon voucher. After you complete this survey, we will reveal whether you have won this prize and explain how to claim it.

[Nonstudents: If you complete this form and register as an ESSEXLab participant before the deadline specified in your email, you will have a 50% chance of winning a £10 Amazon gift certificate. ... ]

In this treatment, participants have an equal chance of any of the following.

1. 'Before ask', wins: Asked, before learning outcome, to donate to either Oxfam or BHF conditional on winning the prize. Wins the prize.
2. 'Before ask', loses: ... Does not win the prize.
3. 'After ask': Asked, after learning of winning £10, to donate to either Oxfam or BHF conditional on winning the prize
4. 'Loses, no ask' (this is self-explanatory)

Sample language ('Before ask'):

> Before we reveal if you have won the £10 Amazon voucher... We are giving you the opportunity to donate from your prize to one of two charities: either Oxfam or the British Heart Foundation. For every pound you donate, we will add an extra 25p. ...

> ... IF you win the £10 Amazon voucher, WOULD you be willing to donate to one of the above charities? This will not affect your chance of winning, as the prize winners have already been chosen through a random draw.

> ...Please enter the amount you would like to donate (if anything) if you win the prize, in the box below. (Enter a number between 0 and 10). 


> [If chooses a positive amount this appears:] Please select the charity you would like to donate to, if you win [may tick either Oxfam or BHF].


###Ambiguous chance of winning

*Note*: we refer to these as 'ambiguous' because the student participants will not know in advance how many other participants there will be, and their chances of winning depend on the number of participants, as explained below.

These treatments will be administered to any students after the first 600 who respond to the Omnibus.

In this context, participants have an equal chance of any of the following.


1. (£10 prize/before ask): Told at the beginning "If you complete this survey, you have a chance of winning: One of thirty remaining £10 Amazon vouchers." [Asked later] "IF you win the £10 Amazon voucher, WOULD you be willing to donate to one of the above charities?..."


2. (£10 prize/before ask): Told at the beginning "If you complete this survey, you have a chance of winning: One of six remaining £50 Amazon vouchers." [Asked later] "IF you win the £50 Amazon voucher, WOULD you be willing to donate to one of the above charities?..."


3. (£10 and £50 prize/before ask): Told at the beginning "If you complete this survey, you have a chance of winning: One of thirty remaining £10 Amazon vouchers, and a chance of winning one of six remaining £50 Amazon vouchers." [Asked later] "IF you win the £10 Amazon voucher or the £50 Amazon voucher, WOULD you be willing to donate to one of the above charities?" Asked separately (given two boxes) to make a donation choice for the possible £10 and for the possible £50 prizes. (They must choose the same charity for either, but they can enter a different amount, including zero, for either case).


The description and depiction of the charities, how the donation will be made, and the 25% match rate are the same as in the '1/2 chance of winning' treatments.


For each of these three treatments the description at the beginning includes the language "Prizes will be chosen and emailed to the winners at the end of this Omnibus Survey promotion. Winners will be determined by random draw among all the participants in this part who completed the omnibus survey."

***

Note that for each of the Giving and Probability treatments the happiness (Likert scale) question is asked *after* the donation request in Before treatments (or the information about not winning). We do not envision this being part of our main analysis for the 'does being asked to give affect happiness' questions, as the context is different.

##Requested Salary and Gender


#Experimental Design

We are interested in how the subjects' gender correlates with their answers in a vignette study. Participants are asked (by hypothetical interviewers for an Assurance Trainee position) to state a desired starting salary; next given industry salary information; and then asked again.. This vignette occurs at the *beginning* of the employability survey. The vignette asks respondents about how they would answer specific questions within an interview context. (For space reasons, we give this in a separate file)

***

30 Jul 2017 addition: see "Interventions (Hidden)" box
Randomization Method
The 2600 (of 5000) nonstudents were allocated via simple randomization by Pat Lown (assigning random numbers to each address via an Excel spreadsheet).

Of these, 2571 had unique emails; duplicates were dropped. Simple tests indicated approximate balance.

Next, David Reinstein randomized, within the 4 blocks 'child present x sex', into five groups for the 'batches of 514' mentioned below, using Stata-generated random numbers.

- See 'testrandomisedatahq.do'

Further randomization is done through Qualtrics survey flow randomizer, selecting 'evenly present elements' in each case.

We are blocking this further randomization to a limited extent. For the first 400 Nonstudent participants we are separately randomizing (into the 3x2 treatments in the Substitution experiment first phase) within 2 predetermined groups: those recorded as having children present, and those recorded as not having children present (from the marketing data). We chose this variable to block on as a brief analysis of our Prolific data on a similar population suggested households with children were more likely to contribute (a significant extensive-margin difference).

For the next 200 Nonstudents (in the Giving and Probability treatments) we block the randomization by gender.

We will block the randomisation for the Student respondents by gender, subject to some feasibility issues.

For the timing dimension (time between first and second ask), we are releasing invitations in batches of (approximately) 514 with 48 hour intervals in between.
As we have only 400 rewards to give away, and we cannot know the response rate in advance, the distribution of these timings is not fully controlled.

For example, we may get few responses each time, leading to a fairly even distribution of responses over the 15 days, or we may get many responses early on.

However, the assigned order we are *sending* the emails has been randomized, thus those receiving the invitation in each of the five batches (if we get to five) should have the same characteristics on average.

The requested salary and gender component does not involve randomization.

___
Note 18 Aug 2017: The Prolific trial used pure randomisation (no blocking/clustering/stratification) for convenienve on this platform.
Randomization Unit
We randomize essentially at the individual level, blocking the randomization as explained above.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
NA
Sample size: planned number of observations
1. Substitution experiment: A targeted and maximum sample of 400 Non-students, plus 200 Students, each observed at two separate points in time (with possible additional follow-ups) 2. Giving-in-probability: a. 1/2 chance of winning - 400 Students (targeted) for 1/2 chance of £10 - Up to 200 non-students (targeted) for 1/2 chance of £10 - plus possible reallocation from substitution experiment; this change will be made if there is 'extremely low giving' among the 400 Non-students the first time they are asked. b. Ambiguous chance of winning - Plus any remaining regular-sample (participants in the ESSExLab pool who do the Omnibus, up to roughly 2000, considering the size of the pool): 30/N chance of one of 30 £10 Amazon vouchers, 6/N chance of winning one of six £50 Amazon vouchers, or both (treatment variation discussed above); equal division between these treatments. 3. Happiness and giving: 750 participants (400 Non-students and 200 students earning certain £10); of which (up to) 600 observed at two separate points in time (with possible additional follow-ups) 4. Requested salary and gender: 200 participants (the 200 regular-sample participants listed in item 1) *** 30 Jul 2017 addition: 320 Prolific participants
Sample size (or number of clusters) by treatment arms
NA
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We produce rough power calculations in the file "powercalcsforaearegistry.do" for the simplest t-test comparisons. These calculations rely on means and standard deviations from data from previous similar studies. However, those contexts differ somewhat from the present case. We summarize the computed MDE sizes below: Note that in considering the reporting of null effects in our main analysis, we will rely on confidence intervals for estimated effects. If these are narrowly bounded around zero, we will interpret this as evidence of the lack of a substantial effect. 1. Substitution - For comparisons of 'no ask' vs 'some previous ask', pooling students and nonstudents: +56% of base group - If s.d.=mean (as in case of 'simple income' in previous laboratory experiments): MDE of 24% of base group -Comparing 'previous ask for similar charity' vs 'previous ask for different charity' - if sd=mean, MDE of 28% of base group An alternative power analysis, based on a survey of the variety of previous papers, is given in the file "additional materials - Experiments on Political Ideology.pdf". This analysis suggests that we should be able to detect an effect of half of a standard deviation with a sample size of at least 223 per group. 2. Giving and probability - For basic 1/2 probability comparison, MDE=71% of mean for base group if we do not pool with other experiments - If we pool with all previous experiments, MDE=40% of mean for base group - For extensive margin comparison (no pooling), MDE=+44% of incidence rate - For each comparison between each of ambiguous prizes (see above): MDE of +140% of base group (similar for incidence rate) if we get 200 per group 3. Happiness: the rough analysis suggests we should be able to detect an effect of roughly 1/3 of a Likert scale point, or 6.1% of the base rate. 4. Gender difference in scenario requesting starting salary: For Mann-Whitney Tests: -With 95% power, at a significance level of 5% assuming that 100 males and 100 females complete the survey, we can detect a moderate effect size (i.e. d=0.52). -At a more standard 80% power, at a significance level of 5%, assuming 100 males and 100 females take part, we can detect a small to moderate effect size (d=0.41).
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Adrian Bailey, Research Ethics Officer, University of Exeter Business School
IRB Approval Date
2016-07-21
IRB Approval Number
Not numbered, but a record is stored on the "...staff research form"
Analysis Plan

Analysis Plan Documents

analysis plan for additional (30 Jul 2017) Prolific run

MD5: 913e0cd4e4e1d9bb89d3444326c260e1

SHA1: ca57922bcb775cfee2a85c912362b85f431598d8

Uploaded At: July 30, 2017

Slightly updated analysis plan (added weather instrument for mood discussion)

MD5: 0f0617347b2003433a81c8cab130e242

SHA1: e0f33047f5435be47e167ee3ba9c492bb64cfbba

Uploaded At: May 08, 2017

analysis plan

MD5: b4e42b02d212d5bf6ccb984de6cfe185

SHA1: 1a7c63bc4370144471ae419e4516dd8a7398e493

Uploaded At: May 05, 2017

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials