Experimental Design Details
Note: Most of the design elements can be seen in the included Qualtrics instruments. Key details are below.
##Substitution (and happiness)
[2018 clarification: These were administered to the Student respondents to the Omnibus after the first 600, and to *all* the Nonstudent respondents.]
Reinstein (2006) ran multi-stage lab experiments to measure and test how one appeal/ask (and direct donation responses) affects later and simultaneous donation choices. We extend this, focusing on a small set of treatments, and relying mainly on *between-subject* variation in an isolated decision, avoiding contrast and experimenter demand effects.
Our basic design involves a participant’s decisions, at one or two points in time (henceforth "phases"), to divide an endowment between herself and one or more charities. Between participants, we vary
1. Whether the participant is asked to make a donation from her earnings in both the first and the second phase (separated by days or weeks), or only in the second phase.
2. Whether the charities in each phase are (typically seen as) similar or very distinct.
- Charities were chosen based on prominence in the UK and the potential to easily divide into disjoint similarity classes. We excluded charities with very similar names.
- We confirmed the similarity classes in a separate survey (via Prolific Academic on 4 March 2017) of a demographically-similar group (N=104). Similarity was measured using both unincentivized and beauty-contest elicitation. See: similaritysurveymaterial.zip.
3. The time gap between the first and the second time the participant is *invited to* participate in each phase.
###Phase 1
The context and presentation can be seen in the instruments attached. These differ slightly: for Nonstudents this is paired with a reward for signing up for the ESSExLab pool; for students ... with a reward for completing the Omnibus.
A participant is randomized into one of three ask treatments:
1. No ask
2. Asked to donate to Oxfam
3. Asked to donate to the British Heart Foundation
Sample text:
> Before we explain how to claim your reward… We are giving you the opportunity to donate from your reward to Oxfam. For every pound you donate, we will add an extra 25p. Please click on the image below for further information about this charity (link will open in a new tab). From your £10 Amazon gift certificate WOULD you be willing to donate to Oxfam? If you donate, your donation will be automatically deducted from your reward and passed on to this charity, plus an additional 25% from our own funds. Donations will be made within 7 days and receipts will be kept at the ESSExLab office. We will not pass your personal information on to the charity. Please enter the amount you would like to donate, if anything, in the box below. (Enter a whole number between 0 and 10; you do not need to enter the £ sign.)
Notes: The respondent must enter *some* number (possibly 0). We are using this 'mandated decision' in all treatments (all of the treatments mentioned in this file) because we believe it is likely to increase the baseline incidence of giving, allowing for more powerful statistical measures of the impact of the treatments. We are only allowing integer responses to aid our administration; in previous similar trials non-integer responses are rare anyways.
**Happiness**: As a secondary treatment (both here and in the second phase), we ask each participant to rate their happiness on a seven-point Likert scale ranging from "Extremely unhappy" to "Extremely happy".
In each case this question follows the neighbor questions. However, for treatments 2-3 we vary whether this is asked before or after the donation screen, exactly balancing across treatments (i.e., administering this orthogonally to the charity ask treatments).
Those who make a donation in phase 1 are thanked (within the survey) for making this specific donation.
###Phase 2
The context can be seen in the survey/experiment instruments attached. Again, the contexts are slightly different between the Nonstudent and Student samples. For the Nonstudents this is paired with a reward for completing the Omnibus; for Students this is paired with a reward for completing an Employability survey.
Each participant is randomized into one of two charitable ask treatments. We balance this randomization by the phase-1 treatment, so that the empirical probability of being assigned to a phase-2 treatment is exactly equal for each phase-1 treatment. (This is done in Qualtrics by assigning an embedded data variable, and running a separate randomizer for each value of this variable.)
1. Asked to donate to Save the Children
2. Asked to donate to Cancer Research UK
Happiness: this treatment is administered as in Phase 1.
##Giving and probability:
###1/2 chance of winning
[2018 note: ordering changed -- earlier discussion]
These treatments will be administered to the 401-600th Nonstudent responders to the initial email inviting them to sign up to the ESSExLab pool. [2018: Note this number was not reached]
... and to the first 600 students who respond to the Omnibus.
These participants are told:
> If you complete this survey, you have a 50% chance of winning a £10 Amazon voucher. After you complete this survey, we will reveal whether you have won this prize and explain how to claim it.
[Nonstudents: If you complete this form and register as an ESSEXLab participant before the deadline specified in your email, you will have a 50% chance of winning a £10 Amazon gift certificate. ... ]
In this treatment, participants have an equal chance of any of the following.
1. 'Before ask', wins: Asked, before learning outcome, to donate to either Oxfam or BHF conditional on winning the prize. Wins the prize.
2. 'Before ask', loses: ... Does not win the prize.
3. 'After ask': Asked, after learning of winning £10, to donate to either Oxfam or BHF conditional on winning the prize
4. 'Loses, no ask' (this is self-explanatory)
Sample language ('Before ask'):
> Before we reveal if you have won the £10 Amazon voucher... We are giving you the opportunity to donate from your prize to one of two charities: either Oxfam or the British Heart Foundation. For every pound you donate, we will add an extra 25p. ...
> ... IF you win the £10 Amazon voucher, WOULD you be willing to donate to one of the above charities? This will not affect your chance of winning, as the prize winners have already been chosen through a random draw.
> ...Please enter the amount you would like to donate (if anything) if you win the prize, in the box below. (Enter a number between 0 and 10).
> [If chooses a positive amount this appears:] Please select the charity you would like to donate to, if you win [may tick either Oxfam or BHF].
###Ambiguous chance of winning
*Note*: we refer to these as 'ambiguous' because the student participants will not know in advance how many other participants there will be, and their chances of winning depend on the number of participants, as explained below.
These treatments will be administered to any students after the first 600 who respond to the Omnibus.
[2018: Note we revised to have the this number was never reached; we thus removed the details below]
***
Note that for each of the Giving and Probability treatments the happiness (Likert scale) question is asked *after* the donation request in Before treatments (or the information about not winning). We do not envision this being part of our main analysis for the 'does being asked to give affect happiness' questions, as the context is different.
##Requested Salary and Gender
#Experimental Design
We are interested in how the subjects' gender correlates with their answers in a vignette study. Participants are asked (by hypothetical interviewers for an Assurance Trainee position) to state a desired starting salary; next given industry salary information; and then asked again.. This vignette occurs at the *beginning* of the employability survey. The vignette asks respondents about how they would answer specific questions within an interview context. (For space reasons, we give this in a separate file)
***
30 Jul 2017 addition: see "Interventions (Hidden)" box
2018: Noting changed student randomisation ordering + stylistic edits to this form