|
Field
Abstract
|
Before
This is a field experiment in the context of a recruitment drive for the ESSExLab (University of Essex, Colchester). A sample of local participants, drawn from a marketing database, are being paid to join the ESSExLab database as participants, and again to take the Omnibus survey. A sample of already-registered ESSExLab participants (henceforth "Student subjects", although a small share of these are university staff and local residents) are also being paid to take the Omnibus, and again to take an Employability survey. Within these contexts we are running experiments into charitable giving following our previous research, and related research by other authors.
See in particular https://davidreinstein.wordpress.com/research-and-publications/
- "Giving and Probability" (Kellner, Reinstein, Riener); Giveifyouwin.org (popular summary)
- "Substitution Among Charitable Contributions: An Experimental Study.” (Reinstein)
The employability study contains another embedded experiment about gender-differences in perceptions of starting salaries.
Further details are given within the hidden section.
The motivation for the substitution experiment and the broad design are also presented in
"Full_proposal_adapting_LOI_SPI_Reinstein_et_al with timeline.pdf", our 2013 grant application, and again in our BA grant application "Experiments on Political Ideology for BA grant - word version.pdf".
---
[Registered later (but before these were implemented on 30 July - 1 August 2017): Further trials of "Giving and Probability" via the Prolific platform on UK nonstudents; compensating for low turnout of Essex nonstudents]
|
After
This is a field experiment in the context of a recruitment drive for the ESSExLab (University of Essex, Colchester). A sample of local participants, drawn from a marketing database, are being paid to join the ESSExLab database as participants, and again to take the Omnibus survey. A sample of already-registered ESSExLab participants (henceforth "Student subjects", although a small share of these are university staff and local residents) are also being paid to take the Omnibus, and again to take an Employability survey. Within these contexts we are running experiments into charitable giving following our previous research, and related research by other authors.
See in particular https://davidreinstein.wordpress.com/research-and-publications/
- "Giving and Probability" (Kellner, Reinstein, Riener); Giveifyouwin.org (popular summary)
- "Substitution Among Charitable Contributions: An Experimental Study.” (Reinstein)
The employability study contains another embedded experiment about gender-differences in perceptions of starting salaries.
Further details are given within the hidden section.
The motivation for the substitution experiment and the broad design are also presented in
"Full_proposal_adapting_LOI_SPI_Reinstein_et_al with timeline.pdf", our 2013 grant application, and again in our BA grant application
"Experiments on Political Ideology for BA grant - word version.pdf".
---
[Registered later (but before these were implemented on 30 July - 1 August 2017): Further trials of "Giving and Probability" via the Prolific platform on UK nonstudents; compensating for low turnout of Essex nonstudents]
|
|
Field
Last Published
|
Before
August 18, 2017 10:51 AM
|
After
January 04, 2018 01:49 PM
|
|
Field
Study Withdrawn
|
Before
|
After
No
|
|
Field
Intervention Completion Date
|
Before
|
After
August 01, 2017
|
|
Field
Data Collection Complete
|
Before
|
After
Yes
|
|
Field
Final Sample Size: Number of Clusters (Unit of Randomization)
|
Before
|
After
**Giving and Probability**
1. Omnibus, Giving and Probability component: 598 participants, 460 with giving decisions
2. Prolific, Giving and Probability component: 320 participants, 240 with giving decisions
**Substitution**
3. Nonstudents, signup for Essexlab, completed survey, first ask/nonask: 96
4. Nonstudents in above (3), completed first and second (Omnibus) survey: 76
5. Omnibus students in substitution treatments, completed (first ask/non-ask): 218
6. Students in above (5) in substitution treatments, also completed second ask (employability survey): 140
|
|
Field
Was attrition correlated with treatment status?
|
Before
|
After
No
|
|
Field
Final Sample Size: Total Number of Observations
|
Before
|
After
In initial tests for the substitution experiments (the only ones with a followup), the attrition appears unrelated to any treatments. However, this will be analyzed further as we write the paper.
|
|
Field
Final Sample Size (or Number of Clusters) by Treatment Arms
|
Before
|
After
**Giving and Probability**
1. Omnibus, Giving and Probability component: 460 with giving decisions - 155 After-win, 305 Before
2. Prolific, Giving and Probability component: 320 participants, 240 with giving decisions - 78 After-win, 162 Before
**Substitution**
3. Nonstudents, signup for Essexlab, completed survey, first ask/nonask: 96 total = 35 no ask + 28 Domestic ask (14 happy after+14 happy first) + 33 International ask (17 happy after+16 happy first).
4. Nonstudents in above (3), completed first and second (Omnibus) survey,
- by second round treatment: 76 = 38 International + 37 Domestic (+1 missing treatment variable)
- by first-round treatment: 76 = 28 No ask + 22 Domestic + 25 International .
5. Omnibus students in substitution treatments, completed (first ask/non-ask): 218
= 73 Domestic (36 happy after + 37 happy first) + 70 International (35+35) +75 no ask
6. Students in above (5) in substitution treatments, also completed second ask (employability survey): 140
- by second round treatment: 70 Domestic + 70 International
- by first-round treatment: = 49 No ask + 45 Domestic + 46 International .
|
|
Field
Is there a restricted access data set available on request?
|
Before
|
After
No
|
|
Field
Program Files
|
Before
|
After
No
|
|
Field
Data Collection Completion Date
|
Before
|
After
August 01, 2017
|
|
Field
Is data available for public use?
|
Before
|
After
No
|
|
Field
Intervention (Public)
|
Before
This is a field experiment in the context of a recruitment drive for the ESSExLab (University of Essex, Colchester). A sample of local participants, drawn from a marketing database, are being paid to join the ESSExLab database as participants, and again to take the Omnibus survey. A sample of students (current participants) are also being paid to take the Omnibus, and again to take an employability survey. Within these contexts we are running experiments into charitable giving following our previous research, and related research by other authors.
See in particular https://davidreinstein.wordpress.com/research-and-publications/
- "Giving and Probability" (Kellner, Reinstein, Riener); Giveifyouwin.org (popular summary)
- "Substitution Among Charitable Contributions: An Experimental Study.” (Reinstein)
The employability study contains another embedded experiment about gender-differences in perceptions of starting salaries.
Further details are given within the hidden section.
30 Jul 2017: Because of a low response rate to the 'nonstudent' part of the initially planned trial, we have added an additional set of trials for the Giving and Probability and happiness components, involving 320 additional participants recruited on Prolific Academic. This trial will be run on 30 Jul 2017 (except for 20 pilot observations). I am registering this component immediately in advance of running it (160 participants on 30 Jul 2017 and 160 more on 31 July or 1 August). The timing and all details of the trial (and my trial account) on the Prolific site and the Qualtrics site (which includes the time and geocode of each response) can be made available in case of a serious need to verify the integrity of this study and its data.
____
Followup 18 Aug 2017: Prolific run on 30 July (140 participants) and 1 August as planned (160 obs) as planned; plus 20 obs in "Pilot runs" on 29 July.
|
After
This is a field experiment in the context of a recruitment drive for the ESSExLab (University of Essex, Colchester). A sample of local participants, drawn from a marketing database, are being paid to join the ESSExLab database as participants, and again to take the Omnibus survey. A sample of students (current participants) are also being paid to take the Omnibus, and again to take an employability survey. Within these contexts we are running experiments into charitable giving following our previous research, and related research by other authors.
See in particular https://davidreinstein.wordpress.com/research-and-publications/
- "Giving and Probability" (Kellner, Reinstein, Riener); Giveifyouwin.org (popular summary)
- "Substitution Among Charitable Contributions: An Experimental Study.” (Reinstein)
The employability study contains another embedded experiment about gender-differences in perceptions of starting salaries.
Further details given in "hidden" section below.
30 Jul 2017: Because of a low response rate to the 'nonstudent' part of the initially planned trial, we added an additional set of trials for the Giving and Probability and happiness components, involving 320 additional participants recruited on Prolific Academic. This trial will be run on 30 Jul 2017 (except for 20 pilot observations). I am registering this component immediately in advance of running it (160 participants on 30 Jul 2017 and 160 more on 31 July or 1 August). The timing and all details of the trial (and my trial account) on the Prolific site and the Qualtrics site (which includes the time and geocode of each response) can be made available in case of a serious need to verify the integrity of this study and its data.
____
Followup 18 Aug 2017: Prolific run on 30 July (140 participants) and 1 August as planned (160 obs) as planned; plus 20 obs in "Pilot runs" on 29 July.
2018 note: Ordering of student treatments was changed because of low nonstudent response and a revised research priority, see details below.
|
|
Field
Primary Outcomes (End Points)
|
Before
(These variables pertain to each of the parts of the study)
- Charitable donations and commitments to donate; incidence, amounts, and expected values of donation amounts (where individuals make commitments in uncertain situations)
- Choice of charity (a very secondary concern)
- Reported happiness
- Hypothetical responses to scenarios involving starting salaries.
These measures and our proposed analysis are further described below in several sections.
|
After
(These variables pertain to each of the parts of the study)
- Charitable donations and commitments to donate; incidence, amounts, and expected values of donation amounts (where individuals make commitments in uncertain situations)
- Choice of charity (a very secondary concern)
- Reported happiness
- Hypothetical responses to scenarios involving starting salaries.
These measures and our proposed analysis are further described below in several sections.
2018: stylistic edits to this form
|
|
Field
Randomization Method
|
Before
The 2600 (of 5000) nonstudents were allocated via simple randomization by Pat Lown (assigning random numbers to each address via an Excel spreadsheet).
Of these, 2571 had unique emails; duplicates were dropped. Simple tests indicated approximate balance.
Next, David Reinstein randomized, within the 4 blocks 'child present x sex', into five groups for the 'batches of 514' mentioned below, using Stata-generated random numbers.
- See 'testrandomisedatahq.do'
Further randomization is done through Qualtrics survey flow randomizer, selecting 'evenly present elements' in each case.
We are blocking this further randomization to a limited extent. For the first 400 Nonstudent participants we are separately randomizing (into the 3x2 treatments in the Substitution experiment first phase) within 2 predetermined groups: those recorded as having children present, and those recorded as not having children present (from the marketing data). We chose this variable to block on as a brief analysis of our Prolific data on a similar population suggested households with children were more likely to contribute (a significant extensive-margin difference).
For the next 200 Nonstudents (in the Giving and Probability treatments) we block the randomization by gender.
We will block the randomisation for the Student respondents by gender, subject to some feasibility issues.
For the timing dimension (time between first and second ask), we are releasing invitations in batches of (approximately) 514 with 48 hour intervals in between.
As we have only 400 rewards to give away, and we cannot know the response rate in advance, the distribution of these timings is not fully controlled.
For example, we may get few responses each time, leading to a fairly even distribution of responses over the 15 days, or we may get many responses early on.
However, the assigned order we are *sending* the emails has been randomized, thus those receiving the invitation in each of the five batches (if we get to five) should have the same characteristics on average.
The requested salary and gender component does not involve randomization.
___
Note 18 Aug 2017: The Prolific trial used pure randomisation (no blocking/clustering/stratification) for convenienve on this platform.
|
After
The 2600 (of 5000) nonstudents were allocated via simple randomization by Pat Lown (assigning random numbers to each address via an Excel spreadsheet).
Of these, 2571 had unique emails; duplicates were dropped. Simple tests indicated approximate balance.
Next, David Reinstein randomized, within the 4 blocks 'child present x sex', into five groups for the 'batches of 514' mentioned below, using Stata-generated random numbers.
- See 'testrandomisedatahq.do'
Further randomization is done through Qualtrics survey flow randomizer, selecting 'evenly present elements' in each case.
We are blocking this further randomization to a limited extent. For the first 400 Nonstudent participants we are separately randomizing (into the 3x2 treatments in the Substitution experiment first phase) within 2 predetermined groups: those recorded as having children present, and those recorded as not having children present (from the marketing data). We chose this variable to block on as a brief analysis of our Prolific data on a similar population suggested households with children were more likely to contribute (a significant extensive-margin difference).
For the next 200 Nonstudents (in the Giving and Probability treatments) we block the randomization by gender.
[2018 Note: we didn't get this many respondents]
We will block the randomisation for the Student respondents by gender, subject to some feasibility issues.
For the timing dimension (time between first and second ask), we are releasing invitations in batches of (approximately) 514 with 48 hour intervals in between.
As we have only 400 rewards to give away, and we cannot know the response rate in advance, the distribution of these timings is not fully controlled.
For example, we may get few responses each time, leading to a fairly even distribution of responses over the 15 days, or we may get many responses early on.
However, the assigned order we are *sending* the emails has been randomized, thus those receiving the invitation in each of the five batches (if we get to five) should have the same characteristics on average.
The requested salary and gender component does not involve randomization.
___
Note 18 Aug 2017: The Prolific trial used pure randomisation (no blocking/clustering/stratification) for convenience on this platform.
2018: Stylistic edits to this form
|
|
Field
Planned Number of Observations
|
Before
1. Substitution experiment: A targeted and maximum sample of 400 Non-students, plus 200 Students, each observed at two separate points in time (with possible additional follow-ups)
2. Giving-in-probability:
a. 1/2 chance of winning
- 400 Students (targeted) for 1/2 chance of £10
- Up to 200 non-students (targeted) for 1/2 chance of £10
- plus possible reallocation from substitution experiment; this change will be made if there is 'extremely low giving' among the 400 Non-students the first time they are asked.
b. Ambiguous chance of winning
- Plus any remaining regular-sample (participants in the ESSExLab pool who do the Omnibus, up to roughly 2000, considering the size of the pool): 30/N chance of one of 30 £10 Amazon vouchers, 6/N chance of winning one of six £50 Amazon vouchers, or both (treatment variation discussed above); equal division between these treatments.
3. Happiness and giving: 750 participants (400 Non-students and 200 students earning certain £10); of which (up to) 600 observed at two separate points in time (with possible additional follow-ups)
4. Requested salary and gender: 200 participants (the 200 regular-sample participants listed in item 1)
***
30 Jul 2017 addition: 320 Prolific participants
|
After
1. Substitution experiment: A targeted and maximum sample of 400 Non-students, plus 200 Students, each observed at two separate points in time (with possible additional follow-ups)
2. Giving-in-probability:
a. 1/2 chance of winning
- 400 Students (targeted) for 1/2 chance of £10
[2018: Note revision to 600 students, because of low nonstudent turnout]
- Up to 200 non-students (targeted) for 1/2 chance of £10
[2018: None administered because of low turnout]
- plus possible reallocation from substitution experiment; this change will be made if there is 'extremely low giving' among the 400 Non-students the first time they are asked.
b. Ambiguous chance of winning
[2018: None administered because of low-ish turnout]
- Plus any remaining regular-sample (participants in the ESSExLab pool who do the Omnibus, up to roughly 2000, considering the size of the pool): 30/N chance of one of 30 £10 Amazon vouchers, 6/N chance of winning one of six £50 Amazon vouchers, or both (treatment variation discussed above); equal division between these treatments.
3. Happiness and giving: 750 participants (400 Non-students and 200 students earning certain £10); of which (up to) 600 observed at two separate points in time (with possible additional follow-ups)
4. Requested salary and gender: 200 participants (the 200 regular-sample participants listed in item 1)
***
30 Jul 2017 addition: 320 Prolific participants
2018: Noting changed student randomisation ordering, low turnout + stylistic edits to this form
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
We produce rough power calculations in the file "powercalcsforaearegistry.do" for the simplest t-test comparisons. These calculations rely on means and standard deviations from data from previous similar studies. However, those contexts differ somewhat from the present case. We summarize the computed MDE sizes below:
Note that in considering the reporting of null effects in our main analysis, we will rely on confidence intervals for estimated effects. If these are narrowly bounded around zero, we will interpret this as evidence of the lack of a substantial effect.
1. Substitution
- For comparisons of 'no ask' vs 'some previous ask', pooling students and nonstudents: +56% of base group
- If s.d.=mean (as in case of 'simple income' in previous laboratory experiments): MDE of 24% of base group
-Comparing 'previous ask for similar charity' vs 'previous ask for different charity'
- if sd=mean, MDE of 28% of base group
An alternative power analysis, based on a survey of the variety of previous papers, is given in the file "additional materials - Experiments on Political Ideology.pdf".
This analysis suggests that we should be able to detect an effect of half of a standard deviation with a sample size of at least 223 per group.
2. Giving and probability
- For basic 1/2 probability comparison, MDE=71% of mean for base group if we do not pool with other experiments
- If we pool with all previous experiments, MDE=40% of mean for base group
- For extensive margin comparison (no pooling), MDE=+44% of incidence rate
- For each comparison between each of ambiguous prizes (see above): MDE of +140% of base group (similar for incidence rate) if we get 200 per group
3. Happiness: the rough analysis suggests we should be able to detect an effect of roughly 1/3 of a Likert scale point, or 6.1% of the base rate.
4. Gender difference in scenario requesting starting salary:
For Mann-Whitney Tests:
-With 95% power, at a significance level of 5% assuming that 100 males and 100 females complete the survey, we can detect a moderate effect size (i.e. d=0.52).
-At a more standard 80% power, at a significance level of 5%, assuming 100 males and 100 females take part, we can detect a small to moderate effect size (d=0.41).
|
After
We produce rough power calculations in the file "powercalcsforaearegistry.do" for the simplest t-test comparisons. These calculations rely on means and standard deviations from data from previous similar studies. However, those contexts differ somewhat from the present case. We summarize the computed MDE sizes below:
Note that in considering the reporting of null effects in our main analysis, we will rely on confidence intervals for estimated effects. If these are narrowly bounded around zero, we will interpret this as evidence of the lack of a substantial effect.
1. Substitution
- For comparisons of 'no ask' vs 'some previous ask', pooling students and nonstudents: +56% of base group
- If s.d.=mean (as in case of 'simple income' in previous laboratory experiments): MDE of 24% of base group
-Comparing 'previous ask for similar charity' vs 'previous ask for different charity'
- if sd=mean, MDE of 28% of base group
An alternative power analysis, based on a survey of the variety of previous papers, is given in the file "additional materials - Experiments on Political Ideology.pdf".
This analysis suggests that we should be able to detect an effect of half of a standard deviation with a sample size of at least 223 per group.
2. Giving and probability
- For basic 1/2 probability comparison, MDE=71% of mean for base group if we do not pool with other experiments
- If we pool with all previous experiments, MDE=40% of mean for base group
- For extensive margin comparison (no pooling), MDE=+44% of incidence rate
- For each comparison between each of ambiguous prizes (see above): MDE of +140% of base group (similar for incidence rate) if we get 200 per group
3. Happiness: the rough analysis suggests we should be able to detect an effect of roughly 1/3 of a Likert scale point, or 6.1% of the base rate.
4. Gender difference in scenario requesting starting salary:
For Mann-Whitney Tests:
-With 95% power, at a significance level of 5% assuming that 100 males and 100 females complete the survey, we can detect a moderate effect size (i.e. d=0.52).
-At a more standard 80% power, at a significance level of 5%, assuming 100 males and 100 females take part, we can detect a small to moderate effect size (d=0.41).
|
|
Field
Intervention (Hidden)
|
Before
##Intervention (Hidden)
*All payments discussed below are in Amazon gift certificates/gift vouchers.*
The overall timeline and procedures for the ESSExLab recruitment drive, Omnibus, Employability survey, and associated experiments are outlined in the following files:
- 'ESSEXLab Recruitment Timeline.xlsx - Study Overview.pdf' ('Lown-Dietrich (LD) Study' is not discussed here)
- ESSEXLab Recruitment Timeline.xlsx - Study Logistics.pdf ('Lown-Dietrich (LD) Study' is not discussed here )
This has been saved on 24 Apr 2017; we may make some small changes for feasibility as the story progresses, but we will try to avoid this.
###Recruitment drive and promotion (Nonstudents)
The experimental variation takes places within the context of a recruitment drive for the ESSExLab (University of Essex, Colchester), focusing on non-students; and an Omnibus survey being administered to all registered participants, including recent recruits.
Note: This could itself be considered an 'intervention' in the real world, as this recruitment drive is not a part of typical day-to-day life in the local community. However recruitment for surveys and studies, as well as the receipt of letters and emails encouraging participation (including from the University of Essex) is not uncommon in this context.
ESSExLab purchased 5000 emails/addresses/demographics from DataHQ, a marketing firm. To garner the most useful for sample for later lab use, we specifically asked for non-students within reasonable commuting range from the University or the center of town, who had access to email, and oversampled those in relatively more diverse postcodes and those with neighbors in the sample.
- See file "23714_University_of_Essex_(Accounts_Payable).pdf" for complete specification of our data request from DataHQ.
On 'day 1' (targeted 4 May, 2017), we post 5000 letters to the addresses and names purchased from DataHQ inviting recipients to become part of the subject pool.
The text for all of these printed letters is identical.
for proposed email content.
Of these 5000 emails, 2600 addresses are randomly assigned to potentially be part of the treatments mentioned below ('DR1-study'). (This was done through pure rather than block randomization, i.e., with no clustering. Random numbers were generated in Excel by Pat Lown.)
Among the 2600 nonstudents in the DR-1 study, we randomize the order of these contact addresses. We will email (roughly) 514 addresses at a time with a personalised link to the specific Qualtrics survey, and a subject line "Recruiting for ESSEXLab experiments: *special £10 reward* for joining". Some email content is given below. Full proposed emails can be found in
> omnibus_plus_emailtexts.md
***
We recently sent you a letter in the post inviting you to join the [ESSEXLab](http://essexlab.essex.ac.uk/) participant database, which can also be found below. You have been randomly selected for a special reward for signing up - a £10 Amazon voucher!
Here’s what you need to do *within the next 48 hours* to do to claim your voucher:
1. If you have not already done so, sign up for the ESSEXLab participant database [HERE](http://essexlab.essex.ac.uk/hroot) using your recruitment ID code: [ID]. *Remember this ID!*
2. Complete the survey linked here [individualised link sent from Qualtrics] (when you begin the survey we will let you know whether you have responded in time to earn the £10 voucher).
3. Wait for us to email your **_£10 Amazon _****_voucher_**!
**We only have 400 vouchers to give out as part of this promotion - so act fast!**
[More information about participating in ESSExLab follows]
***
Each batch of 514 (approx) emails is sent with 48 hours delay. Once 400 rewards have been claimed, we change the email to mention the total amount of remaining rewards only. This changes over on day 16 even if fewer than 400 rewards have been claimed.
The next 200 Nonstudents who sign up get a different email, and have a 1/2 chance of winning a £10 voucher (see designs below). After this (or after day 36 if fewer than 200 sign up), the incentive ends, and the survey instrument shuts down (of course people can still sign up to be part of the ESSExLab pool).
Those who enter the survey in sufficient time are presented a series of questions. They are asked about their level of contact and willingness to exchange contact information with neighbors (this is relevant background for an unrelated study, and also should provide some distraction from the charity questions).
Of the first 400, 2/3 of these participants are also asked whether they would like to donate from their £10 rewards; details of these charity treatments, and the related happiness questions are given below.
*Payment details*: For the first 400 participants, we will make all payments on day 8 (for those participating in days 1-8) and on day 15 (for those participating on days 9-15) (subject to feasibility). For the remaining participants, we will make all payments within 1 week of completion. Participants are informed of this: 'Rewards will be sent and donations will be made within 7 days and receipts will be kept at the ESSExLab office.'
We will download the list of completed surveys and check this against ESSExLab signups (H-Root). We will enter the email addresses of qualifying participants into the Amazon gift-certificate page, along with the amounts they are owed (the £10 less any donations made). (This will be done at the University of Exeter; we are arranging funds and a university account that can be used).
Payments to charities will be made through the charities' own web sites. We will not be able to claim Gift Aid. Receipts for both donations and Amazon payments will be sent to the ESSExLab office for storage and to allow verification if participants request it.
###Omnibus (Students)
ESSExLab is asking all registered participants (subjects) to take an 'Omnibus' survey, to gain a series of background measures which experimenters will be able to use in their analyses and possibly in their participant selection. Omnibus responses will be connectable to responses in lab experiments, but careful measures will be taken to reasonably assure confidentiality and anonymity of both responses, and prevent experimenters from learning the identity of participants.
The Omnibus content (close to final; minus the charitable giving questions noted below) is given in
> omnibusdownload.pdf
This survey will be built and run via the Qualtrics platform. Some small changes to the content may be made, particularly including additional unincentivized measures of beliefs and preferences relevant to economists.
This will begin on Day 16, 15 days after the first emails from Phase 1.
All registered ESSExLab participants, including new recruits, will be invited and eligible to complete the Omnibus survey. These invitations will all go out at the same time, but respondents will be randomized into different incentive treatments, as detailed below. For the first 600 student respondents, we randomize 200 into treatments involving a certain £10 reward, and 400 into treatments involving a 50% chance of a £10 reward.
###Omnibus (Nonstudents)
Nonstudents who participated in phase 1 and who claimed the £10 rewards will be invited with a specific email offering them an additional £10 reward for completing the Omnibus. These participants will be given a second charitable ask.
(Other nonstudent participants will be given a 'vanilla' invitation without a reward.)
###Employability study
On day 30, students who completed the omnibus and were part of the certain-£10 treatment will be invited to take an Employability survey, and offered a £10 reward for completion. These participants will be given a second charitable ask.
They will also be presented a series of questions, and asked to do several tasks involving looking up jobs they might be interested in on a job web site.
They are also asked the questions mentioned below under the experimental design "Requested Salary and Gender".
***
Added 30 Jul 2017: Because of a low response rate to the 'nonstudent' part of the initially planned trial, we have added an additional set of trials for the Giving and Probability and happiness components, involving 320 additional participants recruited on Prolific Academic. This trial will be run on 30 Jul 2017 (except for 20 pilot observations). I am registering this component immediately in advance of running it.
- It is advertised as “Employment choices (basic payment plus bonus opportunities)"
- Base pay is £1 for a study advertised to take about 10 minutes
Screeners:
- UK resident, nonstudent, English first language, age 18+, not in my previous studies,
- Gave answer to earlier qns on charitable giving, charitable affiliation, gender, religion
Further basic description in order (Qualtrics docs attached)
- Announcement the study is coming from Exeter researchers, non-deception rules, consent form.
- Announce the prize/probability they are eligible for (see below)
- A vignette, questions on requested salary (as presented to students see above)
- Happiness question
- Verbal crystalised intelligence questions (as in omnibus)
- Prize/donation treatments (see below)
- Risk-preference and trust elicitation q's
- Happiness q again.
Prize/donation treatments, randomization using Qualtrics randomiser, 'evenly present elements,' between:
1. Before ask, 50% chance of winning (even randomizer btwn win/lose): 80 participants (close to above 'Before' treatment)
2. After ask, 50% chance of winning (even randomizer btwn win/lose): 160 participants (80 donation choices); (close to above 'After' treatment)
3. Before ask, 10% chance of winning (1 in 10 even randomizer between win/lose): 80 participants; as in the above Before treatment, but with only a 1/10 chance of winning
4. (New) Those in Before treatments who won allowed to revise charity decision
___
Added 18 Aug 2017:
Preview links given
|
After
##Intervention (Hidden)
*All payments discussed below are in Amazon gift certificates/gift vouchers.*
The overall timeline and procedures for ESSExLab recruitment drive, Omnibus, Employability survey, and associated experiments are outlined in the following files:
- 'ESSEXLab Recruitment Timeline.xlsx - Study Overview.pdf' ('Lown-Dietrich (LD) Study' is not discussed here)
- ESSEXLab Recruitment Timeline.xlsx - Study Logistics.pdf ('Lown-Dietrich (LD) Study' is not discussed here )
This has been saved on 24 Apr 2017; we may make some small changes for feasibility as the story progresses, but we will try to avoid this.
###Recruitment drive and promotion (Nonstudents)
The experimental variation takes places within the context of a recruitment drive for the ESSExLab (University of Essex, Colchester), focusing on non-students; and an Omnibus survey being administered to all registered participants, including recent recruits.
Note: This could itself be considered an 'intervention' in the real world, as this recruitment drive is not a part of typical day-to-day life in the local community. However recruitment for surveys and studies, as well as the receipt of letters and emails encouraging participation (including from the University of Essex) is not uncommon in this context.
ESSExLab purchased 5000 emails/addresses/demographics from DataHQ, a marketing firm. To garner the most useful for sample for later lab use, we specifically asked for non-students within reasonable commuting range from the University or the center of town, who had access to email, and oversampled those in relatively more diverse postcodes and those with neighbors in the sample.
- See file "23714_University_of_Essex_(Accounts_Payable).pdf" for complete specification of our data request from DataHQ.
On 'day 1' (targeted 4 May, 2017), we post 5000 letters to the addresses and names purchased from DataHQ inviting recipients to become part of the subject pool.
The text for all of these printed letters is identical.
Of these 5000 emails, 2600 addresses are randomly assigned to potentially be part of the treatments mentioned below ('DR1-study'). (This was done through pure rather than block randomization, i.e., with no clustering. Random numbers were generated in Excel by Pat Lown.)
Among the 2600 nonstudents in the DR-1 study, we randomize the order of these contact addresses. We will email (roughly) 514 addresses at a time with a personalised link to the specific Qualtrics survey, and a subject line "Recruiting for ESSEXLab experiments: *special £10 reward* for joining". Some email content is given below.
Full proposed emails found in
> omnibus_plus_emailtexts.md
***
We recently sent you a letter in the post inviting you to join the [ESSEXLab](http://essexlab.essex.ac.uk/) participant database, which can also be found below. You have been randomly selected for a special reward for signing up - a £10 Amazon voucher!
Here’s what you need to do *within the next 48 hours* to do to claim your voucher:
1. If you have not already done so, sign up for the ESSEXLab participant database [HERE](http://essexlab.essex.ac.uk/hroot) using your recruitment ID code: [ID]. *Remember this ID!*
2. Complete the survey linked here [individualised link sent from Qualtrics] (when you begin the survey we will let you know whether you have responded in time to earn the £10 voucher).
3. Wait for us to email your **_£10 Amazon _****_voucher_**!
**We only have 400 vouchers to give out as part of this promotion - so act fast!**
[More information about participating in ESSExLab follows]
***
Each batch of 514 (approx) emails sent with 48 hours delay. Once 400 rewards have been claimed, we change the email to mention the total amount of remaining rewards only. This changes over on day 16 even if fewer than 400 rewards have been claimed.
The next 200 Nonstudents who sign up get a different email, and have a 1/2 chance of winning a £10 voucher (see designs below). After this (or after day 36 if fewer than 200 sign up), the incentive ends, and the survey instrument shuts down (people can still sign up to be part of the ESSExLab pool).
Those who enter the survey in sufficient time are presented a series of questions. They are asked about their level of contact and willingness to exchange contact information with neighbors (relevant background for an unrelated study, also provides some distraction from the charity questions).
Of the first 400 participants, 2/3 also asked whether they would like to donate from their £10 rewards; details of these charity treatments, and the related happiness questions are given below.
*Payment details*: For the first 400 participants, we will make all payments on day 8 (for those participating in days 1-8) and on day 15 (for those participating on days 9-15) (subject to feasibility). For the remaining participants, we will make all payments within 1 week of completion. Participants are informed of this: 'Rewards will be sent and donations will be made within 7 days and receipts will be kept at the ESSExLab office.'
We will download the list of completed surveys and check this against ESSExLab signups (H-Root). We will enter the email addresses of qualifying participants into the Amazon gift-certificate page, along with the amounts they are owed (the £10 less any donations made). (This will be done at the University of Exeter; we are arranging funds and a university account that can be used).
Payments to charities will be made through the charities' own web sites. We will not be able to claim Gift Aid. Receipts for both donations and Amazon payments will be sent to the ESSExLab office for storage and to allow verification if participants request it.
###Omnibus (Students)
ESSExLab is asking all registered participants (subjects) to take an 'Omnibus' survey, to gain a series of background measures which experimenters will be able to use in their analyses and possibly in their participant selection. Omnibus responses will be connectable to responses in lab experiments, but careful measures will be taken to reasonably assure confidentiality and anonymity of both responses, and prevent experimenters from learning the identity of participants.
The Omnibus content (close to final; minus the charitable giving questions noted below) is given in
> omnibusdownload.pdf
This survey will be built and run via the Qualtrics platform. Some small changes to the content may be made, particularly including additional unincentivized measures of beliefs and preferences relevant to economists.
This will begin on Day 16, 15 days after the first emails from Phase 1.
All registered ESSExLab participants, including new recruits, will be invited and eligible to complete the Omnibus survey. These invitations will all go out at the same time, but respondents will be randomized into different incentive treatments, as detailed below.
[2018 note: Revised initial plan: First 600 student respondents, randomized into treatments with a 50% chance of a £10 reward. Remaining responses with certain £10 reward]
###Omnibus (Nonstudents)
Nonstudents who participated in phase 1 and who claimed the £10 rewards will be invited with a specific email offering them an additional £10 reward for completing the Omnibus. These participants will be given a second charitable ask.
(Other nonstudent participants given a 'vanilla' invitation without a reward.)
###Employability study
On day 30, students who completed the omnibus and were part of the certain-£10 treatment will be invited to take an Employability survey, and offered a £10 reward for completion. These participants will be given a second charitable ask.
They will also be presented a series of questions, and asked to do several tasks involving looking up jobs they might be interested in on a job web site.
They are also asked the questions mentioned below under the experimental design "Requested Salary and Gender".
***
Added 30 Jul 2017: Because of a low response rate to the 'nonstudent' part of the initially planned trial, we added an additional set of trials for the Giving and Probability & happiness components, involving 320 additional participants recruited on Prolific Academic. This trial will be run on 30 Jul 2017 (except for 20 pilot observations). I am registering this component immediately in advance of running it.
- It is advertised as “Employment choices (basic payment plus bonus opportunities)"
- Base pay is £1 for a study advertised to take about 10 min.
Screeners:
- UK resident, nonstudent, English 1st language, age 18+, not in my previous studies,
- Gave answer to earlier qns on charitable giving, charitable affiliation, gender, religion
Further basic description in order (Qualtrics docs attached)
- Announcement study is coming from Exeter researchers, non-deception rules, consent form.
- Announce the prize/probability they are eligible for (see below)
- A vignette, questions on requested salary (as presented to students see above)
- Happiness question
- Verbal crystalised intelligence questions (as in omnibus)
- Prize/donation treatments (see below)
- Risk-preference and trust elicitation q's
- Happiness q again.
Prize/donation treatments, randomization using Qualtrics randomiser, 'evenly present elements,' between:
1. Before ask, 50% chance of winning (even randomizer btwn win/lose): 80 participants (close to above 'Before' treatment)
2. After ask, 50% chance of winning (even randomizer between win/lose): 160 participants (80 donation choices); (close to above 'After' treatment)
3. Before ask, 10% chance of winning (1 in 10 even randomizer between win/lose): 80 participants; as in the above Before treatment, but with only a 1/10 chance of winning
4. (New) Those in Before treatments who won allowed to revise charity decision
___
Added 18 Aug 2017: Preview links given
2018: Noting changed student randomisation ordering + stylistic edits to this form
|
|
Field
Public analysis plan
|
Before
No
|
After
Yes
|