x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
Legitimacy And Social Desirability: MTurk Online Survey
Last registered on May 18, 2019

Pre-Trial

Trial Information
General Information
Title
Legitimacy And Social Desirability: MTurk Online Survey
RCT ID
AEARCTR-0002028
Initial registration date
February 24, 2017
Last updated
May 18, 2019 1:07 PM EDT
Location(s)
Region
Primary Investigator
Affiliation
University of California, San Diego
Other Primary Investigator(s)
PI Affiliation
University of Chicago
Additional Trial Information
Status
Completed
Start date
2017-02-21
End date
2017-03-21
Secondary IDs
Abstract
We will recruit survey participants on MTurk to conduct an online survey. With this survey, we plan to study whether the legitimacy (in the sense of conformity to the law) of a certain political view also affects its social desirability.
First, all subjects will answer a few demographic questions. Second, they will be asked about their opinion on prohibiting the building of minarets. Third, they will be asked to participate in a dictator game where they will have to split $3 between them and another subject. They will be randomized into four groups. Subjects in the control group will only be told that the other subject is a 24-year-old male from Switzerland. Subjects in the "anti-minarets" treatment group will be also told that the other subject is against the building of minarets. Subjects in the "anti-minarets public support" will additionally be told that the 57.5% of Swiss voters is against minarets. Finally, subjects in the "anti-minarets legitimacy" will be additionally told that following the result of a referendum in 2009, building minarets is illegal in Switzerland.
Our main hypotheses are:
1) The donation rates will be reduced in the "anti-minarets" when compared to the donation rates in the control group.
2) The donation rates will be increased in the "anti-minarets legitimacy" when compared to the donation rates in the "anti-minarets public support" group.

In a follow-up experiment, we recruit participants through an online panel survey firm who previously self-identified to the firm as Democrats. We study how the 2016 US presidential election changed people’s interpretation of observed xenophobic expression. First, all participants answer a few demographic questions. Second, the participants (player 1) are asked to play a dictator game with a random participant to a previous study (player 2) that they are matched with. Players 1 are informed that in the other study player 2 authorized a donation to an anti-immigration organization. We cross-randomize whether we tell player 1 that: i) Trump or Clinton won the election in player 2’s area (respectively "Trump won" and "Clinton won" treatment); and ii) player 2’s donation was in private or publicly observable on a website shared with other participants from his/her area (respectively "private" treatment and 'public" treatment). Our hypothesis is that player 1 donation rates to player 2 will be higher in the "Trump won public" treatment than in the other three treatment conditions.
External Link(s)
Registration Citation
Citation
Bursztyn, Leonardo and Stefano Fiorin. 2019. "Legitimacy And Social Desirability: MTurk Online Survey." AEA RCT Registry. May 18. https://doi.org/10.1257/rct.2028-4.0.
Former Citation
Bursztyn, Leonardo and Stefano Fiorin. 2019. "Legitimacy And Social Desirability: MTurk Online Survey." AEA RCT Registry. May 18. https://www.socialscienceregistry.org/trials/2028/history/46714.
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2017-02-24
Intervention End Date
2017-03-03
Primary Outcomes
Primary Outcomes (end points)
Dictator game decisions.
Primary Outcomes (explanation)
Dictator game decisions are the answer to the following question:

"In this exercise, we matched you with a participant from another survey. You will not know who you are paired with; only the researchers will know this. However, we will provide you with some additional background information about the other participant.

The participant you are matched with is a 24-year-old male from Switzerland.

You and the other participant will split a total bonus of $3. You alone will make the decision of how much of the $3 you will receive and how much of the $3 the other participant will receive. Whatever decision you make will be implemented. You can choose to divide the $3 however you like. Whatever you do not give to the other person you get to keep. The amount you keep will be credited to your MTurk account in the form of a bonus payment.For example, if you decide to give $1.70, then you will receive a bonus payment of $1.30.

How much would you like to give to the other person?"

We will examine two outcomes:
a) the average and median amount donated;
b) whether the participant donated a positive amount.

In a second phase of the experiment we slightly modified the text of our survey to the following:

"In this exercise, we matched you with a participant from another anonymous survey. You will not know who you are paired with. However, we will provide you with some additional background information about the other participant."

The purpose is to make clear that the person who expresses the anti-minaret sentiment did so in private, and not because of public pressure.

In the second experiment, the dictator game decisions are the answer to the following question:

"You and the other participant will split a total bonus of $2. You alone will make the decision of how much of the $2 you will receive and how much of the $2 the other participant will receive. You can choose to divide the $2 however you like, and you get to keep whatever you do not give to the other participant. For example, if you decide to give $1.30, then you will receive $0.70. Your survey provider will credit this bonus to your account.
How much would you like to give to the other participant? Keep in mind that when making his donation decision, the other participant did not know that he would be playing this follow-up game. Your decision about how much to give the other participant will be completely anonymous."
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We will recruit survey participants on MTurk to conduct an online survey. With this survey, we plan to study whether the legitimacy (in the sense of conformity to the law) of a certain political view also affects its social desirability.
First, all subjects will answer a few demographic questions. Second, they will be asked what is their opinion on prohibiting the building of minarets. We will focus our analysis on participants who are against prohibiting the building of minarets. Third, they will be asked to participate in a dictator game where they will have to split $3 between them and another subject. They will be randomized into four groups. Subjects in the control group will only be told that the other subject is a 24-year-old male from Switzerland. Subjects in the "anti-minarets" treatment group will be also told that the other subject is against minarets. Subjects in the "anti-minarets public support" will additionally be told that 57.5% of Swiss voters is against minarets. Finally, subjects in the "anti-minarets legitimacy" will be also told that following the result of a referendum, building minarets is illegal in Switzerland.
Participants in the control group will be asked whether they think minarets are illegal in Switzerland, and what percentage of Swiss people they think are against minarets.

In a follow-up experiment, we recruit participants through an online panel survey firm who previously self-identified to the firm as Democrats. First, all participants answer a few demographic questions. Second, we randomize the participants in one of two groups. In the "Trump won" group we tell participants that a respondent from another survey comes from an area where Trump won the election. In the "Clinton won" group we tell participants that a respondent from another survey comes from an area where Clinton won the election. Then all subjects are presented with two anti-immigrant quotes and are asked to predict the share of voters in the other participant’s area that
they believed would agree with the quotes. Third, the participants (player 1) are asked to play a dictator game with a random participant to a previous study (player 2) that they are matched with. Players 1 are informed that in the other study player 2 authorized a donation to an anti-immigration organization. We cross-randomize with the Trump/Clinton won treatments whether we tell player 1 that player 2’s donation was in private or publicly observable on a website shared with other participants from his/her area (respectively "private" treatment and 'public" treatment).
Experimental Design Details
Control: The participant you are matched with is a 24-year-old male from Switzerland. Anti-Minarets: The participant you are matched with is a 24-year-old male from Switzerland. He supports the prohibition of the building of minarets in Switzerland. Anti-Minarets Public Support: The participant you are matched with is a 24-year-old male from Switzerland. Like 57.5% of Swiss respondents, the participant supports the prohibition of the building of minarets in Switzerland. Anti-Minarets Referendum: The participant you are matched with is a 24-year-old male from Switzerland. Building minarets is illegal in Switzerland, following a 2009 referendum. Like 57.5% of Swiss respondents, the participant supports the prohibition of the building of minarets in Switzerland. However, he did not vote in the referendum since he was under legal voting age. In a second phase of the experiment we slightly modified the text of our survey to the following: Anti-Minarets: The participant you are matched with is a 24-year-old male from Switzerland. In our anonymous survey, like the one you just completed, he said he supports the prohibition of the building of minarets in Switzerland. Anti-Minarets Public Support: The participant you are matched with is a 24-year-old male from Switzerland. In our anonymous survey, like the one you just completed, he said he supports the prohibition of the building of minarets in Switzerland. According to numbers from 2009, 57.5% of Swiss respondents are in favor of prohibiting the building of minarets. Anti-Minarets Referendum: The participant you are matched with is a 24-year-old male from Switzerland. In our anonymous survey, like the one you just completed, he said he supports the prohibition of the building of minarets in Switzerland. Building minarets is illegal in Switzerland, following a 2009 referendum. According to numbers from 2009, 57.5% of Swiss respondents are in favor of prohibiting the building of minarets. However, the person you are matched with did not vote in the referendum since he was under legal voting age. The purpose is to make clear that the person who expresses the anti-minaret sentiment did so in private, and not because of public pressure.
Randomization Method
The randomization will be done on Qualtrics, a website for conducting online surveys.
Randomization Unit
The unit of randomization will be the individual subject.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
N/A
Sample size: planned number of observations
400 In the second experiment, 1,800
Sample size (or number of clusters) by treatment arms
100 control group, 100 anti-minarets, 100 anti-minarets public support, 100 anti-minarets referendum

In the second experiment: 450 Trump won public, 450 Trump won private, 450 Clinton won public, 450 Clinton won private
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
N/A
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
UCLA IRB
IRB Approval Date
2017-02-24
IRB Approval Number
IRB#16-001667
IRB Name
University of Chicago Social & Behavioral Sciences IRB
IRB Approval Date
2018-06-22
IRB Approval Number
IRB18-0594
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers