Legitimacy And Social Desirability: MTurk Online Survey

Last registered on July 06, 2020

Pre-Trial

Trial Information

General Information

Title
Legitimacy And Social Desirability: MTurk Online Survey
RCT ID
AEARCTR-0002028
Initial registration date
February 24, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 24, 2017, 4:11 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
July 06, 2020, 8:39 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Bocconi University

Other Primary Investigator(s)

PI Affiliation
PI Affiliation
University of Chicago

Additional Trial Information

Status
Completed
Start date
2017-02-21
End date
2017-03-21
Secondary IDs
Abstract
We will recruit survey participants on MTurk to conduct an online survey. With this survey, we plan to study whether the legitimacy (in the sense of conformity to the law) of a certain political view also affects its social desirability.
First, all subjects will answer a few demographic questions. Second, they will be asked about their opinion on prohibiting the building of minarets. Third, they will be asked to participate in a dictator game where they will have to split $3 between them and another subject. They will be randomized into four groups. Subjects in the control group will only be told that the other subject is a 24-year-old male from Switzerland. Subjects in the "anti-minarets" treatment group will be also told that the other subject is against the building of minarets. Subjects in the "anti-minarets public support" will additionally be told that the 57.5% of Swiss voters is against minarets. Finally, subjects in the "anti-minarets legitimacy" will be additionally told that following the result of a referendum in 2009, building minarets is illegal in Switzerland.
Our main hypotheses are:
1) The donation rates will be reduced in the "anti-minarets" when compared to the donation rates in the control group.
2) The donation rates will be increased in the "anti-minarets legitimacy" when compared to the donation rates in the "anti-minarets public support" group.

In a follow-up experiment, we recruit participants through an online panel survey firm who previously self-identified to the firm as Democrats. We study how the 2016 US presidential election changed people’s interpretation of observed xenophobic expression. First, all participants answer a few demographic questions. Second, the participants (player 1) are asked to play a dictator game with a random participant to a previous study (player 2) that they are matched with. Players 1 are informed that in the other study player 2 authorized a donation to an anti-immigration organization. We cross-randomize whether we tell player 1 that: i) Trump or Clinton won the election in player 2’s area (respectively "Trump won" and "Clinton won" treatment); and ii) player 2’s donation was in private or publicly observable on a website shared with other participants from his/her area (respectively "private" treatment and 'public" treatment). Our hypothesis is that player 1 donation rates to player 2 will be higher in the "Trump won public" treatment than in the other three treatment conditions.
External Link(s)

Registration Citation

Citation
Bursztyn, Leonardo, Georgy Egorov and Stefano Fiorin. 2020. "Legitimacy And Social Desirability: MTurk Online Survey." AEA RCT Registry. July 06. https://doi.org/10.1257/rct.2028-5.0
Former Citation
Bursztyn, Leonardo, Georgy Egorov and Stefano Fiorin. 2020. "Legitimacy And Social Desirability: MTurk Online Survey." AEA RCT Registry. July 06. https://www.socialscienceregistry.org/trials/2028/history/71870
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2017-02-24
Intervention End Date
2017-03-03

Primary Outcomes

Primary Outcomes (end points)
Dictator game decisions.
Primary Outcomes (explanation)
Dictator game decisions are the answer to the following question:

"In this exercise, we matched you with a participant from another survey. You will not know who you are paired with; only the researchers will know this. However, we will provide you with some additional background information about the other participant.

The participant you are matched with is a 24-year-old male from Switzerland.

You and the other participant will split a total bonus of $3. You alone will make the decision of how much of the $3 you will receive and how much of the $3 the other participant will receive. Whatever decision you make will be implemented. You can choose to divide the $3 however you like. Whatever you do not give to the other person you get to keep. The amount you keep will be credited to your MTurk account in the form of a bonus payment.For example, if you decide to give $1.70, then you will receive a bonus payment of $1.30.

How much would you like to give to the other person?"

We will examine two outcomes:
a) the average and median amount donated;
b) whether the participant donated a positive amount.

In a second phase of the experiment we slightly modified the text of our survey to the following:

"In this exercise, we matched you with a participant from another anonymous survey. You will not know who you are paired with. However, we will provide you with some additional background information about the other participant."

The purpose is to make clear that the person who expresses the anti-minaret sentiment did so in private, and not because of public pressure.

In the second experiment, the dictator game decisions are the answer to the following question:

"You and the other participant will split a total bonus of $2. You alone will make the decision of how much of the $2 you will receive and how much of the $2 the other participant will receive. You can choose to divide the $2 however you like, and you get to keep whatever you do not give to the other participant. For example, if you decide to give $1.30, then you will receive $0.70. Your survey provider will credit this bonus to your account.
How much would you like to give to the other participant? Keep in mind that when making his donation decision, the other participant did not know that he would be playing this follow-up game. Your decision about how much to give the other participant will be completely anonymous."

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will recruit survey participants on MTurk to conduct an online survey. With this survey, we plan to study whether the legitimacy (in the sense of conformity to the law) of a certain political view also affects its social desirability.
First, all subjects will answer a few demographic questions. Second, they will be asked what is their opinion on prohibiting the building of minarets. We will focus our analysis on participants who are against prohibiting the building of minarets. Third, they will be asked to participate in a dictator game where they will have to split $3 between them and another subject. They will be randomized into four groups. Subjects in the control group will only be told that the other subject is a 24-year-old male from Switzerland. Subjects in the "anti-minarets" treatment group will be also told that the other subject is against minarets. Subjects in the "anti-minarets public support" will additionally be told that 57.5% of Swiss voters is against minarets. Finally, subjects in the "anti-minarets legitimacy" will be also told that following the result of a referendum, building minarets is illegal in Switzerland.
Participants in the control group will be asked whether they think minarets are illegal in Switzerland, and what percentage of Swiss people they think are against minarets.

In a follow-up experiment, we recruit participants through an online panel survey firm who previously self-identified to the firm as Democrats. First, all participants answer a few demographic questions. Second, we randomize the participants in one of two groups. In the "Trump won" group we tell participants that a respondent from another survey comes from an area where Trump won the election. In the "Clinton won" group we tell participants that a respondent from another survey comes from an area where Clinton won the election. Then all subjects are presented with two anti-immigrant quotes and are asked to predict the share of voters in the other participant’s area that
they believed would agree with the quotes. Third, the participants (player 1) are asked to play a dictator game with a random participant to a previous study (player 2) that they are matched with. Players 1 are informed that in the other study player 2 authorized a donation to an anti-immigration organization. We cross-randomize with the Trump/Clinton won treatments whether we tell player 1 that player 2’s donation was in private or publicly observable on a website shared with other participants from his/her area (respectively "private" treatment and 'public" treatment).
Experimental Design Details
Control: The participant you are matched with is a 24-year-old male from Switzerland.

Anti-Minarets: The participant you are matched with is a 24-year-old male from Switzerland. He supports the prohibition of the building of minarets in Switzerland.

Anti-Minarets Public Support: The participant you are matched with is a 24-year-old male from Switzerland. Like 57.5% of Swiss respondents, the participant supports the prohibition of the building of minarets in Switzerland.

Anti-Minarets Referendum: The participant you are matched with is a 24-year-old male from Switzerland. Building minarets is illegal in Switzerland, following a 2009 referendum. Like 57.5% of Swiss respondents, the participant supports the prohibition of the building of minarets in Switzerland. However, he did not vote in the referendum since he was under legal voting age.

In a second phase of the experiment we slightly modified the text of our survey to the following:

Anti-Minarets: The participant you are matched with is a 24-year-old male from Switzerland. In our anonymous survey, like the one you just completed, he said he supports the prohibition of the building of minarets in Switzerland.

Anti-Minarets Public Support: The participant you are matched with is a 24-year-old male from Switzerland. In our anonymous survey, like the one you just completed, he said he supports the prohibition of the building of minarets in Switzerland. According to numbers from 2009, 57.5% of Swiss respondents are in favor of prohibiting the building of minarets.

Anti-Minarets Referendum: The participant you are matched with is a 24-year-old male from Switzerland. In our anonymous survey, like the one you just completed, he said he supports the prohibition of the building of minarets in Switzerland. Building minarets is illegal in Switzerland, following a 2009 referendum. According to numbers from 2009, 57.5% of Swiss respondents are in favor of prohibiting the building of minarets. However, the person you are matched with did not vote in the referendum since he was under legal voting age.

The purpose is to make clear that the person who expresses the anti-minaret sentiment did so in private, and not because of public pressure.
Randomization Method
The randomization will be done on Qualtrics, a website for conducting online surveys.
Randomization Unit
The unit of randomization will be the individual subject.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
400 In the second experiment, 1,800
Sample size (or number of clusters) by treatment arms
100 control group, 100 anti-minarets, 100 anti-minarets public support, 100 anti-minarets referendum

In the second experiment: 450 Trump won public, 450 Trump won private, 450 Clinton won public, 450 Clinton won private
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
N/A
IRB

Institutional Review Boards (IRBs)

IRB Name
UCLA IRB
IRB Approval Date
2017-02-24
IRB Approval Number
IRB#16-001667
IRB Name
University of Chicago Social & Behavioral Sciences IRB
IRB Approval Date
2018-06-22
IRB Approval Number
IRB18-0594

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
Yes

Program Files

Program Files
Yes
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials