|
Field
Intervention (Hidden)
|
Before
After Amazon Mechanical Turk users click select our task and complete the mandatory parts of the task (demographic questions and rate and classify three jokes), they will be taken to a page that contains the text below:
***
You have now completed the HIT. Thank you!
If you would like to finish, then please click on the [Finish] button below. You will then receive the survey code for Amazon Mechanical Turk.
However, we do have more jokes that need to be rated and classified. If you like you can help us classify more jokes.
Please note that rating and classifying the following jokes is purely voluntary and your payment does not depend on this.
If you would like to continue, please click on the [Continue] button below. You can stop at any point by clicking [Finish] button at the bottom of the page. When you click [Finish] button, you will receive the survey code for Amazon Mechanical Turk.
***
This forms our baseline. Our treatments are in two dimensions:
Emphasizing the need for contributions by minorities: the participants in this group are shown one of the following two messages, emphasizing the need for female participants and racial minorities respectively:
“Our goal is to ensure that the jokes are not offensive to anyone. If you are a woman, your answers would be extremely valuable to us. However, we appreciate answers from everyone.”
“Our goal is to ensure that the jokes are not offensive to anyone. If you belong to a racial minority group, your answers would be extremely valuable to us. However, we appreciate answers from everyone.”
2. Monetary compensation for the three additional contributions. We offer two bonuses: 5 cents per joke or 10 cents per joke (up to 15 or 30 cents respectively). The voluntary contributions paragraph in both cases is replaced by the following:
“For each of the following jokes you rate and classify (up to 3 jokes), the payment is $X per joke. For example, if you rate all 3 jokes, then your additional payment is $3X. This will be paid in the form of a bonus on Amazon Mechanical Turk, it is on top of the initial payment of the HIT.”
As in both dimensions we have three groups (control + two treatments), we have 3 x 3 = 9 groups. Participants will be randomized into the 9 groups with equal probability.
|
After
Individuals will choose to participate in the study by selecting it on Amazon's Mechanical Turk platform. The study is marketed on Amazon Mechanical Turk as a 5-minute research study that involves rating and classifying jokes. Before the participants choose to participate, they will be provided with a brief description of the study. Among other information the description says that each participant will receive a payment of 50 cents for completing the study, that he is asked to read 3 jokes (anecdotes) and answer questions about these, and asked to answer 5 demographic questions; that it should take on average 5 minutes.
When an individual chooses to participate, he will first be asked 5 demographic questions. After that he will be shown the first joke, and asked to rate the joke in terms of how funny it is, whether it is appropriate or offending to specific groups of people, and classify the topic of the joke. That is repeated for 3 jokes.
When the participant has finished rating and classifying 3 jokes, he will be told that he has finished the study, but if he would like to he can help to rate and classify 3 more jokes. There are 9 randomly chosen treatments which vary by the piece-rate incentives for rating the additional jokes and by the message shown regarding opinions by which group are most valuable to us (either women or racial minorities). At that point each participant can either finish or continue to rate and classify up to 3 additional jokes.
After completing the study, the participants will receive the flat payment of 50 cents plus the additional bonus (according to the randomized treatment and participant's contribution).
Sample size:
The planned number of observations is 1800-5400 individuals. Our goal is to get at least 200 and at most 600 individuals per treatment (or control) group within short timeframe. The task will be kept open on Amazon Mechanical Turk until either (i) the end of October 31st, 2015 or (ii) 5400 subjects have completed the study. If by the end of October 31st less than 1800 subjects have completed the study, then the task will be kept open until 1800 subjects are obtained.
Outcome variables:
The key outcome variable is the number of extra tasks fulfilled (each participant is asked to fulfill 3 tasks and can choose to fulfill 0..3 extra tasks).
After Amazon Mechanical Turk users click select our task and complete the mandatory parts of the task (demographic questions and rate and classify three jokes), they will be taken to a page that contains the text below:
***
You have now completed the HIT. Thank you!
If you would like to finish, then please click on the [Finish] button below. You will then receive the survey code for Amazon Mechanical Turk.
However, we do have more jokes that need to be rated and classified. If you like you can help us classify more jokes.
Please note that rating and classifying the following jokes is purely voluntary and your payment does not depend on this.
If you would like to continue, please click on the [Continue] button below. You can stop at any point by clicking [Finish] button at the bottom of the page. When you click [Finish] button, you will receive the survey code for Amazon Mechanical Turk.
***
This forms our baseline. Our treatments are in two dimensions:
Emphasizing the need for contributions by minorities: the participants in this group are shown one of the following two messages, emphasizing the need for female participants and racial minorities respectively:
“Our goal is to ensure that the jokes are not offensive to anyone. If you are a woman, your answers would be extremely valuable to us. However, we appreciate answers from everyone.”
“Our goal is to ensure that the jokes are not offensive to anyone. If you belong to a racial minority group, your answers would be extremely valuable to us. However, we appreciate answers from everyone.”
2. Monetary compensation for the three additional contributions. We offer two bonuses: 5 cents per joke or 10 cents per joke (up to 15 or 30 cents respectively). The voluntary contributions paragraph in both cases is replaced by the following:
“For each of the following jokes you rate and classify (up to 3 jokes), the payment is $X per joke. For example, if you rate all 3 jokes, then your additional payment is $3X. This will be paid in the form of a bonus on Amazon Mechanical Turk, it is on top of the initial payment of the HIT.”
As in both dimensions we have three groups (control + two treatments), we have 3 x 3 = 9 groups. Participants will be randomized into the 9 groups with equal probability.
|