Back to History

Fields Changed

Registration

Field Before After
Trial Status in_development completed
Last Published March 24, 2021 11:42 AM April 12, 2021 07:57 AM
Primary Outcomes (End Points) Field Experiment The aim of the field experiment is to measure workers’ willingness to perform unethical tasks (i.e., manipulate Covid19 graphs). Primary outcomes: • [Accept] Do workers accept the follow-up HIT? (yes/no) • [Falsification] To what extent do the Covid19 death numbers in the graph deviate from the original data? Falsification = manipulated data – actual data. Manipulation Check The aim of the manipulation check survey is to measure the perceived moral acceptability of the follow-up HIT. We will randomly assign participants to one of two conditions. Participants will either read about the Control HIT or the Treatment/unethical HIT. Primary outcome: • [Moral] “Do you personally believe that working on this follow-up HIT is morally acceptable, morally unacceptable, or is it not a moral issue?” (Morally acceptable, Morally unacceptable, not a moral issue) Downstream Consequences Survey The aim of this survey is to measure the downstream consequences of the manipulated Covid19 graphs. We will randomly assign participants to one of two conditions. Participants will either see a representative graph from the Control HIT or the Treatment/unethical HIT. Primary outcomes: • [Actions] Given the current situation with the Covid19 outbreak, would you feel comfortable or uncomfortable doing each of the following in [state X]? Going out to the grocery store, Eating out in a restaurant, Attending an indoor sporting event or concerts, Visiting with a close friend or family member inside their home, Support mandatory mask wearing on public places in [state X], Travel to [state X] for a pre-paid trip in the next month. (1-7) • [Risk perceptions 1] How worried are you about the health consequences of Covid19 for you? (1-7) • [Risk perceptions 2] How worried are you that the Covid19 mutation will lead to a new wave of infections? (1-7) Field Experiment The aim of the field experiment is to measure workers’ willingness to perform unethical tasks (i.e., manipulate Covid19 graphs). Primary outcomes: • [Accept] Do workers accept the follow-up HIT? (yes/no) • [Falsification] To what extent do the Covid19 death numbers in the graph deviate from the original data? Falsification = manipulated data – actual data. Manipulation Check The aim of the manipulation check survey is to measure the perceived moral acceptability of the follow-up HIT. We will randomly assign participants to one of two conditions. Participants will either read about the Control HIT or the Treatment/unethical HIT. Primary outcome: • [Moral] “Do you personally believe that working on this follow-up HIT is morally acceptable, morally unacceptable, or is it not a moral issue?” (Morally acceptable, Morally unacceptable, not a moral issue) Downstream Consequences Survey The aim of this survey is to measure the downstream consequences of the manipulated Covid19 graphs. We will randomly assign participants to one of two conditions. Participants will either see a representative graph from the Control HIT or the Treatment/unethical HIT. Primary outcomes: • [Actions] After seeing this graph, would you feel comfortable or uncomfortable doing each of the following in [state X]? Going out to the grocery store, Eating out in a restaurant, Attending an indoor sporting event or concerts, Visiting with a close friend or family member inside their home, Support mandatory mask wearing on public places in [state X], Travel to [state X] for a pre-paid trip in the next month. (1-7) • [Risk perceptions 1] After seeing this graph, how worried are you about the health consequences of Covid19 for you? (1-7) • [Risk perceptions 2] After seeing this graph, how worried are you that the Covid19 mutation will lead to a new wave of infections? (1-7) • [Risk perceptions 3] Overall, this information makes me less/more likely to be vaccinated. (1-7)
Experimental Design (Public) The basic setup of this experiment is to hire workers on MTurk and ask them to create misleading graphs about the current Covid-19 pandemic. Then, we will measure whether they are willing to perform this task, and to what extent they are manipulating the data. We plan to recruit 1,200 workers on MTurk (400 for each of the three conditions). When recruiting subjects on MTurk, we will set as a requirement that an MTurker is US based, has a HIT approval rate of at least 98%, and that the MTurker has successfully completed at least 1,000 HITs (this is standard to ensure high-quality data). The experiment consists of two stages: 1. We will approach MTurkers and ask if they are interested in creating professional looking graphs (i.e., plotting data into graphs) showing the rate of Covid19 infections for a specific US state. We will recruit 50% Democrats and 50% Republicans. Workers will be paid $1.20 for the HIT, which takes about six minutes to complete. Stage 1 is the same for all workers. 2. All workers who complete the first task will be randomly assigned to one of three conditions (Baseline, Low-Pay, and Unethical). 3. We will then contact those workers and ask them to create similar professional looking graphs, this time about Covid19 deaths. Workers will be paid $1.20 (or $0.60 in the Low-Pay treatment) for the HIT, which again takes about six minutes to complete After all responses have been collected, we will debrief the workers. In addition to the main field experiment, we will also conduct two types of surveys. First, we will conduct a manipulation check (using a between-subjects design) with a new group of MTurkers. The primary goal of the manipulation check is to gauge workers’ perceptions of the moral acceptability of the follow-up HIT. We will also conduct a survey (using a between-subjects design) with a representative sample from the U.S. to measure the downstream consequences of the (manipulated) Covid19 graphs. To this end, we plan to recruit participants via Prolific. We will present representative graphs to the participants that were created by the MTurkers in the different conditions, and then we will ask them several questions about how it would affect their attitudes and behavior related to the Covid19 pandemic. The basic setup of this experiment is to hire workers on MTurk and ask them to create misleading graphs about the current Covid-19 pandemic. Then, we will measure whether they are willing to perform this task, and to what extent they are manipulating the data. We plan to recruit 1,200 workers on MTurk (400 for each of the three conditions). When recruiting subjects on MTurk, we will set as a requirement that an MTurker is US based, has a HIT approval rate of at least 98%, and that the MTurker has successfully completed at least 1,000 HITs (this is standard to ensure high-quality data). The experiment consists of two stages: 1. We will approach MTurkers and ask if they are interested in creating professional looking graphs (i.e., plotting data into graphs) showing the rate of Covid19 infections for a specific US state. Workers will be paid $1.20 for the HIT, which takes about six minutes to complete. Stage 1 is the same for all workers. 2. All workers who complete the first task will be randomly assigned to one of three conditions (Baseline, Low-Pay, and Unethical). 3. We will then contact those workers and ask them to create similar professional looking graphs, this time about Covid19 deaths. Workers will be paid $1.20 (or $0.60 in the Low-Pay treatment) for the HIT, which again takes about six minutes to complete After all responses have been collected, we will debrief the workers. In addition to the main field experiment, we will also conduct two types of surveys. First, we will conduct a manipulation check (using a between-subjects design) with a new group of MTurkers. The primary goal of the manipulation check is to gauge workers’ perceptions of the moral acceptability of the follow-up HIT. We will also conduct a survey (using a between-subjects design) with a representative sample from the U.S. to measure the downstream consequences of the (manipulated) Covid19 graphs. To this end, we plan to recruit participants via Prolific. We will present representative graphs to the participants that were created by the MTurkers in the different conditions, and then we will ask them several questions about how it would affect their attitudes and behavior related to the Covid19 pandemic.
Randomization Method Field Experiment We will place a job recruitment advertisement on MTurk for the first HIT, balancing the number of Democrats and Republicans we invite. There is no randomization at this stage yet, jobs are filled on a “first come, first-serve” basis. After completion of the first HIT, we will randomly assign workers to one of the three treatments. The randomization will be done in excel, with workers having the same probability to end up in one of the three treatments. We will use block randomization with respect to political party affiliation. Field Experiment We will place a job recruitment advertisement on MTurk for the first HIT, balancing the number of Democrats and Republicans we invite. There is no randomization at this stage yet, jobs are filled on a “first come, first-serve” basis. After completion of the first HIT, we will randomly assign workers to one of the three treatments. The randomization will be done in excel, with workers having the same probability to end up in one of the three treatments.
Intervention (Hidden) Field Experiment Here are the messages that we will send to the MTurkers for the follow-up HIT: Invitation of the Human Intelligence Task (HIT): CONTROL INVITATION [Header] Simple Data Visualization (Follow-Up) for $1.20 [Text] You recently completed a HIT visualizing Covid19 infections. I’m inviting you for a follow-up HIT to create a similar graph for Covid19 deaths. The HIT will pay $1.20 and it will again take about 6 minutes to complete. I will publicly post this new graph on Facebook and Twitter. Please search for “Michael Miller” or “Simple Data Visualization (Follow-Up)” to accept the HIT. LOW PAY INVITATION [Header] Simple Data Visualization (Follow-Up) for $0.60 [Text] You recently completed a HIT visualizing Covid19 infections. I’m inviting you for a follow-up HIT to create a similar graph for Covid19 deaths. The HIT will pay $0.60 and it will again take about 6 minutes to complete. I will publicly post this new graph on Facebook and Twitter. Please search for “Michael Miller” or “Simple Data Visualization (Follow-Up)” to accept the HIT. UNETHICAL INVITATION [Header] Simple Data Visualization (Follow-Up) for $1.20 [Text] You recently completed a HIT visualizing Covid19 infections. I’m inviting you for a follow-up HIT to create a similar graph for Covid19 deaths, but this time I need you to reduce the numbers compared to what the “original” numbers would look like. The HIT will pay $1.20 and it will again take about 6 minutes to complete. When you are done, the Covid19 death curve must look much flatter than the original entries. I don’t care how you change the numbers as long as someone can tell that the Covid19 death rate looks less worrying after viewing the manipulated graph. I will publicly post this new graph on Facebook and Twitter. Please search for “Michael Miller” or “Simple Data Visualization (Follow-Up)” to accept the HIT. Description of the Human Intelligence Task (HIT) First HIT (the same for all three treamtents) [Title] Simple Data Visualization [Description] Create a graph by dragging and dropping data points to match the number of Covid19 infection rates for each week listed in a provided table. [HIT instructions] To complete this HIT, drag and drop the data points below to create a graph that matches the number of Covid19 infections for each week listed in the table below. I need this graph so that I can better visualize the Covid19 infection rates. Follow up HIT CONTROL (same low pay treatment) [Title] Simple Data Visualization (Follow up) [Description] Create a graph by dragging and dropping data points to match the number of Covid19 death rates for each week listed in a provided table. [HIT instructions] To complete this HIT, drag and drop the data points to create a graph that matches the number of Covid19 deaths for each week listed in the table below. I will publicly post this new graph on Facebook and Twitter. Follow up HIT (UNETHICAL) [Title] Simple Data Visualization (Follow up) [Description] Create a graph by dragging and dropping data points such that the Covid19 death rate looks less worrying for each week listed in a provided table. [HIT instructions] To complete this HIT, drag and drop the data points to create a graph again, but this time reduce the numbers of Covid19 deaths (especially after Dec 7-13) so that the manipulated turquoise line looks much flatter than the original purple line. I don't really care how you change the turquoise line as long as someone looking at the manipulated line can tell that the Covid19 death rate looks less worrying than the original curve. I will publicly post this new graph on Facebook and Twitter. Follow-up Survey: Manipulation check https://erasmusuniversity.eu.qualtrics.com/jfe/form/SV_b7nxvEuWUYV5fIW Follow-up Survey: Downstream consequences https://erasmusuniversity.eu.qualtrics.com/jfe/form/SV_3XfPQYsvsCQ2dZs Field Experiment Here are the messages that we will send to the MTurkers for the follow-up HIT: Invitation of the Human Intelligence Task (HIT): CONTROL INVITATION [Header] Simple Data Visualization (Follow-Up) for $1.20 [Text] You recently completed a HIT visualizing Covid19 infections. I’m inviting you for a follow-up HIT to create a similar graph for Covid19 deaths. The HIT will pay $1.20 and it will again take about 6 minutes to complete. I will publicly post this new graph on Facebook and Twitter. Please search for “Michael Miller” or “Simple Data Visualization (Follow-Up)” to accept the HIT. LOW PAY INVITATION [Header] Simple Data Visualization (Follow-Up) for $0.60 [Text] You recently completed a HIT visualizing Covid19 infections. I’m inviting you for a follow-up HIT to create a similar graph for Covid19 deaths. The HIT will pay $0.60 and it will again take about 6 minutes to complete. I will publicly post this new graph on Facebook and Twitter. Please search for “Michael Miller” or “Simple Data Visualization (Follow-Up)” to accept the HIT. UNETHICAL INVITATION [Header] Simple Data Visualization (Follow-Up) for $1.20 [Text] You recently completed a HIT visualizing Covid19 infections. I’m inviting you for a follow-up HIT to create a similar graph for Covid19 deaths, but this time I need you to reduce the numbers compared to what the “original” numbers would look like. The HIT will pay $1.20 and it will again take about 6 minutes to complete. When you are done, the Covid19 death curve must look much flatter than the original entries. I don’t care how you change the numbers as long as someone can tell that the Covid19 death rate looks less worrying after viewing the manipulated graph. I will publicly post this new graph on Facebook and Twitter. Please search for “Michael Miller” or “Simple Data Visualization (Follow-Up)” to accept the HIT. Description of the Human Intelligence Task (HIT) First HIT (the same for all three treamtents) [Title] Simple Data Visualization [Description] Create a graph by dragging and dropping data points to match the number of Covid19 infection rates for each week listed in a provided table. [HIT instructions] To complete this HIT, drag and drop the data points below to create a graph that matches the number of Covid19 infections for each week listed in the table below. I need this graph so that I can better visualize the Covid19 infection rates. Follow up HIT CONTROL (same low pay treatment) [Title] Simple Data Visualization (Follow up) [Description] Create a graph by dragging and dropping data points to match the number of Covid19 death rates for each week listed in a provided table. [HIT instructions] To complete this HIT, drag and drop the data points to create a graph that matches the number of Covid19 deaths for each week listed in the table below. I will publicly post this new graph on Facebook and Twitter. Follow up HIT (UNETHICAL) [Title] Simple Data Visualization (Follow up) [Description] Create a graph by dragging and dropping data points such that the Covid19 death rate looks less worrying for each week listed in a provided table. [HIT instructions] To complete this HIT, drag and drop the data points to create a graph again, but this time reduce the numbers of Covid19 deaths (especially after Dec 7-13) so that the manipulated turquoise line looks much flatter than the original purple line. I don't really care how you change the turquoise line as long as someone looking at the manipulated line can tell that the Covid19 death rate looks less worrying than the original curve. I will publicly post this new graph on Facebook and Twitter. Follow-up Survey: Manipulation check https://erasmusuniversity.eu.qualtrics.com/jfe/form/SV_b7nxvEuWUYV5fIW Follow-up Survey: Downstream consequences https://erasmusuniversity.eu.qualtrics.com/jfe/form/SV_02negMnIoxzmNka
Back to top