Revealed work preferences of MTurkers

Last registered on August 05, 2021

Pre-Trial

Trial Information

General Information

Title
Revealed work preferences of MTurkers
RCT ID
AEARCTR-0007466
Initial registration date
August 03, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 05, 2021, 5:31 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region
Region

Primary Investigator

Affiliation
University Bremen

Other Primary Investigator(s)

PI Affiliation
University Bremen

Additional Trial Information

Status
On going
Start date
2021-08-03
End date
2021-10-01
Secondary IDs
DFG HO 5296/3-1
Abstract
This study will contribute to a better understanding of the motivation of workers active on microtask online platforms. Recent literature has focused mainly on self-reported motivation and thus measured stated preferences (SP), but little has been done to investigate revealed work preferences (RP). Therefore add to the existing literature by investigating RP of MTurkers in the US and India, the two largest MTurker groups.
Because of the higher average income in the USA (Prydz & Wadhwa, 2019), the amount of payment on MTurk constitutes a low stake in the USA and a high stake in India. Since low stakes and high stakes can have an impact on worker motivation, we examine possible differences between RP of workers from the USA and India.
In this study we will run a partial profile design conjoint analysis in which MTurkers can choose the job they later perform. In particular, we will investigate (1) whether the RP of MTurkers are in line with the findings of previous research; (2) whether SP and RP of MTurkers are inline in our study; and (3) whether high or low stakes in terms of payment affect SP and RP of MTurkers.
External Link(s)

Registration Citation

Citation
Hornuf, Lars and Lisa Nagel. 2021. "Revealed work preferences of MTurkers." AEA RCT Registry. August 05. https://doi.org/10.1257/rct.7466-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
In our choice experiment there is one intervention:
Each participant is presented with 14 choice tasks. In each task the participant has to choose the preferred of two alternatives. Each alternative is characterized by a set of four mostly discretely valued job attributes. In advance, a total of 300 combinations with four variables each were generated. Each participant is randomly shown 14 choice tasks, meaning the participants will not see the same alternatives.
Intervention Start Date
2021-08-03
Intervention End Date
2021-10-01

Primary Outcomes

Primary Outcomes (end points)
Our first outcome of interest is if the RP of MTurkers are in line with the findings of previous research.
Our second outcome of interest is whether SP and RP of MTurkers are identical in our study.
Our third outcome of interest is whether high or low stakes in terms of payment affect SP and RP of MTurkers.



Primary Outcomes (explanation)
RP - revealed preferences measured with conjoint analysis (partial profile design)
SP - stated preferences measured with ranking based conjoint analysis

Secondary Outcomes

Secondary Outcomes (end points)
Our first secondary outcome of interest is whether age has an influence on work motivation (intrinsic, extrinsic).
Our second secondary outcome of interest is whether sex has an influence on work motivation (intrinsic, extrinsic).
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will collect the data by creating a task on MTurk. MTurkers will see our task in the same manner as any other tasks and have to decide if they want to participate based to the task description and conditions (e.g., duration, payment). We conduct an incentivized decision experiment in the field. To identify RP of workers we use a partial profile conjoint analysis that determines the task they are later going to perform. We measure the SP based on a ranking based conjoint analysis that does not determine the task workers are going to perform.
We will run the identical study on MTurk for workers based in the USA and India.
Participant will start filling out a short questionnaire. Second, MTurkers are going to conduct the decision experiment. MTurkers will be informed that they can influence the task they have to do at the end of the study, based on their decisions in this part of the study. As part of a partial profile conjoint analysis workers decide which work characteristics the task they are going to work on should have, which enables us to investigate the RP of the MTurkers. Participants will several times see two job descriptions and have to decide which job they prefer. Overall, there will be 14 rounds with two job descriptions each. In every round, participants will choose which of the jobs presented they like better. Each job contains four measures of motivation, which were chosen based on the existing literature on motivation of workers. Hence, we have formulated 17 short statements of job attributes based upon the literature, which measure the different characteristics of work motivation. The combination of the variables and levels shown in the partial profile design conjoint analysis will be randomly assigned to the participants. Each choice set contains two job descriptions, each consisting of four statements. Each round both jobs show the same set of motivational factors but levels might differ (e.g. fun high vs. low).
After finishing the task of the partial profile conjoint analysis, participants will answer a survey.
Then Participants are asked to simply rank the job characteristics they saw at the beginning of the experiment according to the attractiveness of the characteristics, without having an influence on the job workers will perform. This ranking-based conjoint analysis allows us to capture the SP of participants, which we can then compare with the RP from the incentivized partial profile design conjoint analysis.
The last part of this experiment is a task based on the decisions the MTurkers made during the
study. Based on their statements they will either perform a real effort task that has a high degree of intrinsic motivation (fun) or a real effort task that has a high degree of extrinsic motivation (payment). If for example a participant stated that he or she would prefer a job that is fun over a job that has a high monetary reward the participant will be able to work on the fun task. In our case participant will see eight pictures of colorful landscapes. In one of the pictures is a hidden animal they have to find. If participants choose payment over fun the workers are paid 10% extra and will see eight pictures containing each three rows with numbers. Participants should find the picture that contains the number zero at least three times. The participants will see those eight pictured for seven rounds. To check whether our incentivization has worked, we are going to ask the participant, whether they believe that the task they just did was chosen for them because of the statements they made during the partial profile conjoint analysis.
Experimental Design Details
Randomization Method
The randomization is based on a model we generated with the software Sawtooth.
Using randomized combinations of our variables, 300 decision tasks were generated.
We tested our model of decision tasks, measuring different concepts of extrinsic and intrinsic motivation, in Sawtooth and a good fit was predicted, which implies that the occurrence probability of the items is equally distributed for our targeted number of participants. From the pool of 300 decision tasks created in this way, 14 tasks are randomly displayed to each subject in our experiment.
Randomization Unit
The randomization is on individual level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Since the experiment is not clustered, the number of clusters is the same as the sample size (see
below).
Sample size: planned number of observations
1500 individual workers from the crowdsourcing platform Amazon Mechanical Turk. 750 workers from the USA 750 workers from India
Sample size (or number of clusters) by treatment arms
There is only one treatment. The randomization of the shown choice sets is the basis for this decision experiment. As our model prediction could show, the occurrence probability of the items is equally distributed for our targeted number of participants.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethikkommission Universit├Ąt Bremen
IRB Approval Date
2021-01-29
IRB Approval Number
2021 - 02

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials