Response of Productivity to Relative Performance Pay on Amazon's Mechanical Turk

Last registered on May 12, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
Response of Productivity to Relative Performance Pay on Amazon's Mechanical Turk
RCT ID
AEARCTR-0010820
Initial registration date
January 24, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 27, 2023, 2:33 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 12, 2023, 10:13 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
United States of America
Region
Online

Primary Investigator

Name
Weijia Wang
Affiliation
Norwegian School of Economics
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2023-01-24
End date
2023-04-30
Keywords
Behavior, Labor
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study will collect baseline data for the the main study of the project "Let Them Race - the role of beliefs and fairness views on choosing incentive schemes for others" (to be registered). In this study, MTurk workers will work on a real-effort task under different bonus schemes. The average productivity of the workers in each bonus scheme will be recorded and used for the elicitation of beliefs and the construction of incentive structures in the main study.
External Link(s)

Registration Citation

Citation
Wang, Weijia. 2023. "Response of Productivity to Relative Performance Pay on Amazon's Mechanical Turk." AEA RCT Registry. May 12. https://doi.org/10.1257/rct.10820-2.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
The MTurk participants first practice the key-pressing task in a trial round for 10 seconds, then work on the bonus round for 5 minutes in which they may earn bonus payments.
Before they start to work on the key-pressing task, they receive different information on how they will earn bonus payments depending on their assigned treatments. Their instructions read as the follows:

"In this round, you will be randomly matched to another worker recruited for this study.
You and your paired other worker will independently work on the task for 5 minutes. The pair of you will earn a joint bonus reward of $2 for completing this task."

For participants assigned to the equal treatment, they will be paid by an equal bonus scheme, and read:
"You and the other worker will each earn $1, regardless of your scores."

For participants assigned to the tournament treatment, they will be paid by a tournament bonus scheme, and read:
"The worker with the higher score will earn $2, the other worker will not earn any bonus."

For participants assigned to the proportional treatment, they will be paid by a proportional bonus scheme, and read:
"You and the other worker will split the $2 bonus proportional to your scores."
Intervention (Hidden)
Intervention Start Date
2023-01-24
Intervention End Date
2023-04-30

Primary Outcomes

Primary Outcomes (end points)
The number of points scored by the subject.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
MTurk participants will work on a real-effort task of alternatively pressing 'a' and 'b' on their keyboard (DellaVigna & Pope, 2018) for five minutes and will have the opportunity to earn bonus payments.

They will first read the instructions of the task and have the opportunity to try the task for 10 seconds. Then, depending on the treatment group they are assigned to, they will be given different information regarding whether they need to compete for a 2 USD bonus payment. They will be answer a simple comprehension question on the expected bonus payment to make sure that they understand the bonus scheme. If they answer correctly, they will proceed to the 5-minute round of the task for bonus payments. Otherwise, they will be shown the instructions on their assigned bonus scheme again before proceeding to the task.

The averages score in the equal bonus scheme (do not compete), the proportional bonus scheme (compete for a larger share of the $2 bonus), and the tournament bonus scheme (compete for $2) will be used in the main experiment of this project as baseline data and to construct the incentive structure for the participants in that experiment. The main experiment will be registered as "Let Them Race - the role of beliefs and fairness views in choosing incentive schemes for others".
Experimental Design Details
Randomization Method
Participants will be assigned to different treatment groups by the sequence they enter the online experiment. The first participant will be in equal treatment, second in tournament treatment, third in equal, fourth in tournament, and etc. Participants in the same treatment group will then be matched in pairs also by their entering sequence.
Participants in the proportional treatment is collected in another round. Data collection will exclude participants in the first round (equal or tournament treatments). They will be randomly matched in pairs by computer.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
300 MTurk participants
Sample size (or number of clusters) by treatment arms
Around 100 in each of the equal, proportional and tournament bonus schemes
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
NHH Institutional Review Board
IRB Approval Date
2023-01-20
IRB Approval Number
NHH-IRB 42/22
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
January 25, 2023, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
January 25, 2023, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
199 individuals
Final Sample Size (or Number of Clusters) by Treatment Arms
97 in equal bonus scheme, 102 in tournament bonus scheme
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials