Back to History Current Version

The effect of incentives in a puzzle-solving game

Last registered on April 28, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
Overestimation of Self-evaluation of One’s Performance
RCT ID
AEARCTR-0009301
Initial registration date
April 25, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 28, 2022, 6:07 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
Amy Kang
Affiliation
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Heidi Tucholski
PI Affiliation
Contact Investigator
PI Name
Maya Thorson
PI Affiliation
Contact Investigator

Additional Trial Information

Status
In development
Start date
2022-05-09
End date
2022-05-23
Keywords
Behavior
Additional Keywords
competition, performance, self-evaluation
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Competition tied in with self-evaluation can trigger dishonesty in even the most honest person. In many professional settings, including in some parts of the United States military, subordinates are required to provide significant input about their work performance to their superiors as a part of the performance appraisal process.  In some cases, subordinates are expected to draft their own performance appraisals.  These appraisals may then be used to rank-order subordinates to determine who receives a limited number of rewards, such as a “Definitely Promote” (DP) rating of an officer in the United States military. The research question under investigation in this experiment is whether this form of competition for a limited number of rewards induces individuals to overstate the quality of their performance, something which could be considered a mild form of lying. We are planning to conduct this research through a matrix-solving game through an Amazon program called Amazon Mechanical Turk (MTurk). In the game, participants will be given 20 matrices of 12 numbers and will be asked to select the two numbers in each matrix that add up to 10. The participants will report the number of matrices that they believe that they solved correctly. We hypothesize that under a competition-based compensation scheme, one is more likely to overstate the quality of their performance.
External Link(s)

Registration Citation

Citation
Kang, Amy, Maya Thorson and Heidi Tucholski. 2022. "Overestimation of Self-evaluation of One’s Performance ." AEA RCT Registry. April 28. https://doi.org/10.1257/rct.9301-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Through this experiment we will be testing how many matrices a participant can solve correctly. They will earn more money if they score in the higher percentile.
Intervention Start Date
2022-05-09
Intervention End Date
2022-05-16

Primary Outcomes

Primary Outcomes (end points)
We hypothesize that under a competition-based compensation scheme, one is more likely to overstate the quality of their performance. With this data we can build a test statistic by making an outcome variable “Overstatement.”
Primary Outcomes (explanation)
Our null hypothesis is that in a competition-based environment, there is no difference in results. The alternative hypothesis is that in a competition-based environment, people are more likely to cheat. If the test statistic found is statistically significant, we reject the null that there is.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
To investigate this question, we use the matrix solving game utilized by Rigdon and D’Esterre (2015, “The effects of competition on the nature of cheating behavior,” Southern Economic Journal 81(4), 1012-1024).  The game will be conducted through an Amazon program called Amazon Mechanical Turk (MTurk). In the game, participants will be given 20 matrices of 12 numbers and will be asked to select the two numbers in each matrix that add up to 10.  After completing all 20 matrices, the participants will report the number of matrices that they believe that they solved correctly. In the no competition treatment, participants’ payment amount will be determined by whether they report above the threshold level of 17 matrices correctly solved.  Participants who do better than both thresholds will receive a larger payment than those who do not.  In the treatment condition of competition, not only must subjects outperform the minimum thresholds of 17 correct to earn the higher payment, but they must also score in the top 10% of subjects to earn the higher payment. 
Experimental Design Details
To investigate this question, we use the matrix solving game utilized by Rigdon and D’Esterre (2015, “The effects of competition on the nature of cheating behavior,” Southern Economic Journal 81(4), 1012-1024).  The game will be conducted through an Amazon program called Amazon Mechanical Turk (MTurk). In the game, participants will be given 20 matrices of 12 numbers and will be asked to select the two numbers in each matrix that add up to 10.  After completing all 20 matrices, the participants will report the number of matrices that they believe that they solved correctly. In the no competition treatment, participants’ payment amount will be determined by whether they report above the threshold level of 17 matrices correctly solved.  Participants who do better than both thresholds will receive a larger payment than those who do not.  In the treatment condition of competition, not only must subjects outperform the minimum thresholds of 17 correct to earn the higher payment, but they must also score in the top 10% of subjects to earn the higher payment. 
Randomization Method
The game will be conducted through an Amazon program called Amazon Mechanical Turk (MTurk). The randomization occurs whithin the population taking surveys for MRurk. This program allows individuals and companies to “harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development” (MTurk Website, https://www.mturk.com/). t is also important to note that these participants must answer questions pertaining to if they have received a high school diploma. This is a baseline that we must implement in the experiment because our data would be inaccurate if the participants have significantly different intellect levels.
Randomization Unit
NA
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
2
Sample size: planned number of observations
73
Sample size (or number of clusters) by treatment arms

Sample size= 73
Control group= 36
Experimental group=36
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
power onemean 1 2, sd(3) power(0.8)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials