Field
Trial Title

Before
Overestimation of Selfevaluation of One’s Performance

After
The effect of incentives in a puzzlesolving game

Field
Investigator

Before
Amy Kang

After
Katherine SilzCarson

Field
Abstract

Before
Competition tied in with selfevaluation can trigger dishonesty in even the most honest person. In many professional settings, including in some parts of the United States military, subordinates are required to provide significant input about their work performance to their superiors as a part of the performance appraisal process. In some cases, subordinates are expected to draft their own performance appraisals. These appraisals may then be used to rankorder subordinates to determine who receives a limited number of rewards, such as a “Definitely Promote” (DP) rating of an officer in the United States military. The research question under investigation in this experiment is whether this form of competition for a limited number of rewards induces individuals to overstate the quality of their performance, something which could be considered a mild form of lying. We are planning to conduct this research through a matrixsolving game through an Amazon program called Amazon Mechanical Turk (MTurk). In the game, participants will be given 20 matrices of 12 numbers and will be asked to select the two numbers in each matrix that add up to 10. The participants will report the number of matrices that they believe that they solved correctly. We hypothesize that under a competitionbased compensation scheme, one is more likely to overstate the quality of their performance.

After
Competition tied in with selfevaluation can trigger dishonesty in even the most honest person. In many professional settings, including in some parts of the United States military, subordinates are required to provide significant input about their work performance to their superiors as a part of the performance appraisal process. In some cases, subordinates are expected to draft their own performance appraisals. These appraisals may then be used to rankorder subordinates to determine who receives a limited number of rewards, such as a “Definitely Promote” (DP) rating of an officer in the United States military. The objective of this study is to examine the effects of two alternative incentive systems designed to simulate alternative performance reward systems on stated performance in a matrixsolving game.

Field
Trial Start Date

Before
May 09, 2022

After
August 08, 2022

Field
Trial End Date

Before
May 23, 2022

After
December 16, 2022

Field
JEL Code(s)

Before

After
C91

Field
Last Published

Before
April 28, 2022 06:07 PM

After
July 13, 2022 04:29 PM

Field
Intervention (Public)

Before
Through this experiment we will be testing how many matrices a participant can solve correctly. They will earn more money if they score in the higher percentile.

After
The objective of this study is to examine the effects of two alternative incentive systems on stated performance in a matrixsolving game . In this game, participants are given a set of 20 matrices containing 12 numbers each. For each matrix, participants are asked to select the two numbers that sum to 10. After completing all 20 matrices, participants will report on a separate page the total number of matrices that they believe that they solved correctly. Participants will be compensated based on their stated number of matrices solved using one of two possible incentive systems. Participants will be randomly assigned to one of two incentive systems to determine their experimental compensation.

Field
Intervention Start Date

Before
May 09, 2022

After
August 15, 2022

Field
Intervention End Date

Before
May 16, 2022

After
September 30, 2022

Field
Primary Outcomes (End Points)

Before
We hypothesize that under a competitionbased compensation scheme, one is more likely to overstate the quality of their performance. With this data we can build a test statistic by making an outcome variable “Overstatement.”

After
Outcome 1: Stated performance in Incentive System #1
Outcome 2: Stated performance in Incentive System #2

Field
Primary Outcomes (Explanation)

Before
Our null hypothesis is that in a competitionbased environment, there is no difference in results. The alternative hypothesis is that in a competitionbased environment, people are more likely to cheat. If the test statistic found is statistically significant, we reject the null that there is.

After
Overstatement is the difference between the number of matrices that a participant claims to solve and the number of matrices that they actually solve (observable based on their responses to the 20 matrix problems). Overstatement = Stated number solved  actual number solved

Field
Experimental Design (Public)

Before
To investigate this question, we use the matrix solving game utilized by Rigdon and D’Esterre (2015, “The effects of competition on the nature of cheating behavior,” Southern Economic Journal 81(4), 10121024). The game will be conducted through an Amazon program called Amazon Mechanical Turk (MTurk). In the game, participants will be given 20 matrices of 12 numbers and will be asked to select the two numbers in each matrix that add up to 10. After completing all 20 matrices, the participants will report the number of matrices that they believe that they solved correctly. In the no competition treatment, participants’ payment amount will be determined by whether they report above the threshold level of 17 matrices correctly solved. Participants who do better than both thresholds will receive a larger payment than those who do not. In the treatment condition of competition, not only must subjects outperform the minimum thresholds of 17 correct to earn the higher payment, but they must also score in the top 10% of subjects to earn the higher payment.

After
The objective of this study is to examine the effects of two alternative incentive systems on stated performance in a matrixsolving game . In this game, participants are given a set of 20 matrices containing 12 numbers each. For each matrix, participants are asked to select the two numbers that sum to 10. After completing all 20 matrices, participants will report on a separate page the total number of matrices that they believe that they solved correctly. Participants will be compensated based on their stated number of matrices solved using one of two possible incentive systems. Participants will be randomly assigned to one of two incentive systems to determine their experimental compensation.

Field
Randomization Method

Before
The game will be conducted through an Amazon program called Amazon Mechanical Turk (MTurk). The randomization occurs whithin the population taking surveys for MRurk. This program allows individuals and companies to “harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development” (MTurk Website, https://www.mturk.com/). t is also important to note that these participants must answer questions pertaining to if they have received a high school diploma. This is a baseline that we must implement in the experiment because our data would be inaccurate if the participants have significantly different intellect levels.

After
Participants will be randomly assigned to one of two incentive systems by a computer.

Field
Randomization Unit

Before
NA

After
Random assignment occurs at the individual participant level

Field
Planned Number of Clusters

Before
2

After
N/A  observations in this experimental design are not clustered

Field
Planned Number of Observations

Before
73

After
150 individual participants  75 per incentive system

Field
Sample size (or number of clusters) by treatment arms

Before
Sample size= 73
Control group= 36
Experimental group=36

After
75 individual participants in incentive system #1
75 individual participants in incentive system #2

Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes

Before
power onemean 1 2, sd(3) power(0.8)

After
A difference of means test of the null hypothesis that the amount of overstatement is the same in Incentive Systems #1 and #2 results in a minimum detectable effect size of 11.5 puzzles. The analysis assumes independent random samples and assumes the following parameters, which were drawn from the literature on prior studies that have used the matrix puzzle test.
Average overstatement in Incentive System #1: 1 puzzle
Standard deviation of overstatement (both Incentive Systems): 3 puzzles
Level of significance: 0.05
Power: 0.8.

Field
Keyword(s)

Before
Behavior

After
Behavior, Labor

Field
Intervention (Hidden)

Before
This experiment not only tested how many matrices a participant could solve correctly, but it also tested to see if there was over exaggeration in self reports on how many they got right based on if they were told they would receive more money or not. We told one group they would earn more money if they selfreported more matrices correctly, while the other one was just told to solve the matrices without any extra incentive. y using a statistical program named Stata, we could use the number collected from the participants and actually see if the participant solved what they said they solved. We plug these numbers into a regression equation that will output whether or not overstatement is popular in the group that was incentivized with more money.

After
The objective of this study is to examine the effects of two alternative incentive systems on stated performance in the matrixsolving game used by Rigdon and D’Esterre (2015). In this game, participants are given a set of 20 matrices containing 12 numbers each . For each matrix, participants are asked to select the two numbers that sum to 10. After completing all 20 matrices, participants will report on a separate page the total number of matrices that they believe that they solved correctly. Participants will be compensated based on their stated number of matrices solved using one of two possible incentive systems:
Incentive System #1: Base rate + Bonus if solve at least 15 matrices
Incentive System #2: Base rate + Bonus if solve at least 15 matrices AND are in the top 10% of subjects assigned to this incentive system (including ties)
Participants will be randomly assigned to one of the two incentive systems. Incentive System #1 is designed to simulate most standard employee performance appraisal systems, in which employees become eligible for performance awards (e.g. time off awards, cash bonuses, step increases in pay) only if they exceed some minimum standard of performance (e.g. In the Federal employee system, employees must rate at least a 3 on a 5point scale to be eligible for these awards). Incentive System #2 also requires participants to meet this minimum standard, but adds competition in the form of a tournament. It is designed to simulate the competition that occurs in performance evaluation systems that only reward the top n% of employees, such as the P/DP promotion rating system that exists for members of the military. The null hypothesis to be tested is:
H0: The amount of overstatement of performance in Incentive System #1 and Incentive System #2 are the same.
We expect to be able to reject this hypothesis. Although our prior is that there will be more overstatement in Incentive system #2, some prior research (e.g. Cadsby, Song, and Tapon (2010)) has found more overclaims in targetbased schemes than in tournament schemes. Thus, there is a possibility that there will be more overstatement in Incentive System #1 than in Incentive System #2. For this reason, we use a 2sided alternative hypothesis when specifying the parameters of the power analysis.

Field
Did you obtain IRB approval for this study?

Before
No

After
Yes

Field
Secondary Outcomes (End Points)

Before

After
None

Field
Pi as first author

Before
No

After
Yes

Field
Public locations

Before
Yes

After
No
