The effect of incentives in a puzzle-solving game

Last registered on July 13, 2022

Pre-Trial

Trial Information

General Information

Title
The effect of incentives in a puzzle-solving game
RCT ID
AEARCTR-0009301
Initial registration date
April 25, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 28, 2022, 6:07 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
July 13, 2022, 4:29 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation

Other Primary Investigator(s)

PI Affiliation
U.S. Air Force Academy
PI Affiliation
U.S. Air Force Academy
PI Affiliation
U.S. Air Force Academy

Additional Trial Information

Status
In development
Start date
2022-08-08
End date
2022-12-16
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Competition tied in with self-evaluation can trigger dishonesty in even the most honest person. In many professional settings, including in some parts of the United States military, subordinates are required to provide significant input about their work performance to their superiors as a part of the performance appraisal process.  In some cases, subordinates are expected to draft their own performance appraisals.  These appraisals may then be used to rank-order subordinates to determine who receives a limited number of rewards, such as a “Definitely Promote” (DP) rating of an officer in the United States military. The objective of this study is to examine the effects of two alternative incentive systems designed to simulate alternative performance reward systems on stated performance in a matrix-solving game.
External Link(s)

Registration Citation

Citation
Silz-Carson, Katherine et al. 2022. "The effect of incentives in a puzzle-solving game." AEA RCT Registry. July 13. https://doi.org/10.1257/rct.9301
Experimental Details

Interventions

Intervention(s)
The objective of this study is to examine the effects of two alternative incentive systems on stated performance in a matrix-solving game . In this game, participants are given a set of 20 matrices containing 12 numbers each. For each matrix, participants are asked to select the two numbers that sum to 10. After completing all 20 matrices, participants will report on a separate page the total number of matrices that they believe that they solved correctly. Participants will be compensated based on their stated number of matrices solved using one of two possible incentive systems. Participants will be randomly assigned to one of two incentive systems to determine their experimental compensation.
Intervention Start Date
2022-08-15
Intervention End Date
2022-09-30

Primary Outcomes

Primary Outcomes (end points)
Outcome 1: Stated performance in Incentive System #1
Outcome 2: Stated performance in Incentive System #2
Primary Outcomes (explanation)
Overstatement is the difference between the number of matrices that a participant claims to solve and the number of matrices that they actually solve (observable based on their responses to the 20 matrix problems). Overstatement = Stated number solved - actual number solved

Secondary Outcomes

Secondary Outcomes (end points)
None
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The objective of this study is to examine the effects of two alternative incentive systems on stated performance in a matrix-solving game . In this game, participants are given a set of 20 matrices containing 12 numbers each. For each matrix, participants are asked to select the two numbers that sum to 10. After completing all 20 matrices, participants will report on a separate page the total number of matrices that they believe that they solved correctly. Participants will be compensated based on their stated number of matrices solved using one of two possible incentive systems. Participants will be randomly assigned to one of two incentive systems to determine their experimental compensation.
Experimental Design Details
Not available
Randomization Method
Participants will be randomly assigned to one of two incentive systems by a computer.
Randomization Unit
Random assignment occurs at the individual participant level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A - observations in this experimental design are not clustered
Sample size: planned number of observations
150 individual participants - 75 per incentive system
Sample size (or number of clusters) by treatment arms
75 individual participants in incentive system #1
75 individual participants in incentive system #2
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
A difference of means test of the null hypothesis that the amount of overstatement is the same in Incentive Systems #1 and #2 results in a minimum detectable effect size of 1-1.5 puzzles. The analysis assumes independent random samples and assumes the following parameters, which were drawn from the literature on prior studies that have used the matrix puzzle test. Average overstatement in Incentive System #1: 1 puzzle Standard deviation of overstatement (both Incentive Systems): 3 puzzles Level of significance: 0.05 Power: 0.8.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
U.S. Air Force Academy Institutional Review Board
IRB Approval Date
2022-06-15
IRB Approval Number
FAC20220019E