Supervisory Feedback, Effort, and Performance - a Randomized Experiment Using an Online Game

Last registered on November 27, 2020

Pre-Trial

Trial Information

General Information

Title
Supervisory Feedback, Effort, and Performance - a Randomized Experiment Using an Online Game
RCT ID
AEARCTR-0003984
Initial registration date
March 07, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 08, 2019, 3:59 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
November 27, 2020, 10:36 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Centre for Economic and Regional Studies, Hungarian Academy of Sciences

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2019-04-01
End date
2020-02-28
Secondary IDs
Abstract
We study the effects of various elements of supervisory communication - peer performance information and subjective feedback - on the effort and performance of individuals. To do so, we randomize the provision of such communication elements during a simple online game of visual perception. The game is advertised as a challenging game of self-improvement using social media outlets. We study the effect on various outcomes: intensity of play and number of games played (effort), as well as accuracy and game score (performance). We study the mean effect on players, as well as the effect on subgroups based on task-related confidence and gender.
External Link(s)

Registration Citation

Citation
Lovasz, Anna. 2020. "Supervisory Feedback, Effort, and Performance - a Randomized Experiment Using an Online Game." AEA RCT Registry. November 27. https://doi.org/10.1257/rct.3984-2.1
Former Citation
Lovasz, Anna. 2020. "Supervisory Feedback, Effort, and Performance - a Randomized Experiment Using an Online Game." AEA RCT Registry. November 27. https://www.socialscienceregistry.org/trials/3984/history/80569
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The intervention includes elements of performance feedback (leader board) and subjective feedback (encouragement, praise, trash talk), given prior to and during the online game.
Intervention Start Date
2019-04-01
Intervention End Date
2019-11-30

Primary Outcomes

Primary Outcomes (end points)
We plan to measure individual effort (number of games played, intensity of game playing (clicks)), as well as performance measures (accuracy of clicks, game score).
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We collect data using an online game that is advertised using social media outlets.
During the 2 minute game, treatment is given using pop-up text boxes using simple phrases and graphics.
Treatment is randomized among the interactions of two dimensions: (1) performance information and (2) subjective feedback.
Experimental Design Details
We collect data using an online game that is advertised using social media outlets.
During the 2 minute game, treatment is given using pop-up text boxes using simple phrases and graphics.

Treatment is randomized among the interactions of two dimensions:
A. performance information (only current game score or also leader board is shown)
B. subjective feedback (none, encouragement, praise, trash talk)
This gives us a total of 8 treatment groups:
1. own score only, no subjective feedback
2. own score only, encouragement
3. own score only, praise
4. own score only, no trash talk
5. leader board, no subjective feedback
6. leader board, encouragement
7. leader board, praise
8. leader board, trash talk
9. leader board, praise
Randomization Method
Randomization via computer (algorithm).
Randomization Unit
Randomization will be done at the individual game session level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1 online game
Sample size: planned number of observations
8000 individuals
Sample size (or number of clusters) by treatment arms
1000 individuals for each treatment arm (8 arms including control group)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
February 28, 2020, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
February 28, 2020, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
5009 individuals
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
5009 individuals
Final Sample Size (or Number of Clusters) by Treatment Arms
616 no leaderboard no feedback, 656 no leaderboard supportive feedback, 635 no leaderboard rewarding feedback, 608 no leaderboard trashtalk, 638 leaderboard no feedback, 611 leaderboard supportive feedback, 639 leaderboard rewarding feedback, 606 leaderboard trashtalk.
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials