An experimental study investigating the effects of rank feedback on performance

Last registered on August 24, 2021

Pre-Trial

Trial Information

General Information

Title
An experimental study investigating the effects of rank feedback on performance
RCT ID
AEARCTR-0006877
Initial registration date
December 10, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 10, 2020, 12:30 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
August 24, 2021, 11:38 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Technology Sydney

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2020-01-07
End date
2020-10-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The effect of relative performance feedback, especially ordinal rank feedback, on performance has been broadly discussed in recent decades. However, there are few studies on the relationship between the impacts of providing relative performance feedback and the choice of comparison group. This trail intends to investigate the effects of rank feedback provision on subsequent performance, and whether the effect tends to be enhanced in smaller same-ability groups.
External Link(s)

Registration Citation

Citation
Lou, Hanlin. 2021. "An experimental study investigating the effects of rank feedback on performance." AEA RCT Registry. August 24. https://doi.org/10.1257/rct.6877-1.1
Experimental Details

Interventions

Intervention(s)
The basic experimental design requires students to participate in two tests. We randomise subjects into one of three conditions:

Control group: No feedback
There was no feedback provided to students in the control group.

Treatment 1: Overall rank feedback
Students receive their performance ranking in respective classes after the first test.

Treatment 2: Group rank feedback
Students receive their performance ranking in a same-ability group of size five after the first test.
Intervention Start Date
2020-01-10
Intervention End Date
2020-01-11

Primary Outcomes

Primary Outcomes (end points)
Performance in each test.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Study 1:
Teachers held a contest without prizes to motivate students to learn English words after the final exam of the autumn semester 2019. 342 students (156 female) from grades 5 and 6 participated in this contest, and they were asked to remember 60 English words in 5 days. On the morning of the first day, students received a word list consisting of two parts with 30 words each, and they were informed that they would be examined on the first part and the second part on the morning of the third day and the sixth day, respectively. All English words were selected from the textbook from which students were supposed to learn in the next semester. Hence, students, in principle, were not familiar with these words. Each test consisted of 30 questions of three types (Chinese--English translation, English--Chinese translation, and crossword puzzle). To obtain a high score, students needed to memorise as many English words and their Chinese meanings as possible.

Before the first test, teachers randomly assigned classes in each grade into three groups: an unranked control group and two treatment groups. Teachers gave treated students an individualised note with the rank feedback on the afternoon of the third day, and they were encouraged not to share their ranking with others. By contrast, students in the control group were reminded only that there would be another test 2 days later. The feedback provided to students varied across treatment groups. To examine whether providing rank feedback had positive impacts on performance, as some existing research has claimed, teachers gave two classes of students overall rank feedback. Students in this treatment group learned their relative position among all students in the classroom. Meanwhile, the group rank feedback was provided to four classes of students in another treatment group. Specifically, teachers first ranked students within the class by their scores for the first test and then divided them into groups of five from top to bottom. Hence, the notes given to students presented their rank in a group of five members. However, students had no idea who the other members of the group were, or whether they had similar performance.

Study 2:
On the morning of the first day, a Thursday, teachers gave the first test to their students. On the exam paper were listed 100 incomplete English words, each of which had several letters removed alongside their corresponding Chinese meaning. The students had to correctly complete as many English words as they could. On the afternoon of the following day (Friday), teachers provided students in different treatment groups with their corresponding relative performance feedback. Similar to the operation in Study 1, there was no feedback provided to students in the control group, and students in the treatment groups received either their overall rank or group rank. Then, all students received the answers for the initial test, a word list, and they were informed that there would be a test exactly like the earlier one 3 days later; 365 students (176 female) attended the initial test, but 13 of them were absent from the second test, and hence, we had 352 observations (171 female) in the end.
Experimental Design Details
Randomization Method
Randomization by a computer prior to the start of the experiment.
Randomization Unit
We randomise all treatments at the class level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
8 classes in each study.
Sample size: planned number of observations
694 students.
Sample size (or number of clusters) by treatment arms
5 classes Control Group, 4 classes Treatment 1, 7 classes Treatment 2
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials