The effect of relative performance feedback on academic outcomes

Last registered on February 24, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
The effect of relative performance feedback on academic outcomes
RCT ID
AEARCTR-0008977
Initial registration date
February 18, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 24, 2022, 1:00 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Austria
Region

Primary Investigator

Name
Anna Schwarz
Affiliation
Vienna University of Economics and Business
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Rupert Sausgruber
PI Affiliation
Vienna University of Economics and Business
PI Name
Mario Lackner
PI Affiliation
Johannes Kepler University Linz
PI Name
Rudolf Winter-Ebmer
PI Affiliation
Johannes Kepler University Linz

Additional Trial Information

Status
In development
Start date
2022-02-21
End date
2025-02-21
Keywords
Behavior, Education
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Despite extensive research on the effect of relative performance feedback, a consensus on the direction and mechanics of the effect, especially in the higher education context, is far. Moreover, the role of students' prior beliefs about their performance is poorly understood. Using a large-scale survey at a big European university, we can cleanly elicit these beliefs before providing relative performance feedback to a randomly selected treatment group. By linking the survey data to administrative data, we can subsequently measure the effect on academic performance in the short- and long-run and by prior beliefs. Additionally, the treatment group has to actively decide to see the information, which allows us to analyze selection into receiving relative performance feedback. The survey context further enables us to estimate the effect of relative performance feedback on several secondary outcomes, such as competitiveness, stress, academic self-concept. Our clean experimental design using survey as well as administrative data lends itself to comprehensively analyze the effect of relative performance feedback in higher education and provide evidence on the role of prior performance beliefs.
External Link(s)

Registration Citation

Citation
Lackner, Mario et al. 2022. "The effect of relative performance feedback on academic outcomes." AEA RCT Registry. February 24. https://doi.org/10.1257/rct.8977-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
The intervention is placed within a university-survey, that is sent to all bachelor beginners at the end of their first semester. The treatment consists of providing one half of students with the opportunity to be informed about their position in the GPA (Grade Point Average) as well as the ECTS (European Credit Transfer and Accumulation System) distribution. It is thus an intention-to-treat design. We implement this by telling students the quartile of the distribution in which their performance is located. The reference group consists of all students who started their studies in the same semester and for the same field of study. The decision whether students choose to see their relative performance will be recorded. This enables us to analyze, who selects into receiving information. Allowing students to opt into receiving performance feedback may also attenuate negative effects, that are documented in the literature.
Intervention (Hidden)
Intervention Start Date
2022-02-21
Intervention End Date
2025-02-21

Primary Outcomes

Primary Outcomes (end points)
Information retrieved
ECTS completed
GPA
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Survey measures: These include study satisfaction, study effort, stress and perceived competitiveness,
academic self-concept, and prosociality in their studies. (exploratory analysis)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The students will be randomized into treatment and control group and see the following items in the official university-survey:

Survey items to be answered before the intervention:
- Competitive preferences and locus of control
- Questions on how much students interact with fellow students
- Prior beliefs about relative performance, separated by ECTS and GPA

Treatment group: Question, whether they want to see their actual quartile for ECTS and GPA, respectively. If yes, they see the treatment. If no, they don't.
Control group: directly continues to next block of questions.

Survey items to be answered after the intervention:
- Satisfaction with the studies
- Study effort in the past and upcoming semester
- Stress and perceived competitiveness in the studies
- Academic self-concept
- Own and perceived prosociality in the studies
Experimental Design Details
Randomization Method
Stratified randomization
Strata: gender, study program, secondary school qualification
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
-
Sample size: planned number of observations
3,000 students
Sample size (or number of clusters) by treatment arms
1,500 students control, 1,500 students treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
see comprehensive Power Analysis in Pre-Analysis plan
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
WU Competence Center for Experimental Research
IRB Approval Date
2022-02-03
IRB Approval Number
WU-HSRP-2022-003
Analysis Plan

Analysis Plan Documents

PAP_relative performance feedback

MD5: c07a45b0f744cf4ed7dcf975858ed490

SHA1: 128169066ca22ad9bef04d2380044c3f50faf74d

Uploaded At: February 18, 2022

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials