Back to History Current Version

Normatively Framed Relative Performance Feedback in Higher Education

Last registered on March 31, 2021
View Trial History

Pre-Trial

Trial Information

General Information

Title
Normatively Framed Relative Performance Feedback in Higher Education
RCT ID
AEARCTR-0007442
Initial registration date
March 30, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 31, 2021, 10:43 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Germany
Region

Primary Investigator

Name
Raphael Brade
Affiliation
ifo Institute
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Robert Jäckle
PI Affiliation
Nuremberg Institute of Technology
Contact Investigator
PI Name
Oliver Himmler
PI Affiliation
University of Erfurt, Max Planck Institute for Research on Collective Goods
Contact Investigator

Additional Trial Information

Status
On going
Start date
2020-10-01
End date
2026-09-30
Keywords
Education
Additional Keywords
Higher Education, Relative Performance Feedback
JEL code(s)
I23, C93
Secondary IDs
Abstract
Little is known about how relative performance feedback in higher education based on performance measures such as obtained credits can be designed in order to increase academic achievement of students across the entire performance distribution. In addition, past research was often conducted with students from specific study programs. Against this background, we will assign students from 21 different study programs to different types of relative performance feedback and track their study progress in the short and in the long run.
External Link(s)

Registration Citation

Citation
Brade, Raphael, Oliver Himmler and Robert Jäckle. 2021. "Normatively Framed Relative Performance Feedback in Higher Education." AEA RCT Registry. March 31. https://doi.org/10.1257/rct.7442-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
The intervention will be conducted with a cohort of university students studying towards 21 different degrees, who are still enrolled in their initial study program at the beginning of the second term. In all experimental groups, students will be eligible to receive feedback twice per semester.
Intervention (Hidden)
Feedback will be mailed (physical letters) from the second semester onward until the end of the regular study duration of seven semesters. The letters will include a combination of the following treatments:

Absolute performance feedback (APF): Letters with absolute performance feedback include information about the student’s own performance in terms of obtained credit points and grade point average (GPA). This information can also be accessed by students via the web portal of the university.

Relative performance feedback (RPF): Letters with relative performance feedback compare students’ own performance in terms of obtained credits to the performance of other students in the same study program. This information is not available to students, via the web portal of the university or otherwise. The letters also quote their GPA, but there is no comparison to the GPA of other students.

Normative message: Depending on the position in the performance distribution students receive different types of normative messages that aim to encourage them or that approve of their previous performance.

Opt-out: All letters include information that students can opt-out of receiving further feedback letters in the future by sending an email. Additionally, once at the beginning of the intervention this will be made particularly salient to students, in order to measure avoidance of further feedback.

Opt-in: All letters include information that students can opt-out of receiving further letters in the future by sending an email. Additionally, once at the beginning of the intervention, students will be informed that they must send an email if they want to receive further letters in the future. This is to measure students’ demand for further feedback.
Intervention Start Date
2021-03-31
Intervention End Date
2024-03-14

Primary Outcomes

Primary Outcomes (end points)
We will study effects on academic achievement in the short and the long run, leading to two sets of primary outcomes.

Short run outcomes: We measure academic achievement on the dimension that we give performance feedback on, i.e., we will study effects on obtained credits at the end of the first treatment semester. In addition, we will measure demand for and avoidance of further feedback as well as the effects of repeated treatment on obtained credits in the second treatment semester.

Long run: Because credit accumulation is directly linked to degree attainment, in the long run, we plan to study whether our treatments raise graduation rates and/or lower the time to degree.
Primary Outcomes (explanation)
We plan to study whether effects on academic achievement after the first treatment semester depend on the demand for and avoidance of further feedback.

Secondary Outcomes

Secondary Outcomes (end points)
Both in the short and the long run we plan to study whether students trade off performance increases on the primary outcome dimension with decreases on other outcome dimensions.

Short run: Other short-run outcomes are the grade point average (GPA), dropout, and non-cognitive outcomes measured by online surveys.

Long run: Other long-run outcomes are credit accumulation, dropout, GPA, and non-cognitive outcomes measured by online surveys.
Secondary Outcomes (explanation)
Both in the short and the long run we plan to study effects on students’ beliefs about their relative performance measured by online surveys.

Experimental Design

Experimental Design
Students will be randomly allocated to control and four treatment groups.
Experimental Design Details
Reflecting the different interventions described above, students will randomly be allocated to one of the following five groups:
1) APF + Opt-out (=Control group)
2a) RPF + Opt-out:
2b) RPF + Opt-in:
3a) RPF + Normative message + Opt-out:
3b) RPF + Normative message + Opt-in:
Randomization Method
Randomization was performed with the software Stata using stratification and re-randomization.
Randomization Unit
Student-level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
2370 students
Sample size: planned number of observations
2370 students
Sample size (or number of clusters) by treatment arms
Group 1: 597
Group 2a: 591
Group 2b: 295
Group 3a: 592
Group 3b: 295
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethics Committee of the University of Erfurt
IRB Approval Date
2021-02-24
IRB Approval Number
N/A
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials