The Effects of Personalized Feedback

Last registered on January 29, 2021
View Trial History

Pre-Trial

Trial Information

General Information

Title
The Effects of Personalized Feedback
RCT ID
AEARCTR-0007105
Initial registration date
January 29, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 29, 2021, 3:19 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
Esteban Aucejo
Affiliation
Arizona State University
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2021-01-11
End date
2022-05-01
Keywords
Education
Additional Keywords
JEL code(s)
I2
Secondary IDs
Abstract
The purpose of this study is to investigate the impact of giving personalized email feedback to students attending a large public institution in the US. We aim to learn more about what type of feedback is the most impactful on student learning and retention (i.e., what information is more likely to affect student behavior). We also plan to assess heterogeneity on how different group of students (i.e. first generation, gender group, field of study, type of instruction) respond to feedback after receiving their exam scores. To assess the impact of feedback, we will not only look at educational outcomes but we will also elicit from the students how their beliefs have changed after receiving their exam scores and personalized feedback.

External Link(s)

Registration Citation

Citation
Aucejo, Esteban. 2021. "The Effects of Personalized Feedback." AEA RCT Registry. January 29. https://doi.org/10.1257/rct.7105-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We will send email to students (that attend introductory classes in economics) with different feedback messages. The content of these messages will depend on students' performance in the exams. These emails will be submitted at random conditional on performance. We also plan to administer several surveys to students in order to gather background information and to understand how they update their beliefs about their own abilities.
Intervention Start Date
2021-02-02
Intervention End Date
2021-05-03

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes that we plan to analyze are: final grade in the class, whether the student dropouts from the class, whether the student changes major/field of study, whether the student dropouts from university (conditional on being able to have access to this information), whether the student updates beliefs regarding own abilities, and whether the student believes that receiving certain type of feedback is helpful.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Students attending a large public higher education institution will receive different feedback messages (at random) conditional on their exam performance. The goal is to determine how different types of feedback influence student outcomes. We will focus on students enrolled in introductory classes in economics.
Experimental Design Details
The feedback messages will have different components: (1) information about students relative performance in the class, (2) information on the share of students in the class that fail in predict own performance at the beginning of the term (i.e. students at the beginning of the term are surveyed about their expected grade in the class, and therefore we will provide them information about the share of students that over predict their performance conditional on similar performance in the test), and (3) encouragement statements that depend on student performance in the exam. In total, we will have a control group and three treatments, where the treatments are:
A) (1)
B) (1) + (2)
C) (1) + (2) + (3)

The control group will also receive an email but the information will mainly remind students that they can attend for example, office hours if they want to do so.
Randomization Method
The randomization will be done in office by a computer using the Stata command "stratarand".
Randomization Unit
Randomization will be done at the student level conditional on attending the same class (we have approximately 18 classes in our experiment). Students will receive a feedback email at random conditional on their performance in the class. Assignment to treatment will be done by stratifying the sample.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
No clusters, we will randomize students into the tretament and control groups within classroom. We plan to perform the randomizatoin on approximately 18 classrooms.
Sample size: planned number of observations
Our sample involves approximately 1350 students that are divided across approximately 18 different classrooms.
Sample size (or number of clusters) by treatment arms
Approximately 300 students will be randomly allocated to each of the three treatment groups and 300 students in the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Arizona State University
IRB Approval Date
2020-12-29
IRB Approval Number
STUDY00013095
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials