College Success and Personalized Feedback

Last registered on January 17, 2025

Pre-Trial

Trial Information

General Information

Title
College Success and Personalized Feedback
RCT ID
AEARCTR-0015150
Initial registration date
January 16, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 17, 2025, 7:28 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Arizona State University

Other Primary Investigator(s)

PI Affiliation
Arizona State University
PI Affiliation
Arizona State University

Additional Trial Information

Status
In development
Start date
2025-01-20
End date
2025-05-15
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study is designed to create and administer a survey that identifies key areas of struggle for students, including financial stress, academic success, social life, and study habits. Based on the survey results, we will generate personalized emails for treated participant, providing targeted advice and resources to address their primary area of concern. The objective is to evaluate the effectiveness of these personalized interventions by examining their impact on various student outcomes, such as academic scores and personal perceptions of their struggles.
External Link(s)

Registration Citation

Citation
Affonso Peyre , Agustina, Esteban Aucejo and Stephanie El Khoury. 2025. "College Success and Personalized Feedback." AEA RCT Registry. January 17. https://doi.org/10.1257/rct.15150-1.0
Experimental Details

Interventions

Intervention(s)
Students at Arizona State University that have enrolled in our study will be randomly assigned into receiving feedback and advice around the area concerning their academic journey that they are scoring the worst in. This includes information on their study habits, academic life, social life in college, or financial situation. We use our survey to identify the area in which a student struggles the most on an individual level, allowing us to provide personalized feedback tailored to their experience.
Intervention Start Date
2025-03-03
Intervention End Date
2025-05-15

Primary Outcomes

Primary Outcomes (end points)
Scores on exams and scores on feelings concerning struggles that they report to us, scores of "how are you feeling" question that is asked half way through the experiment. In addition to this, we aim to look at the heterogeneous effects of treatment by looking at first generation students, GPA, dummy for being above/below average score on areas, and gender.
Primary Outcomes (explanation)
the scores on the questions asked in the survey are constructed by assigning the questions that are relevant in each area on a scale of 1 to 5, where 1 refers to the student doing bad and 5 refers to the student doing well. The overall score for each area is determined by averaging with equal weights the scores of all the questions within that area. These questions are chosen from preexisting questionnaires in psychology and sociology that deal with the areas that we are interested in testing.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
1.At the beginning of the semester, the research team and professors will inform students about a survey available for participation in a study conducted by ASU professors, contingent upon their consent. Instructors are encouraged to offer extra credit to students who complete the survey, with the amount varying by instructor. However, students may opt out of the study and still earn extra credit by completing the survey.

2. Based on the responses from the initial survey, students are grouped according to the area in which they performed the poorest. Within each group, students are stratified into treatment and control groups based on whether their score in their weakest area is above or below the average score for that area (calculated based on students with the same weakest area)., their first-generation status, and their GPA (below or above the average GPA of the student population at ASU). If the sample size is too small to maintain the number of desired strata, grouping will be simplified to a dummy for above/below average score in the weakest area and GPA.

3. Using the collected information, the research team will then send personalized messages to treated students about the area that they scored the lowest in. Students in the treatment group will receive personalized emails detailing their scores across all areas, along with a link to a video tailored to the area they scored lowest in. This email will be sent after their first exam. If students achieve perfect scores in all areas or tie in their lowest scores across multiple areas, a "priority list" dictates the video that will be sent to students: the academic area is first, followed by study habits, social life, and finally financial issues. For example, a student with perfect scores across the board will receive the academic video, while a student scoring lowest in both study habits and social life will receive the study habits video.

4. Following the distribution of personalized information, all students will receive a follow-up email to gauge their feelings about the areas discussed in the survey.

5. Later, treated students will receive a reminder email highlighting the advice most relevant to their needs. Where the focus is on their worst area and the sub-areas that exist within it. At this stage all videos relevant to this area are shared as individual links with treated students.

A second survey will be administered at the semester's end to assess how students' perceptions of their college struggles have evolved.
Experimental Design Details
Not available
Randomization Method
Randomization done in the office on a computer using STATA.
Randomization Unit
Individual level (student)
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1,500 students
Sample size: planned number of observations
1,500 students
Sample size (or number of clusters) by treatment arms
750 treated students, 750 control students
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Knowledge Enterprise Research Integrity and Assurance
IRB Approval Date
2025-01-06
IRB Approval Number
STUDY00021401