Improving the college experience: An encouragement experiment

Last registered on January 27, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
Improving the college experience: An encouragement experiment
RCT ID
AEARCTR-0008908
Initial registration date
January 27, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 27, 2022, 4:40 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Danila Serra
Affiliation
Texas A&M University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Daniel Gomez-Vasquez
PI Affiliation
Texas A&M University
Contact Investigator
PI Name
Catherine Porter
PI Affiliation
Lancaster University and Young Lives
Contact Investigator

Additional Trial Information

Status
On going
Start date
2021-10-11
End date
2025-01-31
Keywords
Behavior, Education, Gender
Additional Keywords
economics, diversity and inclusion
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Women and students from minority groups are severely under-represented in the field of economics. Previous studies have documented that under-represented university students are less likely to feel like they “belong” in their field and they tend to score lower in measures of growth mindset (e.g., Bayer et al. 2020 AEA P&P). This might contribute to differences in course performance, majoring choices and graduation rates. In this project, we implement a field experiment to investigate whether weekly encouraging emails sent out by the Department of Economics, and centered around the message “You belong here and you can do this,” have a significant impact on the college experience – broadly defined – of economics majors, and in particular of women and under-represented minority students. Additionally, we test whether including in the emails profiles of a diverse set of young alumni (peer role models), and an additional encouraging message from the alumni, has a significant impact on students.
External Link(s)

Registration Citation

Citation
Gomez-Vasquez, Daniel, Catherine Porter and Danila Serra. 2022. "Improving the college experience: An encouragement experiment." AEA RCT Registry. January 27. https://doi.org/10.1257/rct.8908-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We implement a field experiment involving all current economics majors at a large public university where students declare their major at enrollment. We randomly assigned students either to a control group or to one of 3 treatment conditions. The control group does not receiving any emails as part of the intervention. The remaining students receive 6 encouraging emails on behalf of the Department of Economics, one per week, starting mid-semester and ending during finals week. The emails emphasize that the students belong in the department and they "can do it." In addition, in two of the three treatments, each weekly email also includes the profile of a recent graduate, his or her photo, graduation year, and current job. Finally, in one of these two role model treatments, the emails include an encouraging message from the profiled alumni.
Intervention Start Date
2021-11-15
Intervention End Date
2021-12-17

Primary Outcomes

Primary Outcomes (end points)
About 2 months after the receipt of the last encouragement email, all students will be invited to participate in an online survey. This will have the purpose to generate measures of:
a. Sense of belonging;
b. Growth mindset;
c. Self-efficacy;
d. Emotional well-being;
e. Intention to graduate with an economics major (i.e., not to change major).

We also aim to follow this cohort of economics majors over time and use administrative data to measure:
a. Number of economics classes taken in Spring 2022 (semester after the intervention), and following semesters (if still enrolled);
b. Grades obtained in economics classes, in Fall 2021 (semester of the intervention), Spring 2021 (semester after the intervention), and following semesters;
c. Likelihood to major in economics (not to change major).
Primary Outcomes (explanation)
The survey-generated measures of belonging, self-efficacy, growth-mindset and emotional well-being will be aggregated indexes of four different sets of survey questions. We will also further aggregate these four indexes to generate a comprehensive "college experience index."

Secondary Outcomes

Secondary Outcomes (end points)
Use of support resources provided by the Department of Economics and by the university.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We randomly assign students either to a control group or to one of 3 treatment conditions, described below:
• T1: Control: Students received no encouragement emails;
• T2: Encouragement emails: Starting from the week of November 15, 2021, students received 6 weekly emails sent on behalf of the Department of Economics, with the main message: “you belong here and you can do this”;
• T3: Encouragement emails plus peer role models: Same as T2 but each email also includes the profile (photo, name, graduation year and current job) of an economics graduate from the class of 2018 or the class of 2019;
• T4: Encouragement emails plus peer role model encouragement: Same as T3, but the student profile also includes an encouragement message from the alumni.
Experimental Design Details
Not available
Randomization Method
Stratified randomization, at the individual student level. Strata are: gender, Hispanic ethnicity, non-white race and course year.
Randomization Unit
Individual-level randomization.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
All current majors, as of Fall 2021, are included in the study. The only excluded students are those who had registered to graduate in December 2021. We have a total of 1108 economics majors in the sample.
Sample size: planned number of observations
1108 students. We anticipate the sample size for the analysis of survey-generated outcomes to be lower than 1108, as it will depend on the survey response rate.
Sample size (or number of clusters) by treatment arms
• T1 - Control: 275 students;
• T2 - Encouragement emails: 281 students;
• T3 - Encouragement emails + Peer role models: 278 students;
• T4 - Encouragement emails + Peer role model encouragement: 274 students.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Texas A&M IRB
IRB Approval Date
2021-11-11
IRB Approval Number
IRB2021-1351D
Analysis Plan