Texting Students to Help Achieve Their Goals

Last registered on January 12, 2018

Pre-Trial

Trial Information

General Information

Title
Texting Students to Help Achieve Their Goals
RCT ID
AEARCTR-0000810
Initial registration date
September 03, 2015

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 03, 2015, 11:18 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
January 12, 2018, 9:17 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Toronto

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2015-09-08
End date
2017-08-16
Secondary IDs
Abstract
Many social scientists and policy makers express concern over low levels of college completion and poor overall academic performance. One explanation, drawing on recent insights from behavioral science, suggests that youth often overemphasize the present or rely too much on routine. Another, drawing on social-psychology, suggests incoming students with weak academic identities (perhaps due to being a first generational or international student) struggle in transitioning to their new environment. This study explores ways to counter these tendencies using online exercises and electronic messaging. Randomly selected students at a 4-year college are randomized into three treatment groups and a control. The first group is given an online goal-setting exercise to think about their future and what steps to take now to help them achieve their goals. The second group is offered additional electronic messages containing advice, information, motivation, and reminders, with the aim of improving performance, experience, and completion. The third group's online exercise involves reading through past 'testimonials' how previous students struggled with their college transition yet persevered and were successful.
External Link(s)

Registration Citation

Citation
Oreopoulos, Philip. 2018. "Texting Students to Help Achieve Their Goals." AEA RCT Registry. January 12. https://doi.org/10.1257/rct.810-5.0
Former Citation
Oreopoulos, Philip. 2018. "Texting Students to Help Achieve Their Goals." AEA RCT Registry. January 12. https://www.socialscienceregistry.org/trials/810/history/24890
Experimental Details

Interventions

Intervention(s)
Many social scientists and policy makers express concern over low levels of college completion and poor overall academic performance. One explanation, drawing on recent insights from behavioral science, suggests that youth often overemphasize the present or rely too much on routine. Another approach showing promise has been the use of electronic messages to motivate, remind, or inform individuals at opportune times. This study incorporates these behavioral insights and tests an intervention designed to help improve student performance and experience. Randomly selected students at a 4-year college are given an online goal-setting exercise to think about their future and what steps to take now to help them achieve their goals. Students are subsequently sent electronic messages containing advice, information, motivation, and reminders, with the aim of improving performance, experience, and completion. The results of will be used to develop cost-effective programs that can be scaled up and tested in multiple settings (including high school or 2-year college level and across North America).
Intervention Start Date
2015-09-08
Intervention End Date
2017-05-01

Primary Outcomes

Primary Outcomes (end points)

Key outcomes are course grade, overall grade, course dropout, overall continuation and graduation.

Secondary outcomes (if obtained) are self-reported satisfaction of first year experience, feelings of support, feelings of belonging, and access of student services.
Primary Outcomes (explanation)
Key outcomes are from administrative data.

Secondary outcomes will be obtained through survey, ideally in class or through tutorial exercises.

Self reported first year experience: Overall, on a scale from 1 to 7, how pleased are you with your first year experience so far?

Indicate the extent to which you agree to the following statement using a 7 point scale, where 1 = fully agree and 7 = fully disagree: Being a student at X University is an important part of how I see myself

I feel supported as a University X student

What student services have you used this past year (check all that apply)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
First year students go online to complete a warmup exercise. They register an account, consent, complete an initial survey, and then are randomized into the four groups mentioned above.
Experimental Design Details
All first year economics students at the University of Toronto are asked in their first week to go online and complete a 'warmup exercise' for 2 percent of their final grade.

Students register an account and complete a seventeen question survey. They consent to have their data collected be used anonymously for research.

They are then randomized into 4 groups.
Randomization Method
Randomization is done by using alternating last digits of student numbers (and it was verified that these are random)
Randomization Unit
Randomization is at the student level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
no clusters
Sample size: planned number of observations
6000 students
Sample size (or number of clusters) by treatment arms
~30% control group
~20% treatment 1
~30% treatment 2
~20% treatment 3
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The analysis is best split between comparing control with treatment 1 and treatment 2, and comparing control with treatment 1 and treatment 3 For the first comparison, the MDE for treatment 1 relative to the control is a 10% standard deviation effect with the full sample (with 80% power and an alpha of 5%). (same with comparing the two treatments together). For the second comparison, the MDE for treatment 1 relative to treatment 3 is a 12% standard deviation effect with the full sample. In general, it is anticipated that subgroup analysis would be for a subset of more at risk student, about 25% of the full sample. For this subsample the MDE is approximately 23% of a standard deviation.
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Toronto Social Sciences, Humanities, and Education Research Ethics Board
IRB Approval Date
2015-07-16
IRB Approval Number
30541
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials