Texting Students to Help Achieve Their Goals
Last registered on February 13, 2017

Pre-Trial

Trial Information
General Information
Title
Texting Students to Help Achieve Their Goals
RCT ID
AEARCTR-0000810
Initial registration date
September 03, 2015
Last updated
February 13, 2017 9:30 AM EST
Location(s)
Region
Primary Investigator
Affiliation
University of Toronto
Other Primary Investigator(s)
Additional Trial Information
Status
In development
Start date
2015-09-08
End date
2017-08-16
Secondary IDs
Abstract
Many social scientists and policy makers express concern over low levels of college completion and poor overall academic performance. One explanation, drawing on recent insights from behavioral science, suggests that youth often overemphasize the present or rely too much on routine. Another, drawing on social-psychology, suggests incoming students with weak academic identities (perhaps due to being a first generational or international student) struggle in transitioning to their new environment. This study explores ways to counter these tendencies using online exercises and electronic messaging. Randomly selected students at a 4-year college are randomized into three treatment groups and a control. The first group is given an online goal-setting exercise to think about their future and what steps to take now to help them achieve their goals. The second group is offered additional electronic messages containing advice, information, motivation, and reminders, with the aim of improving performance, experience, and completion. The third group's online exercise involves reading through past 'testimonials' how previous students struggled with their college transition yet persevered and were successful.
External Link(s)
Registration Citation
Citation
Oreopoulos, Philip. 2017. "Texting Students to Help Achieve Their Goals." AEA RCT Registry. February 13. https://www.socialscienceregistry.org/trials/810/history/13958
Experimental Details
Interventions
Intervention(s)
Many social scientists and policy makers express concern over low levels of college completion and poor overall academic performance. One explanation, drawing on recent insights from behavioral science, suggests that youth often overemphasize the present or rely too much on routine. Another approach showing promise has been the use of electronic messages to motivate, remind, or inform individuals at opportune times. This study incorporates these behavioral insights and tests an intervention designed to help improve student performance and experience. Randomly selected students at a 4-year college are given an online goal-setting exercise to think about their future and what steps to take now to help them achieve their goals. Students are subsequently sent electronic messages containing advice, information, motivation, and reminders, with the aim of improving performance, experience, and completion. The results of will be used to develop cost-effective programs that can be scaled up and tested in multiple settings (including high school or 2-year college level and across North America).
Intervention Start Date
2015-09-08
Intervention End Date
2017-05-01
Outcomes
Outcomes (end points)
Key outcomes are course grade, overall grade, course dropout, overall continuation and graduation. Secondary outcomes (if obtained) are self-reported satisfaction of first year experience, feelings of support, feelings of belonging, and access of student services.
Outcomes (explanation)
Key outcomes are from administrative data.

Secondary outcomes will be obtained through survey, ideally in class or through tutorial exercises.

Self reported first year experience: Overall, on a scale from 1 to 7, how pleased are you with your first year experience so far?

Indicate the extent to which you agree to the following statement using a 7 point scale, where 1 = fully agree and 7 = fully disagree: Being a student at X University is an important part of how I see myself

I feel supported as a University X student

What student services have you used this past year (check all that apply)

Experimental Design
Experimental Design
First year students go online to complete a warmup exercise. They register an account, consent, complete an initial survey, and then are randomized into the four groups mentioned above.
Experimental Design Details
All first year economics students at the University of Toronto are asked in their first week to go online and complete a 'warmup exercise' for 2 percent of their final grade. Students register an account and complete a seventeen question survey. They consent to have their data collected be used anonymously for research. They are then randomized into 4 groups.
Randomization Method
Randomization is done by using alternating last digits of student numbers (and it was verified that these are random)
Randomization Unit
Randomization is at the student level
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
no clusters
Sample size: planned number of observations
6000 students
Sample size (or number of clusters) by treatment arms
~30% control group
~20% treatment 1
~30% treatment 2
~20% treatment 3
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The analysis is best split between comparing control with treatment 1 and treatment 2, and comparing control with treatment 1 and treatment 3 For the first comparison, the MDE for treatment 1 relative to the control is a 10% standard deviation effect with the full sample (with 80% power and an alpha of 5%). (same with comparing the two treatments together). For the second comparison, the MDE for treatment 1 relative to treatment 3 is a 12% standard deviation effect with the full sample. In general, it is anticipated that subgroup analysis would be for a subset of more at risk student, about 25% of the full sample. For this subsample the MDE is approximately 23% of a standard deviation.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Toronto Social Sciences, Humanities, and Education Research Ethics Board
IRB Approval Date
2015-07-16
IRB Approval Number
30541
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers