Texting Students to Help Achieve Their Goals

Last registered on January 12, 2018

View Trial History

Pre-Trial

Trial Information

General Information

Title

Texting Students to Help Achieve Their Goals

RCT ID

AEARCTR-0000810

Initial registration date

September 03, 2015

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

September 03, 2015, 11:18 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

January 12, 2018, 9:17 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

Canada

Region

Toronto

Primary Investigator

Name

Philip Oreopoulos

Affiliation

University of Toronto

Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status

Completed

Start date

2015-09-08

End date

2017-08-16

Keywords

Education

Additional Keywords

JEL code(s)

Secondary IDs

Abstract

Many social scientists and policy makers express concern over low levels of college completion and poor overall academic performance. One explanation, drawing on recent insights from behavioral science, suggests that youth often overemphasize the present or rely too much on routine. Another, drawing on social-psychology, suggests incoming students with weak academic identities (perhaps due to being a first generational or international student) struggle in transitioning to their new environment. This study explores ways to counter these tendencies using online exercises and electronic messaging. Randomly selected students at a 4-year college are randomized into three treatment groups and a control. The first group is given an online goal-setting exercise to think about their future and what steps to take now to help them achieve their goals. The second group is offered additional electronic messages containing advice, information, motivation, and reminders, with the aim of improving performance, experience, and completion. The third group's online exercise involves reading through past 'testimonials' how previous students struggled with their college transition yet persevered and were successful.

External Link(s)

Registration Citation

Citation

Oreopoulos, Philip. 2018. "Texting Students to Help Achieve Their Goals." AEA RCT Registry. January 12. https://doi.org/10.1257/rct.810-5.0

Former Citation

Oreopoulos, Philip. 2018. "Texting Students to Help Achieve Their Goals." AEA RCT Registry. January 12. https://www.socialscienceregistry.org/trials/810/history/24890

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Many social scientists and policy makers express concern over low levels of college completion and poor overall academic performance. One explanation, drawing on recent insights from behavioral science, suggests that youth often overemphasize the present or rely too much on routine. Another approach showing promise has been the use of electronic messages to motivate, remind, or inform individuals at opportune times. This study incorporates these behavioral insights and tests an intervention designed to help improve student performance and experience. Randomly selected students at a 4-year college are given an online goal-setting exercise to think about their future and what steps to take now to help them achieve their goals. Students are subsequently sent electronic messages containing advice, information, motivation, and reminders, with the aim of improving performance, experience, and completion. The results of will be used to develop cost-effective programs that can be scaled up and tested in multiple settings (including high school or 2-year college level and across North America).

Intervention Start Date

2015-09-08

Intervention End Date

2017-05-01

Primary Outcomes

Primary Outcomes (end points)

Key outcomes are course grade, overall grade, course dropout, overall continuation and graduation.

Secondary outcomes (if obtained) are self-reported satisfaction of first year experience, feelings of support, feelings of belonging, and access of student services.

Primary Outcomes (explanation)

Key outcomes are from administrative data.

Secondary outcomes will be obtained through survey, ideally in class or through tutorial exercises.

Self reported first year experience: Overall, on a scale from 1 to 7, how pleased are you with your first year experience so far?

Indicate the extent to which you agree to the following statement using a 7 point scale, where 1 = fully agree and 7 = fully disagree: Being a student at X University is an important part of how I see myself

I feel supported as a University X student

What student services have you used this past year (check all that apply)

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

First year students go online to complete a warmup exercise. They register an account, consent, complete an initial survey, and then are randomized into the four groups mentioned above.

Experimental Design Details

All first year economics students at the University of Toronto are asked in their first week to go online and complete a 'warmup exercise' for 2 percent of their final grade.

Students register an account and complete a seventeen question survey. They consent to have their data collected be used anonymously for research.

They are then randomized into 4 groups.

Randomization Method

Randomization is done by using alternating last digits of student numbers (and it was verified that these are random)

Randomization Unit

Randomization is at the student level

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

no clusters

Sample size: planned number of observations

6000 students

Sample size (or number of clusters) by treatment arms

~30% control group
~20% treatment 1
~30% treatment 2
~20% treatment 3

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

The analysis is best split between comparing control with treatment 1 and treatment 2, and comparing control with treatment 1 and treatment 3 For the first comparison, the MDE for treatment 1 relative to the control is a 10% standard deviation effect with the full sample (with 80% power and an alpha of 5%). (same with comparing the two treatments together). For the second comparison, the MDE for treatment 1 relative to treatment 3 is a 12% standard deviation effect with the full sample. In general, it is anticipated that subgroup analysis would be for a subset of more at risk student, about 25% of the full sample. For this subsample the MDE is approximately 23% of a standard deviation.

Supporting Documents and Materials

IRB