Addressing Barriers to Student Success in Higher Education
Last registered on December 03, 2018

Pre-Trial

Trial Information
General Information
Title
Addressing Barriers to Student Success in Higher Education
RCT ID
AEARCTR-0002744
Initial registration date
September 14, 2018
Last updated
December 03, 2018 1:00 PM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
Oregon State University
Other Primary Investigator(s)
PI Affiliation
Reed College
Additional Trial Information
Status
In development
Start date
2018-09-20
End date
2019-06-30
Secondary IDs
Abstract
More than two of every five students who enroll in college fail to graduate within 6 years. Prior research has identified ineffective study habits as a major barrier to success. Using insights from behavioral economics, this study will assess whether focusing student attention on improving study habits can change behavior and improve outcomes. We will conduct a randomized controlled trial at Oregon State University to assess multiple interventions designed to overcome barriers to college completion. The interventions include encouragement to receive academic coaching on study strategies, to attend academic tutoring, and to increase study effort. We will also randomize the medium, timing, frequency, and offer of a lottery-based incentive for each encouragement, allowing us to measure how message delivery alters student behavior. Our findings will provide evidence on how to increase use of these services and the causal effects of these services.
External Link(s)
Registration Citation
Citation
Pugatch, Todd and Nicholas Wilson. 2018. "Addressing Barriers to Student Success in Higher Education." AEA RCT Registry. December 03. https://www.socialscienceregistry.org/trials/2744/history/38248
Experimental Details
Interventions
Intervention(s)
1. Improved study strategies: encouragement to visit the university Academic Success Center for academic coaching, or the Economics Tutoring Lab for tutoring in Economics courses.
2. Additional study effort: encouragement to complete extra practice problems.

Encouragement will occur via email and text messages, as follows:
1. None: Current standard for notifications [control condition].
2. Email: Emails encouraging use of support service, where number of emails sent will be randomized.
3. Text message: Text encouragement, where number of messages will be randomized.
4. Email with incentive: Students will receive an email indicating that they will be entered in a lottery to receive $250 credit at the campus dining halls and bookstore if they access the support service before a specified date.
5. Text message with incentive: Students will receive the incentive encouragement via text.

Frequency and timing of messages will vary randomly within the 10 weeks of each academic term, as follows:
1. Week 3
2. Week 6
3. Week 9
4. Weeks 3/6
5. Weeks 3/9
6. Weeks 6/9
7. Weeks 3/6/9
Intervention Start Date
2018-10-08
Intervention End Date
2019-06-15
Primary Outcomes
Primary Outcomes (end points)
Take-up of academic services:
--visits to Academic Success Center
--visits to Economics Tutoring Lab
--usage of and performance on extra practice problems
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Student academic performance:
--course grades
--graduation
Secondary Outcomes (explanation)
Course grades will include courses that are subject of intervention (Economics Principles courses) and subsequent Economics courses.
Experimental Design
Experimental Design
We will encourage students to take specific actions to improve their performance in introductory economics courses. All students will be eligible for the full range of academic support services and receive the current standard for notifications, but subsets of students will be encouraged to access particular services. We will also randomize the medium (email or text), timing, frequency, and offer of a lottery-based incentive for each encouragement, allowing us to measure how message delivery alters student behavior. See the description of interventions for additional details.
Experimental Design Details
Not available
Randomization Method
randomization done in office by computer using random number generator
Randomization Unit
individual student enrolled in Economics Principles courses
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
3,000 students enrolled in 12 sections of Economics Principles courses (ECON 201/202 at Oregon State University, Corvallis campus)
Sample size: planned number of observations
3,000 students
Sample size (or number of clusters) by treatment arms
750 students in control group.

Remaining students will be divided evenly between cells defined by encouragement intervention, communication medium, and frequency/timing.

The encouragement interventions are:
1. academic coaching
2. Economics peer tutoring
3. extra practice

The communication media are:
1. email
2. text message
3. email, with lottery incentive
4. text message, with lottery incentive

The frequency/timing combinations during the 10 weeks of the term are:
1. Week 3
2. Week 6
3. Week 9
4. Weeks 3/6
5. Weeks 3/9
6. Weeks 6/9
7. Weeks 3/6/9

Within the subsample of treated students who do not provide a phone number, there are thus 42 possible combinations of treatment characteristics (3 messages x 2 incentives x 7 frequency/timing combinations). Within the subsample of treated students who provide a phone number, these 42 combinations exist for both media, leaving 84 possible combinations. We will divide students as evenly as possible among these treatments, though actual cell sizes may vary due to the large number of treatment combinations.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Minimum detectable effects (MDEs) between any two study arms for our intermediate and main outcomes: • Visit academic coaching or peer tutoring: 3.1 percentage points (assumed baseline mean: 10%) • Complete at least 80% of extra practice problems: 4.1 percentage points (assumed baseline mean: 20%) • Grade of D, F, or Withdraw: 4.5 percentage points (assumed baseline mean: 20%) All power calculations assume a sample size of 3,000 students, a significance level (i.e. alpha) of 10%, power (i.e. beta) of 80%, and study arms of equal size. Specifically, we assume three treatment arms (corresponding to each encouragement message) and a control group of 750 students each. Baseline means are approximations from past sections of introductory economics courses. The MDEs represent comparisons of a single treatment arm versus the control group.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Oregon State University
IRB Approval Date
2018-02-08
IRB Approval Number
8402