Evaluation of High-Touch vs. Low-Touch Interventions to Increase Community College Persistence and Completion

Last registered on November 05, 2021

Pre-Trial

Trial Information

General Information

Title
Evaluation of High-Touch vs. Low-Touch Interventions to Increase Community College Persistence and Completion
RCT ID
AEARCTR-0008483
Initial registration date
November 04, 2021
Last updated
November 05, 2021, 8:24 PM EDT

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
University of Notre Dame

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2020-08-11
End date
2025-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project is a collaboration with Ivy Tech Community College in Indiana to design a program that will increase rates of community college completion in a scalable and college-sustainable format. Ivy Tech Community College of South Bend-Elkhart will enroll students into a support intervention aimed at helping students navigate the complexity of college and overcome potential barriers to completion. To study the effects of the intervention, participating students will be randomly assigned into one of three groups: an arm offering high-touch services, an arm offering low-touch services, or a control group that has access to the usual college services. This study will use a randomized controlled trial to evaluate the two interventions against each other and against the business as usual condition. The study will enroll 360 students over three years into each intervention treatment arms and an additional 360 students into the control group, for a total of 1,080 study participants. We hypothesize that personalized, ongoing support—high-touch services—are necessary for measurable impact on graduation rates. Outcomes evaluated will include college persistence, degree attainment, and future earnings. The study will also investigate treatment effect heterogeneity across different groups of students and include cost/benefit analyses of the high- and low-touch interventions.
External Link(s)

Registration Citation

Citation
Kroeger, Sarah. 2021. "Evaluation of High-Touch vs. Low-Touch Interventions to Increase Community College Persistence and Completion." AEA RCT Registry. November 05. https://doi.org/10.1257/rct.8483-1.0
Experimental Details

Interventions

Intervention(s)
Our project designs and studies both a low-touch and high-touch college success intervention. The high-touch version, named ASPIRE at Ivy Tech, builds on previous research that highlights the importance of individualized support (Weiss et al., 2019; Evans et al., 2020). Students will be matched with a dedicated, personal success coach who will offer ongoing support and referrals to a wide range of available services. In contrast to previous student support programs, the coach will be a college employee who is fully funded by the college and who is fully integrated with other academic advisors and career coaching offices at the institution. Coaches will check in regularly with their students and offer in-person meetings, virtual sessions, calls or text message support. Their primary objective will be to connect students with available services and resources existing on the campus and in the community, but they will also offer ongoing emotional support and accountability.

In order to identify specific mechanisms of impact, a second, low-touch, arm of the experiment will provide the same general information to students via text message, without the personalized coaching. The experimental design of this study will allow the research team to measure the information channel of the intervention separately from the comprehensive, high-touch components.
Intervention Start Date
2020-08-20
Intervention End Date
2023-12-31

Primary Outcomes

Primary Outcomes (end points)
Degree completion, earnings, employment
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
persistence, course accumulation, college GPA
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Our specific research questions are the following:
1. What is the impact of a high-touch, personalized student coaching program on community college persistence, degree attainment, and future earnings?
2. What is the impact of a low-touch, text message-message based support on community college persistence, degree attainment, and future earnings?
3. How does impact and cost-effectiveness compare between the high- and low-touch support interventions?

To investigate these questions, study subjects are randomly assigned into either the high-touch program, the low-touch program, or the control group, with roughly 1/3 of the sample in each group. Students assigned to the high- or low-touch programs receive access to those respective programs, while
students assigned to the control group retain access to standard “business as usual” services offered by Ivy Tech South Bend-Elkhart, including academic advising and student outreach services. Because the high-touch arm will receive the identical stream of text messages in addition access to a success coach, differences between these two arms can be attributed to the individualized support component, rather
than the information contained in the text messages. Both the high-touch and low-touch arms of the intervention will receive the text messaging program. These messages highlight available resources on the campus and in the community and remind students of important upcoming deadlines throughout the academic year. The messages are sent out using Ivy Tech’s existing text-based communication platform. If a student replies to one of the program messages, and automated reply will refer the student to the college’s general help desk number.
Among the high-touch arm, the specific topics covered by individual success coaches will vary. However, coaches are expected to be ready to discuss personal issues, course planning, career planning, financial aid, and any other obstacles to persistence and graduation. Success coaches will also stress the importance of utilizing the available resources on campus and/or in the community, complementing the informational text messages. Coaches will be flexible in finding ways to adapt to COVID-19, using socially-distanced in-person meetings when possible (and when agreed to by both parties) and virtual meetings otherwise. Because the high-touch arm will receive the identical stream of text messages that the low-touch subjects receive, in addition to the access to a success coach, differences between these two arms can be attributed to the individualized support component rather than the information contained in the text messages.

The components of the intervention will be evaluated via a randomized controlled trial conducted by researchers at the University of Maryland, Dartmouth College, and the University of Notre Dame. Any Ivy Tech South Bend-Elkhart student will be considered eligible for study participation if they:
1) are 18 years of age or older,
2) are enrolling at Ivy Tech South Bend-Elkhart for the first time (excluding any coursework completed at Ivy Tech as a high school student), 3) are entering a degree-seeking program,
4) are currently enrolled in at least six non-remedial credit hours.

While the majority of student characteristics will be collected from college administrative records, all study participants will be encouraged to complete the Short Grit Scale (Duckworth and Quinn, 2009). We will analyze responses to determine which characteristics are predictive of either college outcomes or intervention efficacy.

We will measure both academic and labor market outcomes. We will examine intermediate academic outcome of interest in Ivy Tech institutional data at 2, 4 and 6 semesters after enrollment (GPA, credit accumulation, average credit load, and persistence). We will track our primary academic outcome, degree completion, at 4, 6 and 8 semesters after enrollment using data from the National Student Clearinghouse. We will also measure differences in employment and earnings starting two years after enrollment and then annually thereafter, using both Indiana's Management Performance Hub (MPH) and IRS tax data. The research team is already familiar with the logistical process of collecting data on the key outcomes of this experiment from MPH, the National Student Clearinghouse, and IRS tax data.
Experimental Design Details
Not available
Randomization Method
Randomization conducted by the research team using a computer program.
Randomization Unit
The unit of randomization is the individual. The randomization is stratified on gender and race.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1080 students
Sample size: planned number of observations
1080 students
Sample size (or number of clusters) by treatment arms
360 control students
360 low touch treatment
360 high touch treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Our target sample size for the full experiment is 1,080 students, with 360 students in the high-touch intervention, 360 students in the low-touch intervention, and 360 students in the control group. Following Duflo, Glennerster, and Kremer (2007) and assuming a conservative baseline of 25 percent completion within six semesters, we estimate this sample will be powered to detect an effect of 8.3 percentage points or greater. Ivy Tech estimates that on-time completion rates in the absence of any intervention would be closer to 10 percent, which would imply a minimum detectable effect (MDE) of 6.3 points.
IRB

Institutional Review Boards (IRBs)

IRB Name
The University of Notre Dame Institutional Review Board
IRB Approval Date
2020-08-11
IRB Approval Number
20-02-5896