Success Coaches as a Strategy to Mitigate Pandemic-Related Educational Disruptions: Evidence from a Field Experiment at Two Tennessee Community Colleges

Last registered on August 18, 2022

Pre-Trial

Trial Information

General Information

Title
Success Coaches as a Strategy to Mitigate Pandemic-Related Educational Disruptions: Evidence from a Field Experiment at Two Tennessee Community Colleges
RCT ID
AEARCTR-0009920
Initial registration date
August 15, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 18, 2022, 3:19 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Tennessee Board of Regents

Other Primary Investigator(s)

PI Affiliation
Tennessee Board of Regents
PI Affiliation
Tennessee Board of Regents

Additional Trial Information

Status
In development
Start date
2022-08-24
End date
2025-01-31
Secondary IDs
R305S220005
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Low first-year persistence rates at Tennessee community colleges pose challenges for the state's educational attainment and workforce development goals. Historically, only about 75% of first-time students at Tennessee community colleges persist beyond their first semester. The COVID-19 pandemic exacerbated these persistence challenges at community colleges, as first-semester persistence fell eight percentage points, from 77% for the 2019 cohort to 69% for the 2020 cohort. Persistence rates are lowest for academically underprepared students. Among underprepared students who were placed into corequisite learning support courses (i.e., remedial or developmental courses) for all three subject areas (i.e., math, reading, and writing) in fall 2020, only 55% persisted to the following semester (a 13 percentage point decrease from 2019 to 2020). To address these widening gaps in persistence between academically underprepared students and their college-ready peers, we will implement a high-dosage coaching intervention embedded into learning support courses at two pilot colleges (Jackson State Community College and Northeast State Community College). These coaches will have a caseload of no more than 150 students, which is substantially lower than the average caseload of 569 students per relational advisor at Tennessee community colleges. Coaches will provide frequent, personalized, and in-depth outreach during students’ first year of enrollment. They will meet with students three times during their first semester (either in person or virtually) and will reach out at least weekly via text message and email, with the goals of connecting students with campus resources, helping them identify programs of study that are most relevant to their goals, and supporting their progress through learning support. Students are eligible to participate in the intervention if they meet the following criteria: first-time student, recent high school graduate, over age 18, placed into one or more learning support courses, and not receiving enhanced advising from another program on campus. We will provide coaching for two cohorts (i.e., students beginning in fall 2022 and fall 2023), and students will receive coaching during the fall and spring semesters of their first year. For each cohort, we will randomize students into a treatment group and a control group after stratifying by college (i.e., site) and number of learning support placements (i.e., placement into 1, 2, or 3 learning support courses). Randomization occurs at the individual level. We will select up to 450 students for treatment at Jackson and up to 300 students for treatment at Northeast to maintain a caseload of no more than 150 students per coach. The remaining students who meet the eligibility criteria will be assigned to the control group. Based on historic enrollment data, we expect that there will be approximately 300 students in the treatment group and 208 students in the control group at Jackson. We expect that there will be approximately 300 students in the treatment group and 345 students in the control group at Northeast. Our primary outcomes of interest are fall-to-spring persistence, fall-to-fall persistence, credit attainment in the first year, and college grade point average (GPA). We will treat all other outcomes and subgroup analyses as exploratory.

Registration Citation

Citation
Deaton, Russ, Madison Dell and Amy Moreland. 2022. "Success Coaches as a Strategy to Mitigate Pandemic-Related Educational Disruptions: Evidence from a Field Experiment at Two Tennessee Community Colleges." AEA RCT Registry. August 18. https://doi.org/10.1257/rct.9920-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-08-29
Intervention End Date
2024-05-31

Primary Outcomes

Primary Outcomes (end points)
To assess the impact of the intervention, we will focus on five primary outcomes of interest:
1. Fall-to-spring persistence: a binary indicator of whether a student was enrolled at any Tennessee community college in the spring semester of their first year
2. Fall-to-fall persistence: a binary indicator of whether a student was enrolled at any Tennessee community college in the fall semester of their second year
3. Binary credit attainment in the first semester: a binary indicator of whether the student earned any credit hours in their first semester of enrollment
4. Continuous credit attainment in the first year: a continuous measure of credit hours earned in the fall and spring semesters of the student’s first year
5. College grade point average (GPA) in the first year: a continuous measure of GPA in the fall and spring semesters of the student’s first year
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
At the end of the registration period for fall 2022 and fall 2023, each of the two pilot colleges will send us a list of all students registered for fall courses. After restricting the sample to the students who meet our eligibility criteria, we will randomize students into a treatment group and a control group using random number generation in Stata. Randomization will be stratified by college (i.e., site). We will select up to 450 students for treatment at Jackson and up to 300 students for treatment at Northeast to maintain a caseload of no more than 150 students per coach. The remaining students who meet the eligibility criteria will be assigned to the control group. Based on historic enrollment data, we expect that there will be approximately 300 students in the treatment group and 208 students in the control group at Jackson. At Northeast, we expect that there will be approximately 300 students in the treatment group and 345 students in the control group. We will also stratify by the student’s number of learning support course placements (i.e., placement into remedial or developmental courses) to ensure that the treatment and control groups are balanced on students’ academic preparedness. Students can place into learning support for math, reading, and/or writing, resulting in three strata (i.e., placement into 1, 2, or 3 learning support courses). We will account for this stratification in our analysis by including (cohort * site * number of learning support placements) fixed effects in our models.
Experimental Design Details
Not available
Randomization Method
We will randomize students into a treatment group and a control group using random number generation in Stata. We will randomly assign up to 450 students to the treatment group at Jackson and 300 students to the treatment group at Northeast. The remaining students who meet our eligibility criteria will be assigned to the control group.
Randomization Unit
Randomization will occur at the individual level, after stratifying by college (i.e., site) and number of learning support course placements. (Students can place into 1, 2, or 3 learning support courses.)
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
Based on historic enrollment data, we anticipate having approximately 1,153 total observations: 508 at Jackson and 645 at Northeast. At Jackson, we expect there to be approximately 300 students in the treatment group and 208 students in the control group. At Northeast, we expect that there will be approximately 300 students in the treatment group and 345 students in the control group.
Sample size (or number of clusters) by treatment arms
There is only one treatment arm in this trial, so there will be up to 750 students in the treatment arm (450 at Jackson and 300 at Northeast). Based on historic enrollment data, we expect there will be approximately 600 students in the treatment group (300 at Jackson and 300 at Northeast).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Tennessee Board of Regents Institutional Review Board (TBR IRB)
IRB Approval Date
2022-06-06
IRB Approval Number
tbr_202205_01