Pathways to Success for Adult Learners at Two Tennessee Community Colleges: Evidence from a Field Experiment

Last registered on January 23, 2023

Pre-Trial

Trial Information

General Information

Title
Pathways to Success for Adult Learners at Two Tennessee Community Colleges: Evidence from a Field Experiment
RCT ID
AEARCTR-0010784
Initial registration date
January 16, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 23, 2023, 6:45 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Tennessee Board of Regents

Other Primary Investigator(s)

PI Affiliation
Tennessee Board of Regents
PI Affiliation
Tennessee Board of Regents

Additional Trial Information

Status
In development
Start date
2023-01-20
End date
2026-05-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Over the past decade, adult students have constituted between 24% and 36% of fall enrollment in the Tennessee community college system. However, many of these students have struggled to persist and complete a credential. Three-year graduation rates for adult students have held steady at approximately 23% for the last four years. Adults who complete a credential stand to realize significant benefits to their employment prospects and earnings, but too few adults persist to completion to experience these benefits. To address the challenges adult students face on their path to a college credential, we will partner with two colleges (Jackson State Community College and Northeast State Community College) to redesign the experience of adults in their first year of college, including students who are new to college and students who are returning to college after time away. The intervention includes two components: redesigned general education courses and success coaches who are embedded within those courses. We will offer stipends to five faculty members per year at each partner college to incorporate adult-friendly teaching strategies into the general education courses they teach. Half of the eligible adult students who enroll in these redesigned courses will be randomly assigned to coaching. Coaches will have a caseload of no more than 150 students, which is substantially lower than the average caseload of 569 students per relational advisor at Tennessee community colleges. Coaches will meet with students two times in each semester (either in person or virtually) and will reach out at least weekly via text message and email, with the goals of connecting students with campus resources, helping them identify majors and careers that are most relevant to their goals, and supporting their academic progress. For students enrolled in the redesigned courses, coaches will work closely with instructors to monitor the progress of their assigned students and proactively intervene as students encounter difficulties. Because we will estimate the impact of each component of the intervention separately as well as the two components together, the experiment will have three treatment arms: (1) coaching only; (2) course redesign only; and (3) coaching and course redesign. Students are eligible to participate in the intervention if they meet the following criteria: enrolled as a degree- or certificate-seeking student at Jackson State or Northeast State; graduated from high school more than five years ago; either in their first-ever semester of college or in their first semester back in college after a break of at least one semester (excluding summer); and not part of any special populations that will be excluded from the sample (i.e., students who have access to enhanced advising through another program on campus and students in prison education programs). We will provide coaching for five cohorts (i.e., students beginning in each of the fall and spring semesters between spring 2023 and spring 2025), and students will receive coaching for a full academic year. Spring 2023 will serve as a pilot of the intervention, and data from this semester will not be included in the experimental evaluation. For each cohort, we will randomize students into the treatment and control conditions after stratifying by college (i.e., site), student type (i.e., first-time or returning student), and enrollment in a redesigned course section. Randomization occurs at the individual level. Across the two years, we will assign up to 600 students to the two coaches at Jackson State and up to 900 students to the three coaches at Northeast State to maintain a caseload of no more than 150 students per coach in a year. Based on enrollment data from the most recent academic year and expected adult enrollment in the redesigned courses, we estimate that there will be approximately 650 students in the treatment arms and 416 students in the control group at Jackson State. We expect that there will be approximately 950 students in the treatment arms and 768 students in the control group at Northeast State. Our primary outcomes of interest are course success rates, semester-to-semester persistence, year-to-year persistence, and two-year completion rates. We will treat all other outcomes and subgroup analyses as exploratory.
External Link(s)

Registration Citation

Citation
Deaton, Russ, Madison Dell and Amy Moreland. 2023. "Pathways to Success for Adult Learners at Two Tennessee Community Colleges: Evidence from a Field Experiment." AEA RCT Registry. January 23. https://doi.org/10.1257/rct.10784-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2023-01-23
Intervention End Date
2025-05-31

Primary Outcomes

Primary Outcomes (end points)
To assess the impact of the intervention, we will focus on four primary outcomes of interest:
1. Course success rate: a binary indicator equal to 1 if a student received a grade of C or better in a course
2. Semester-to-semester persistence: a binary indicator equal to 1 if a student was enrolled at any Tennessee community college in the next fall or spring semester following their first semester of enrollment (or earned a credential in the intervening period)
3. Year-to-year persistence: a binary indicator equal to 1 if a student was enrolled at any Tennessee community college one year after their first semester of enrollment (or earned a credential in the intervening period)
4. Two-year completion rate: a binary indicator equal to 1 if a student completed a degree or credential within two years of their first semester of enrollment
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
At the end of the registration period for each fall and spring semester between spring 2023 and spring 2025, Jackson State and Northeast State will send course enrollment data for all students registered for the semester. After restricting the sample to the students who meet our eligibility criteria, we will randomly assign students to the control group or to one of three treatment arms (coaching only, course redesign only, or coaching and course redesign). We will assign half of the eligible adult students enrolled in a redesigned course section to coaching. For students not enrolled in a redesigned course section, we will assign as many students to coaching as possible while still maintaining a maximum caseload of 150 students per coach. The remaining students who meet our eligibility criteria will be assigned to the control group. Spring 2023 will serve as a pilot of the intervention, and data from this semester will not be included in the experimental evaluation. Based on enrollment data from the most recent academic year, we anticipate having approximately 2,784 students in our eligible sample in academic years 2023-24 and 2024-25: 1,066 at Jackson State and 1,718 at Northeast State. Randomization will be stratified by college (i.e., site) and student type (i.e., first-time or returning student). From fall 2023 through spring 2025, randomization will also be stratified by whether a student is enrolled in a redesigned course section. We will account for this stratification in our analysis by including (cohort × site × student type × enrollment in redesigned courses) fixed effects in our models.
Experimental Design Details
Not available
Randomization Method
We will randomize students into one of three treatment arms or into the control group using random number generation in Stata. We will assign half of the students enrolled in a redesigned course section to coaching. For students not enrolled in a redesigned course section, we will assign as many students to coaching as possible while still maintaining a maximum caseload of 150 students per coach. The remaining students who meet our eligibility criteria will be assigned to the control group.
Randomization Unit
For the spring 2023 pilot, randomization will occur at the individual level, after stratifying by college (i.e., site) and student type (i.e., first-time or returning student). For each semester from fall 2023 through spring 2025, randomization will also be stratified by whether a student is enrolled in a redesigned course section.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
Based on enrollment data from fall 2021 and spring 2022 (the most recent academic year), we anticipate having approximately 2,784 total observations in academic years 2023-24 and 2024-25: 1,066 at Jackson State and 1,718 at Northeast State. We expect the eligible sample to be larger in the fall semesters (n=1,792) than in the spring semesters (n=992).
Sample size (or number of clusters) by treatment arms
The experiment will have three treatment arms: (1) coaching only; (2) course redesign only; and (3) coaching and course redesign. We conservatively assume that each of the 20 faculty members who redesign their general education courses will teach two redesigned course sections during the academic year (for a total of 40 redesigned course sections across two years). We also assume that there are, on average, 10 adult students in each of the redesigned course sections. Half of the students in the redesigned course sections will be assigned to coaching, which gives us 200 students in the course redesign only treatment arm and another 200 students in the coaching and course redesign treatment arm. To maintain a maximum caseload of 150 students for each of our five coaches, we can offer coaching to a total of 1,500 students across both years. Given our estimate that 200 students in the redesigned course sections will be assigned to coaching, we will assign an additional 1,300 students to coaching who are not enrolled in any redesigned course sections to maximize the number of students who have access to a coach. With 1,700 of our 2,784 total students assigned to one of the three treatment arms, there will be approximately 1,084 students in the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Tennessee Board of Regents Institutional Review Board (TBR IRB)
IRB Approval Date
2022-10-31
IRB Approval Number
tbr_202210_01