The Impact of Student Success Predictions and Dedicated Course Supports on Community College Student Academic Success.

Last registered on April 30, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
The Impact of Student Success Predictions and Dedicated Course Supports on Community College Student Academic Success.
RCT ID
AEARCTR-0015886
Initial registration date
April 28, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 30, 2025, 1:12 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region
Virginia

Primary Investigator

Name
Ben Castleman
Affiliation
University of Virginia
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Kelli Bird
PI Affiliation
University of Virginia
Contact Investigator

Additional Trial Information

Status
On going
Start date
2023-08-01
End date
2025-12-31
Keywords
Education
Additional Keywords
community college, tutoring, advising, prediction, gateway courses
JEL code(s)
I23, I24
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We conduct a multi-semester randomized controlled trial (RCT) with a community college in Virginia of an intervention designed to improve student academic success in foundational Math, English, and Biology courses with historically-high course failure rates. The intervention consists of three distinct components: (1) Early-term predictions for instructors of which students may struggle to succeed academically; (2) embedded academic tutors; and (3) dedicated advisors to support students with broader challenges. Randomization is conducted at the instructor section-level. The primary outcome of interests are whether students pass the course, either with a "C" or higher or a "B" or higher, and whether students withdraw from the course prior to the end of the semester. We will investigate mechanisms underlying program impacts using college administrative data on individual-level take-up of college support services and learning managment system data on student engagement with course resources.
External Link(s)

Registration Citation

Citation
Bird, Kelli and Ben Castleman. 2025. "The Impact of Student Success Predictions and Dedicated Course Supports on Community College Student Academic Success.." AEA RCT Registry. April 30. https://doi.org/10.1257/rct.15886-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The intervention consists of three distinct components: (1) Early-term predictions for instructors of which students may struggle to succeed academically; (2) embedded academic tutors; and (3) dedicated advisors to support students with broader challenges. Early-term predictions were generated by Student Success staff at the college, with guidance from the research team, during the 3rd and 4th week of the semester. Student Success staff and treated instructors reviewed these predictions to identify a priority list of students to intervene with over the balance of the semester. Embedded academic tutors and the dedicated advisor, coordinating with the instructor, proactively reached out to priority students to offer academic and more holistic support over the remainder of the semester.
Intervention (Hidden)
Intervention Start Date
2023-08-01
Intervention End Date
2024-12-31

Primary Outcomes

Primary Outcomes (end points)
The primary outcome of interests are whether students pass the course, either with a "C" or higher or a "B" or higher, and whether students withdraw from the course prior to the end of the semester.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
We will investigate the impact of the intervention on end-of-semester course grades in the participating courses. We will investigate mechanisms underlying program impacts using college administrative data on individual-level take-up of college support services and learning management system data on student engagement with course resources. These include student use of tutoring and other support services (e.g. advising) at the institution; student log-ons, assignment submissions, activity participation, and instructor engagement to and via the Learning Management System.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Several months prior to each semester, we worked with PVCC to identify gateway courses (i.e. courses typically taken by students near the start of their time at PVCC, and satisfies pre-requisites for higher-level courses) to include in the pilot study, and then recruit instructors to participate in the pilot who would be teaching these gateway courses in the upcoming term. At the start of each semester, we randomized gateway course sections to either receive the intervention or to a control condition in which instructors pursued standard instructional practice and standard outreach to students; standard practice could have included instructors supporting students individually as well as referring them to PVCC advising and tutoring resources. Randomization occurred within gateway course, at the section level, e.g., if there were four English Composition I instructors signed up to participate in the project, we first randomly selected two instructors to receive the intervention and two instructors to be in the control condition. Then, for each instructor assigned to the intervention, we randomly chose one of their gateway course sections to receive the intervention; any other sections those instructors taught were not included in the pilot.
Experimental Design Details
Randomization Method
Computer random number generator
Randomization Unit
Course section
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
52 course sections
Sample size: planned number of observations
1133 student x course section observations
Sample size (or number of clusters) by treatment arms
27 course sections in treated condition; 25 course sections in control condition
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Virginia
IRB Approval Date
2023-04-19
IRB Approval Number
5620
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials