Operation Graduation: A controlled experiment targeting persistence and graduation for community college students

Last registered on March 01, 2019

Pre-Trial

Trial Information

General Information

Title
Operation Graduation: A controlled experiment targeting persistence and graduation for community college students
RCT ID
AEARCTR-0003817
Initial registration date
January 24, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 25, 2019, 3:58 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
March 01, 2019, 4:14 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
University of California, Davis

Other Primary Investigator(s)

PI Affiliation
University of Maryland
PI Affiliation
Dartmouth College

Additional Trial Information

Status
On going
Start date
2018-01-08
End date
2020-06-08
Secondary IDs
Abstract
Community colleges offer great promise to help students build human capital. Yet 60 percent of current students report being ill informed about career opportunities, 33 percent want more academic advising, and fewer than half of community college students complete their associates or certificate program. This study will implement and evaluate an intervention aimed at increasing persistence and graduation at community colleges. The research uses a randomized controlled trial (RCT) research design at several community college campuses within the Community College System of New Hampshire (CCSNH). This project helps distinguish among several distinct hypotheses as to why college coaching/ counseling can be effective for students. The three treatment arms include 1) business as usual, 2) major, career and course planning coupled with auto generated reminders, and 3) the same planning sessions coupled with intensive bi-weekly counseling from a professional based at the college. Survey and in process data will be combined with results across treatment arms to identify the barriers to college completion and which elements of these interventions are the most effective.
External Link(s)

Registration Citation

Citation
Carrell, Scott, Melissa Kearney and Bruce Sacerdote. 2019. "Operation Graduation: A controlled experiment targeting persistence and graduation for community college students." AEA RCT Registry. March 01. https://doi.org/10.1257/rct.3817-2.0
Former Citation
Carrell, Scott, Melissa Kearney and Bruce Sacerdote. 2019. "Operation Graduation: A controlled experiment targeting persistence and graduation for community college students." AEA RCT Registry. March 01. https://www.socialscienceregistry.org/trials/3817/history/42303
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This study will implement and evaluate a community college based intervention program aimed at increasing persistence and graduation. We use a randomized controlled trial (RCT) design within the Community College System of New Hampshire (CCSNH). The project will also investigate several distinct hypotheses as to why college coaching/ counseling can be effective for students: a) Students lack basic information about what steps they need to take to succeed or basic information about where to obtain help; b) Students lack a concrete plan for their academic major and early career, but once a plan is put in place, students can be effective at executing this plan thereafter; c) Students need interactions with an older adult as a commitment device or reminder to continue to make progress in their plan; d) Students face a multitude of obstacles that require help from an experienced counselor to surmount.

The study consists of two treatment arms and is designed to help distinguish among several distinct hypotheses as to why college coaching/ counseling can be effective for students. In Treatment Arm 1, called the “Auto Advising Group” students receive 4-5 texts per semester. In Treatment Arm 2, students receive bi-weekly meetings (individual and group) with a “College Access Navigator” (CAN).
Intervention Start Date
2018-01-08
Intervention End Date
2020-06-08

Primary Outcomes

Primary Outcomes (end points)
Key outcomes include: number of credit hours attempted; number of credit hours completed; whether the student re-enrolls the next semester or year; and whether the student obtains a degree or certificate.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Intermediate outcomes such as difficulties selecting courses and difficulties registering for the semester will be collected through pre and post-student surveys. Our surveys measure both the specificity of student’s college and career plans and their levels of determination to complete these plans. We also measure students’ knowledge about campus resources and where to find help on specific topics. Finally we measure sources of family and friend emotional and practical support.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will examine the effectiveness of the two treatment arms relative to a control group by implementing a randomized control trial. Eligible students within each campus will be randomized to one of three groups as follows:

1) The Control group will receive “business as usual” meaning the existing advising system.

2) The Automated Advising group will have a single meeting with a WMCC Dean to discuss a plan for the coming two semesters. This will include outlining course choices. The Auto Advising group will receive 4-5 texts per semester reminding them to a) refile the FAFSA, b) register for courses, c) seek help at midterm time if struggling in courses, d) an email reminder with the list of courses selected. Students will have the option to opt out of receiving reminder texts by replying STOP. The WMCC official performing this work will be someone other than the College Access Navigator. WMCC will maintain a spreadsheet/ database record with each of the contacts (or lack thereof).

3) The College Access Navigator (CAN) group will have biweekly meetings with the CAN. Each meeting (or its cancellation) will be recorded in the spreadsheet along with the time and approximate length of the meeting. The purpose of the meetings will include course selection, staying on track in the desired major, choosing majors that match the student’s skills and interest, making career plans, completing financial aid forms, identifying any academic areas where the student is struggling, identifying (and helping solve) any other obstacles to college completion. For each meeting the CAN should note which of the topics below are covered. The CAN should also note key items including: sources of family and friend support used by the student, major and career interests, any academic struggles, any obstacles to college completion, financial aid questions or concerns, best ways for the CAN to reach the student.
Experimental Design Details
Randomization Method
Randomization in office by a computers
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1670 students
Sample size: planned number of observations
1670 students
Sample size (or number of clusters) by treatment arms
700 students in the control, 700 student in the texting treatment, 270 students in the CAN treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With sample sizes of 270 (CAN treatment), 700 auto advising and 700 control, we are likely to detect effect sizes on graduation of 10 percentage points or larger. If the effect size is 12.5 percentage points or more, the power is 95 percent or greater. These statements apply to both our ability to distinguish between [CAN vs Auto Advising] or [CAN vs control]. The power to detect differences in outcomes between Auto Advising and control is of course much greater since both those groups have sample sizes of 700.
IRB

Institutional Review Boards (IRBs)

IRB Name
CCSNH Institutional Review Board
IRB Approval Date
2018-06-28
IRB Approval Number
SYEX-2018-01
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials