Staying on Track: Comprehensive Case Management to Prevent High School Dropout

This evaluation will assess the effectiveness of a program designed to provide support for students in high school, increasing the likelihood of high school graduation and post-secondary success, in schooling and/or the labor market. This effort is conducted in partnership with two urban school districts in the eastern United States. The program provides students with wraparound services including a professional, trained mentor in addition to academic support (e.g., tutoring, state exam prep). Students also receive job readiness training, employment support, and opportunities to build life skills.

2017-09-25

2022-08-31

Primary outcome: “on-track to graduation” index (e.g., Kemple, 2013; Allensworth & Easton, 2005) that includes dropout, on-time grade promotion, and benchmarks for other common predictors of graduation such as attendance, course credit accumulation, GPA, state exam completion, and disciplinary actions. The index will be a composite measure incorporating a student’s status on each benchmark. The inclusion of some potential components in the index will depend, in part, on whether we are able to obtain data of sufficient quality on the benchmark.

In addition to our primary outcome (the "on-track to graduation" index), we will examine the effect of the program on each of the separate components of the index. As part of our exploratory analyses we will also consider the impact of the program on various subgroups defined by gender, race/ethnicity, age, and predicted outcomes (above and below median, terciles).

We will evaluate the impact of a comprehensive case management program for at-risk high school students (hereafter, “the program”) using an RCT research design. The study includes two separate cohorts of students—the 2017-2018 entering cohort and the 2018-2019 entering cohort, resulting in an expected total study sample of 2,500. Students were enrolled from two mid-size, urban school districts (hereafter, “District A and District B”) in the first cohort, but only from District B in the second cohort. At the start of the school year for each of these cohorts, the school districts sent a de-identified list of eligible incoming ninth grade students to the research team (n=1,005 in District A and n=739 in District B for the 2017-2018 cohort). The research team then generated random numbers for each of these eligible students to specify the order in which they were recruited to enroll in the program (i.e., the treatment). Total capacity for participation in the program for incoming ninth-grade students was approximately 250 in District A and 200 in District B with available program slots varying across schools.

To accommodate differences in capacity, the randomization was stratified by school, generating 14 randomly sorted lists of students for the 14 schools participating in the study in the first cohort. The research team sent the randomly ordered lists to the program’s senior administrators, who then linked this de-identified list of students to personal identifying information. They then provided program staff at each school the names of students that they should recruit for program participation. The names were provided to program staff in the order randomly generated by the research team. With this process, the treatment group will be defined as those from the list who were offered the opportunity to enroll in the program (i.e., those at the top of the list) and the control group will be students who were never offered the opportunity to enroll (i.e., those at the bottom of the list). We will use two definitions of “treatment offer” to address endogeneity of program take-up in a randomized waitlist setting (Behaghel et al., 2017). The first approach, “ever offer,” as described above will assign the treatment offer to any student whose name was released to the program staff for recruitment. Those names not released constitute the control group. We will also use a predetermined “initial offer” in which we assign 60% of each school’s randomly ordered list, in numeric order, to the treatment group and the remaining 40% to the control group. This “initial offer” designation is free of the bias induced by any individual student’s probability of offer being conditional on the take-up rate among the preceding students on the list.

Randomization done in office by a computer

Individual students

No

N/A

Approximately 2500 students

Approximately 1500 students treatment, 1000 students control