Experimental Design
We will evaluate the impact of a comprehensive case management program for at-risk high school students (hereafter, “the program”) using an RCT research design. The study includes two separate cohorts of students—the 2017-2018 entering cohort and the 2018-2019 entering cohort, resulting in an expected total study sample of 2,500. Students were enrolled from two mid-size, urban school districts (hereafter, “District A and District B”) in the first cohort, but only from District B in the second cohort. At the start of the school year for each of these cohorts, the school districts sent a de-identified list of eligible incoming ninth grade students to the research team (n=1,005 in District A and n=739 in District B for the 2017-2018 cohort). The research team then generated random numbers for each of these eligible students to specify the order in which they were recruited to enroll in the program (i.e., the treatment). Total capacity for participation in the program for incoming ninth-grade students was approximately 250 in District A and 200 in District B with available program slots varying across schools.
To accommodate differences in capacity, the randomization was stratified by school, generating 14 randomly sorted lists of students for the 14 schools participating in the study in the first cohort. The research team sent the randomly ordered lists to the program’s senior administrators, who then linked this de-identified list of students to personal identifying information. They then provided program staff at each school the names of students that they should recruit for program participation. The names were provided to program staff in the order randomly generated by the research team. With this process, the treatment group will be defined as those from the list who were offered the opportunity to enroll in the program (i.e., those at the top of the list) and the control group will be students who were never offered the opportunity to enroll (i.e., those at the bottom of the list). We will use two definitions of “treatment offer” to address endogeneity of program take-up in a randomized waitlist setting (Behaghel et al., 2017). The first approach, “ever offer,” as described above will assign the treatment offer to any student whose name was released to the program staff for recruitment. Those names not released constitute the control group. We will also use a predetermined “initial offer” in which we assign 60% of each school’s randomly ordered list, in numeric order, to the treatment group and the remaining 40% to the control group. This “initial offer” designation is free of the bias induced by any individual student’s probability of offer being conditional on the take-up rate among the preceding students on the list.