Impact Evaluation of High School Mentorship Program

The Thread intervention is an existing mentoring program designed to help disadvantaged students bridge opportunity and achievement gaps. Students join Thread halfway through their ninth grade year and remain enrolled in Thread through the remainder of high school and six years thereafter, for a total of ten years. Most of the Thread mentoring program is between the youth and their Thread “family”: a network of up to five community volunteers. Youth work with their family towards finishing high school and then enrolling in post-secondary education, gaining meaningful employment, or joining the military. Thread family members aid in this process in numerous ways, such as helping youth obtain missing documents, packing lunches, providing rides to activities, and being a listening ear and helping hand. The long period of enrollment in Thread allows students to form deep, meaningful connections that elevate and inspire all involved. Large group events also continue throughout a youth’s Thread experience.
Thread builds a social fabric between mentors and students across socioeconomic groups, and differs from typical mentorship programs in five key ways. First, the program exclusively engages the highest risk students in the bottom 25% of their freshman high school class, who have single-digit high school graduation rates. Almost 90% of this population are racial or ethnic minorities, and all of the partner schools qualify for universal free lunch status. Second, Thread links students to important social services, including summer school, summer employment programs, legal support, public housing, and food services. Third, volunteer mentors, who are often college students or recent college graduates, provide access to a high socioeconomic status (SES) network that is usually out of reach to Baltimore public school students. This type of “economic connectedness” has been shown to be a strong predictor of upward income mobility, yet few other interventions promote close connections across SES lines (Chetty et al., 2022). Fourth, the four-to-one ratio of volunteer mentors to students increases the likelihood of an idiosyncratic match between mentors and students, and allows for sufficient redundancy such that volunteers can provide substantially higher levels of continual support compared to typical mentorship programs. Fifth, mentors work with their students for their entire high school career and six years afterwards, helping adolescents transition to young adulthood.

2023-01-01

2030-12-31

The primary outcome of interest is high school graduation rate.

The main secondary outcome is a composite index of student success in high school, likely containing the following measures: grades, disciplinary actions, attendance, service learning hours, and any other available school data.
Other secondary outcomes of interest are college persistence or completion, income and earnings, housing stability, criminal justice involvement, and mortality. We are structured to allow follow-up along these outcomes, but these outcomes are outside the scope of the first paper.

Study enrollment will occur during the spring semester, starting in 2023 and continuing through 2025. The research team will screen 9th grade students at Thread-participating schools for eligibility: first-time 9th graders with adequate English proficiency, taking the majority of their classes with the general student population (i.e. not in ESL or special education programs), and first semester GPA below some cutoff. Historically, Thread recruited students below the 25th percentile and with a 1st semester GPA below 1.0. In order to generate a control group for the RCT we will expand the pool of students included in the lottery. At each school we will choose a GPA percentile upper cutoff under the conservative assumption of a 60% take-up rate, such that half of those below the cutoff are not offered the program. While capacity as a share of the ninth grade cohort will vary slightly across schools, we expect to set school-specific GPA thresholds between the 25th and 40th percentile.
For each year of study enrollment, we will randomize eligible students within school, gender, and GPA strata. Randomization odds will be approximately 50-50. Thread will then recruit students in the treatment group to join the program, with the goal of registering cohorts in multiples of 16. We plan to enroll 1260 people in the study over three years, of which approximately 630 will be offered a spot in Thread and 378 individuals will enroll in the program. We may increase the number of students enrolled or the number randomized into treatment if program take up is lower than expected.
Program effects will be estimated by an intent-to-treat design where high school completion will be regressed on a vector of observed characteristics, strata (i.e. sex-by-cohort-by-school) fixed effects, and a treatment group dummy variable. We will also report treatment-on-treated effects using a two-stage least squares regression approach. As described above, the key outcomes for this evaluation will be high school graduation rates of the students which will be collected through Baltimore City Public School data. Heterogeneous impacts can be obtained by estimating the baseline regression with additional controls for groups and group-by-treatment interaction effects.
During the application process, participants consent to be followed in administrative records. We will track all outcomes of interest as they become available from administrative datasets including Baltimore City Public Schools data, Maryland Longitudinal Data System, and the National Student Clearinghouse. On-time high school completion will be available for the first enrollment cohort in summer of 2026, and for the full sample in the summer of 2028.

Not available

Computer (Stata).

Unit of randomization: Each year, we randomize individual students within gender, school and GPA strata.

No

1260 students.

1260 students.

Approximately 630 students in the control group, 630 students in the treatment group.

We plan to enroll a study sample of 1260 individuals, with approximately half of these assigned to the treatment group (offered a spot in Thread), and a program take-up rate of 60%. Assuming a baseline high school graduation rate of 10% (which implies a standard deviation of 0.3), we are powered to detect a 5.2 percentage point increase in graduation rates which is about one eighth of the observational differences Thread has documented. This would be an increase of over 50% of the control group mean.