In the first four years of the intervention, we had one treatment arm and one control arm. The intervention for the first four years consists of sending materials to the students, their parents, and their school principals. The materials promise four years of free tuition and fees, conditional on acceptance to the Flagship University, and also provide information about the Flagship and the application process. Mailings are delivered in the late summer and early fall of senior year.
In the fifth year, we introduced a second treatment arm, which consisted of a personalized message informing students about the "Go Blue Guarantee", a financial aid policy the University of Michigan introduced in 2017 that provides one year of free tuition and fees (renewable for up to four years if students maintain family income eligibility) for students whose family income is below $65,000, the median for the state of Michigan. In contrast to the HAIL promise, Go Blue Guarantee does not guarantee financial aid for all four years up front, though students can continue to receive the Go Blue Guarantee for up to four years, assuming their family income circumstances do not change. Additionally, while HAIL is awarded unconditional of filling out financial aid forms, the Go Blue Guarantee is only awarded after the FAFSA and CSS profile are filled out.
For all cohorts, we randomized at the school level. That is, every student in a school who meets the sample criteria is assigned the same treatment status. We also stratified randomization by the number of students eligible in each school, so that randomization took place within each of four school size groups (schools with one, two, three, or four or more students meeting our sample inclusion criteria). Assignment to treatment was done once per stratum (pure randomization within strata).
For the second through fourth years of randomization, all schools that had been treated in year one continued to be treated, while all control schools from year one were maintained as control schools in subsequent years. Because many schools only had one or two students eligible to receive HAIL in each year, each year of the intervention there were schools that dropped out of the sample because they had no eligible students, and there were schools that entered the sample because they didn't have a HAIL eligible student in a previous year, but had one in the current year. For all new schools entering the sample in years 2 through 4, we randomized according to the same cluster-stratified method as the first cohort.
In year five, because we introduced a second treatment arm, all schools with eligible students in that year were newly randomized. We stratified on number of HAIL students in the school (1,2-3, 4+) as well as by region of the state (Southeast region, non-Southeast region).
In the sixth cohort, we scaled the intervention to all schools in the state of Michigan. That is, there were no true “control” and “treatment” schools. In this cohort, we varied the test score cutoff, evaluating the effects of lowering the test score cutoff for scholarship eligibility. We set a low and a high test score cutoff. All students, across treatment & control schools who were above the high cutoff received the HAIL Scholarship. In treatment schools, students between the low and high test score cutoff received the HAIL Scholarship. In control schools, students between the low and high test score cutoff did not receive the HAIL Scholarship. Schools were randomized within four strata based on region (Southeast or not) and whether the school was assigned HAIL treatment in at year 4 or year 5 (recently HAIL vs. not). Schools were assigned to treatment within strata, assigning schools to each treatment arm with a probability of ½.
In the seventh and eighth cohorts, we randomized whether principals were contacted. In the control group, only the student and their parent received a letter. In the treatment group, principals, students, and parents were contacted. We assigned students to one of two strata based on region (Southeast or not). We then assigned treatment within strata, re-randomizing each year, assigning schools to each treatment arm with a probability of ½.