Increasing Economic Diversity at a Flagship University: Results from a Large-Scale, Randomized Trial

Last registered on April 18, 2023

Pre-Trial

Trial Information

General Information

Title
Increasing Economic Diversity at a Flagship University: Results from a Large-Scale, Randomized Trial
RCT ID
AEARCTR-0001831
Initial registration date
April 09, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 10, 2017, 12:54 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
April 18, 2023, 9:26 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
University of Michigan

Other Primary Investigator(s)

PI Affiliation
Columbia University
PI Affiliation
University of Michigan
PI Affiliation
Colby College
PI Affiliation
University of Michigan

Additional Trial Information

Status
On going
Start date
2015-05-01
End date
2026-05-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Low-income students are substantially less likely than higher-income students to attend a selective university. In part, this is due to inadequate academic preparation. But even among students who are extremely well-prepared for college, there are substantial income differences in the probability of attending a selective institution (Hoxby and Avery 2012). In Michigan, the pattern is similar. While one in five higher-income students attended a university at least as competitive as the University of Michigan, only one in ten similarly-achieving low-income students did so. Previous research has demonstrated that the college choice decisions of high-achieving, low-income students can be affected by an informational intervention (Hoxby and Turner 2013). We test the effect of delivering information about the state flagship, including an early commitment of financial aid, to high-achieving students in Michigan. In the first set of interventions, which ran for four years, we delivered a simplified message of financial aid via a student packet, principal letter and email, and parent letter to student's home addresses and school addresses. In the fifth year, we construct a second treatment arm that varies the language and the promise of financial aid. In the sixth year, we scaled the HAIL treatment to all schools. We randomized the HAIL test-score cutoff by school, evaluating the effects of lowering the test score cutoff. We set two score cutoffs: one low and one high cutoff. A school in the control group had any student above the high cutoff receive a HAIL packet, with students between the low and high cutoff not receiving the intervention. A school in the treatment group had any student with score above the lower cutoff receive a HAIL packet.
In the seventh and eighth years, the treatment arms once again differed from past years. The initial HAIL intervention involved mailings to students, parents, and principals. In these cohorts, all schools were eligible for the HAIL Scholarship. Students in the control group receive the student and parent letters, but not the principal letter, and students in the treatment group receive the full HAIL treatment.
External Link(s)

Registration Citation

Citation
Burland, Elizabeth et al. 2023. "Increasing Economic Diversity at a Flagship University: Results from a Large-Scale, Randomized Trial." AEA RCT Registry. April 18. https://doi.org/10.1257/rct.1831-4.0
Former Citation
Burland, Elizabeth et al. 2023. "Increasing Economic Diversity at a Flagship University: Results from a Large-Scale, Randomized Trial." AEA RCT Registry. April 18. https://www.socialscienceregistry.org/trials/1831/history/177037
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
In the first four years of the intervention, we had one treatment arm and one control arm. The intervention for the first four years consists of sending materials to the students, their parents, and their school principals. The materials promise four years of free tuition and fees, conditional on acceptance to the Flagship University, and also provide information about the Flagship and the application process. Mailings are delivered in the late summer and early fall of senior year.

In the fifth year, we introduced a second treatment arm, which consisted of a personalized message informing students about the "Go Blue Guarantee", a financial aid policy the University of Michigan introduced in 2017 that provides one year of free tuition and fees (renewable for up to four years if students maintain family income eligibility) for students whose family income is below $65,000, the median for the state of Michigan. In contrast to the HAIL promise, Go Blue Guarantee does not guarantee financial aid for all four years up front, though students can continue to receive the Go Blue Guarantee for up to four years, assuming their family income circumstances do not change. Additionally, while HAIL is awarded unconditional of filling out financial aid forms, the Go Blue Guarantee is only awarded after the FAFSA and CSS profile are filled out.

For all cohorts, we randomized at the school level. That is, every student in a school who meets the sample criteria is assigned the same treatment status. We also stratified randomization by the number of students eligible in each school, so that randomization took place within each of four school size groups (schools with one, two, three, or four or more students meeting our sample inclusion criteria). Assignment to treatment was done once per stratum (pure randomization within strata).

For the second through fourth years of randomization, all schools that had been treated in year one continued to be treated, while all control schools from year one were maintained as control schools in subsequent years. Because many schools only had one or two students eligible to receive HAIL in each year, each year of the intervention there were schools that dropped out of the sample because they had no eligible students, and there were schools that entered the sample because they didn't have a HAIL eligible student in a previous year, but had one in the current year. For all new schools entering the sample in years 2 through 4, we randomized according to the same cluster-stratified method as the first cohort.

In year five, because we introduced a second treatment arm, all schools with eligible students in that year were newly randomized. We stratified on number of HAIL students in the school (1,2-3, 4+) as well as by region of the state (Southeast region, non-Southeast region).

In the sixth cohort, we scaled the intervention to all schools in the state of Michigan. That is, there were no true “control” and “treatment” schools. In this cohort, we varied the test score cutoff, evaluating the effects of lowering the test score cutoff for scholarship eligibility. We set a low and a high test score cutoff. All students, across treatment & control schools who were above the high cutoff received the HAIL Scholarship. In treatment schools, students between the low and high test score cutoff received the HAIL Scholarship. In control schools, students between the low and high test score cutoff did not receive the HAIL Scholarship. Schools were randomized within four strata based on region (Southeast or not) and whether the school was assigned HAIL treatment in at year 4 or year 5 (recently HAIL vs. not). Schools were assigned to treatment within strata, assigning schools to each treatment arm with a probability of ½.
In the seventh and eighth cohorts, we randomized whether principals were contacted. In the control group, only the student and their parent received a letter. In the treatment group, principals, students, and parents were contacted. We assigned students to one of two strata based on region (Southeast or not). We then assigned treatment within strata, re-randomizing each year, assigning schools to each treatment arm with a probability of ½.

Intervention Start Date
2015-09-01
Intervention End Date
2024-09-30

Primary Outcomes

Primary Outcomes (end points)
All outcomes are measured using administrative data from the Flagship and the state Department of Education. The main outcome of interest is attending a college at least as selective as the Flagship University. This outcome, as well as time spent in college, degrees earned, and major of any degrees, will be measured using data from the National Student Clearinghouse. For students who enroll in public colleges in Michigan, we have transcripts that include credits earned, grades and major. Intermediary outcomes include applying to the Flagship University and timing of application (we do not have application information for other colleges).
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Cluster-stratified randomized controlled trial. The intervention will be clustered at the school level, and stratified by the number of students eligible for the intervention in each school.
Experimental Design Details
Not available
Randomization Method
Randomization done by a psuedorandom number generator in Stata.
Randomization Unit
Clusters at the school level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Approximately 500 schools per year
Sample size: planned number of observations
Approximately 2,000 students per year, 10,000 students total over the 5 years of the interventions
Sample size (or number of clusters) by treatment arms
Randomization in the first four years was 50/50 by school. Sample sizes as follows: Year 1: 1,057 treated students in 262 schools and 1,051 control students in 267 schools. Year 2: 1,806 students from 498 schools, with 879 treated students in 239 schools and 927 control students in 259 schools. Year 3: 3,516 students from 666 schools, with 1,833 students treated in 349 schools and 1,683 control students from 317 schools. Year 4: 1,181 students from 324 schools, with 737 treated students from 199 schools, and 444 control students from 125 schools.

In Year 5, a second treatment arm was introduced and schools were randomly assigned to three equally-sized groups. There were roughly 160 schools in each category (treatment 1, treatment 2, control), and roughly 600 students in each category.

In year 6, we were back to 2 treatment arms. There were 545 schools randomized, with approximately 270 schools (with 1850 students) in each treatment group.
In year 7, we were back to 2 treatment arms. There were 452 total schools, randomized into two treatment arms with 226 in each treatment arm (and roughly 730 students in each).
In year 8, there were 397 eligible schools, randomized into two treatment arms with roughly 200 schools in each (with 560 students in each).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Using data from 2013, the latest cohort for whom we have available outcome data, we identified students who met the eligibility requirements for the HAIL scholarship. There were 2,692 such students in 654 schools (4.12 students per school). Matching their data with college enrollment data, we see that schools had an average of 8.7 percent of eligible students enroll in a highly or most selective institution. This provides a baseline estimate of the outcome of interest prior to the implementation of the treatment. In the first year of the intervention, we have 529 schools with roughly half in treatment and half in control. We expect to have a similar sample in year two. We estimate the minimal detectable effect (MDE) size for the study of 0.048. This estimate of the MDE does not account for our strata. This estimate is consistent with other estimates in the literature. Hoxby and Turner (2013) estimate an effect size of 0.12 on a student enrolling in a “peer” college, which the authors define as a college where the median student scores within five percentiles of the student’s own score.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Michigan Department of Education Research Collaborative Institutional Review Board
IRB Approval Date
2015-04-02
IRB Approval Number
UofM_0115SD
Analysis Plan

Analysis Plan Documents

HAIL Preanalysis Plan

MD5: fa33b19f0fee53434cc2ee4016e7df0c

SHA1: 0211ff674b0cec5ca144106fb01c00e2595127dd

Uploaded At: April 09, 2017