Experimental Design
The data comes from two sources: 1) information from files at HCZ and 2) administrative data on student demographics and outcomes from the New York City Department of Education (NYCDOE). The experiment is to analyze the causal impact of attending the Promise Academy charter schools in HCZ on academic outcomes.
To ensure proper randomization, researchers excluded students with siblings who were lottery winners (since then siblings are automatically enrolled). The HCZ data were then matched to the New York City administrative data using last name, first name, date of birth, prior grade, etc. 69.6% of middle school and 59.9% of elementary school lottery winners enrolled in the Promise Academy for at least one year.
The results indicate that the Promise Academy charter schools are effective in increasing achievement as students in the middle school increased their math and ELA scores by 0.229 and 0.047 standard deviations respectively, using the 2SLS estimates, which is enough to close the black-white achievement gap in mathematics by ninth grade. In the elementary school, students increased their scores in math by 0.191 standard deviations and in ELA by 0.114 standard deviations, which is enough to close the achievement gap by third grade.
According to the second empirical test, the distance x cohort instrumental variable strategy for middle school indicates that the effect of attending the Promise Academy is 0.206 standard deviations in math and -0.054 in ELA; therefore, the estimate suggests that math scores will increase 0.618 standard deviations - having a significant impact on math scores but not on ELA scores. For elementary schools, the distance x cohort estimates that students at the Promise Academy gained 0.324 standard deviations in math and 0.420 in ELA for each year. Therefore, the increase in scores was noticeable even if students were not officially in the HCZ zone. And finally, siblings of enrolled lottery winners gained approximately 0.051 standard deviations in math and 0.087 standard deviations in ELA, which is smaller than those who were enrolled in the Promise Academy. Therefore, the experiment suggests that high-quality schools can be more effective than community programs alone.