The Effects of Student Growth Data on School District Choice: Evidence From a Survey Experiment

Last registered on April 30, 2019
View Trial History

Pre-Trial

Trial Information

General Information

Title
The Effects of Student Growth Data on School District Choice: Evidence From a Survey Experiment
RCT ID
AEARCTR-0003401
Initial registration date
October 09, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 09, 2018, 1:56 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
April 30, 2019, 3:13 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
United States of America
Region

Primary Investigator

Name
David Houston
Affiliation
George Mason University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Jeffrey Henig
PI Affiliation
Teachers College, Columbia University
Contact Investigator

Additional Trial Information

Status
On going
Start date
2018-10-17
End date
2019-06-15
Keywords
Education
Additional Keywords
JEL code(s)
Secondary IDs
Abstract
We seek to identify the effects of providing district-level average student achievement data and/or average student growth data on subjects’ hypothetical school district enrollment decisions. Compared to average achievement at a single point in time, average growth is arguably better able to capture schools’ and school systems’ contributions to student learning. Average growth also bears a weaker relationship to the racial and socio-economic composition of the student body, making it easier to identify highly effective schools that serve less advantaged students.

This study consists of an online survey experiment in which each subject is asked to imagine that s/he is a parent who is moving to a new city. When deciding where to live, one of their top priorities is to choose a school district for their elementary school-age child. The survey will provide basic demographic information about the five largest school districts in the metropolitan area. In addition to the demographic information, subjects will be randomly assigned to receive either 1) average achievement data, 2) average growth data, 3) both, or 4) neither. Based on these data, subjects will choose their preferred school district.

Subjects will be asked to complete this process for the metropolitan areas of the five largest cities in the United States: New York City, Los Angeles, Chicago, Houston, and Phoenix. At the end of the survey, subjects will answer a small battery of basic demographic questions.
External Link(s)

Registration Citation

Citation
Henig, Jeffrey and David Houston. 2019. "The Effects of Student Growth Data on School District Choice: Evidence From a Survey Experiment." AEA RCT Registry. April 30. https://doi.org/10.1257/rct.3401-4.0
Former Citation
Henig, Jeffrey and David Houston. 2019. "The Effects of Student Growth Data on School District Choice: Evidence From a Survey Experiment." AEA RCT Registry. April 30. https://www.socialscienceregistry.org/trials/3401/history/45775
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This study consists of an online survey experiment in which each subject is asked to imagine that s/he is a parent who is moving to a new city. When deciding where to live, one of their top priorities is to choose a school district for their elementary school-age child. The survey will provide basic demographic information about the five largest school districts in the metropolitan area. In addition to the demographic information, subjects will be randomly assigned to receive either 1) average achievement data, 2) average growth data, 3) both, or 4) neither. Based on these data, subjects will choose their preferred school district.

Subjects will be asked to complete this process for the metropolitan areas of the five largest cities in the United States: New York City, Los Angeles, Chicago, Houston, and Phoenix. At the end of the survey, subjects will answer a small battery of basic demographic questions.

Please see pre-analysis plan for more detail (also available at https://osf.io/3e8wv/)
Intervention Start Date
2018-10-24
Intervention End Date
2019-02-14

Primary Outcomes

Primary Outcomes (end points)
For each metropolitan area, there will be three outcome measures: subjects’ choices with respect to 1) median household income, 2) the percentage of students eligible for free and reduced priced lunch, and 3) the percentage of white students. The estimands for each metropolitan area will be the average differences in the racial and socio-economic compositions of the school district choices between experimental groups.

To estimate average treatment effects across all five metropolitan areas, we will reorganize the data into a long-form dataset. Each subject will appear in the dataset five times: once for each school district selection. The estimands for these data will also be the average differences in the racial and socio-economic compositions of the school district choices between experimental groups. When conducting these analyses, we will cluster standard errors at the subject level.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We are planning on conducting an online survey experiment with four experimental conditions. For the five largest school districts in the metropolitan areas of the five largest US cities, subjects will be randomly assigned to receive either 1) average achievement data, 2) average growth data, 3) both, or 4) neither. The estimands will be the average differences in the racial and socio-economic compositions of the school district choices between experimental groups.

Please see pre-analysis plan for more details (also available at https://osf.io/3e8wv/)
Experimental Design Details
Randomization Method
We will use the Qualtrics survey platform to administer the survey experiment. Using Qualtrics’ “Randomizer” feature, roughly equal portions of the subject pool will be randomly assigned to the control group (receiving neither achievement nor growth data), the achievement group (receiving only achievement data), the growth group (receiving only growth data), or the combination group (receiving both achievement and growth data).
Randomization Unit
Randomization occurs at the individual level. For a few summary analyses, we will reorganize the data into a "long-form" dataset (each individual will appear in the data five times: once for each metropolitan area school district choice). When analyzing these data, we will cluster standard errors at the subject level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
2,500 individuals
Sample size: planned number of observations
2,500 individuals
Sample size (or number of clusters) by treatment arms
625 control, 625 achievement group, 625 growth group, 625 combination group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With four experimental groups of 625 individuals each, we can detect a standardized effect size of 0.16 (p < 0.05) at 80% power.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Harvard University Institutional Review Board
IRB Approval Date
2018-09-25
IRB Approval Number
18-1500
IRB Name
Teachers College, Columbia University Institutional Review Board
IRB Approval Date
2018-08-08
IRB Approval Number
18-456
Analysis Plan

Analysis Plan Documents

Pre-Analysis Plan (updated)

MD5: a8f472065eab69a85bf04d72018d5a55

SHA1: 3802be5a54841111cdc75f1a6f441f836f55e996

Uploaded At: January 29, 2019

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials