HISD State Takeover Survey Experiment

Last registered on September 18, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
HISD State Takeover Survey Experiment
RCT ID
AEARCTR-0014177
Initial registration date
September 16, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 17, 2024, 1:53 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
September 18, 2024, 5:21 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
United States of America
Region
Texas

Primary Investigator

Name
Blake Heller
Affiliation
University of Houston
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Sara Sands
PI Affiliation
University of Houston
Contact Investigator

Additional Trial Information

Status
In development
Start date
2024-09-16
End date
2025-01-17
Keywords
Education, Governance
Additional Keywords
JEL code(s)
I20, I21, I28
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project assesses the extent to which information about student performance on standardized tests influences perceptions of the recent state takeover of Houston Independent School District (HISD) by Texas Education Agency. Specifically, we test whether providing information to survey respondents about test performance in the HISD schools most directly impacted by the takeover—and growth in performance in these schools—influences ratings of the reform (overall and its individual components), ratings of the newly installed district leadership, and stated preferences of which future actions respondents would prefer policymakers pursue. The survey experiment also collects data on respondents' perceptions of the political process that led to the takeover and how it was implemented. We test whether the role of information in shaping respondent preferences is moderated by proximity to HISD as well as the mediating role of process perceptions.
External Link(s)

Registration Citation

Citation
Heller, Blake and Sara Sands. 2024. "HISD State Takeover Survey Experiment." AEA RCT Registry. September 18. https://doi.org/10.1257/rct.14177-1.2
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
After consenting to participate, this survey begins with a series of screener questions and some baseline demographic questions.

At the beginning of the survey experiment, all respondents are provided with a short description of the 2023 takeover of HISD by Texas Education Agency. Half the respondents receive only this information (control group), and half the respondents are randomly assigned to receive additional information (treatment group) about student performance on standardized tests in different sets of schools.

After viewing this information, both groups answer a series of questions related to their perception of the state takeover (overall and its individual components), HISD district leadership, the process that led to the takeover, and a 2024 bond measure that would provide $4.4 billion to finance improvements within HISD schools.

The survey experiment concludes with both groups answering a set of demographic questions.
Intervention Start Date
2024-09-16
Intervention End Date
2025-01-17

Primary Outcomes

Primary Outcomes (end points)
The primary outcome of interest is an index summarizing respondents rating of the state takeover and its individual components.
Primary Outcomes (explanation)
To construct the primary outcome index, 19 subquestions related to respondents' appraisal of the state takeover and its individual components will be converted from a 7-point Likert scale of "Strongly Disagree" to "Strongly Agree" to numeric values of 1-7, summed, and standardized to have a mean of 0 and a standard deviation of one. The subquestions that contribute to the primary outcome index are listed by their question numbers (see survey materials in "Supporting Documents and Materials"): 4 subquestions of Q6.1, 4 subquestions of Q6.2, 3 subquestions of Q6.3, 5 subquestions of Q6.4, 3 subquestions of Q6.5.

Secondary Outcomes

Secondary Outcomes (end points)
The secondary outcomes of this study will include a series of indices that assess different dimensions of respondents' perceptions of the state takeover of HISD, including 1) An index summarizing respondents' overall rating of the state takeover of HISD; 2) respondents' overall rating of NES reforms related to teacher and principal compensation and evaluation; 3) respondents' overall rating of NES reforms related to classroom instruction; 4) respondents' overall rating of NES reforms related to school culture and staffing; 5) respondents' overall rating of NES reforms related to school operations.

Additionally, we will convert respondents' stated preferences about when TEA should restore governing responsibilities to the elected HISD school board into a binary variable based on the median response of the control group, and report whether the informational intervention influenced respondents to prefer a longer or shorter takeover.

Similarly, we will convert respondents' stated preferences about the conditions under which TEA should restore governing responsibilities to the elected HISD school board into a binary variable based on the median response of the control group, and report whether the informational intervention influenced respondents to prefer more or less restrictive conditions for ending the takeover.

Additionally, we will convert respondents' stated preferences about the $4.4 billion 2024 HISD bond measure into a binary variable based on the median response of the control group, and report whether the informational intervention influenced respondents to express more or less support for the bond measure.

Finally, we will conduct a moderation and mediation analyses. For our primary moderation analysis, we will compare the influence of information on respondent preferences based on distance from Houston. First, we will compare the impact of information on the preferences of survey respondents who take the survey in Texas versus those who live outside of Texas. If we have at least 150 survey participants who live in Harris County, Texas, we will compare the impact on Harris County residents versus respondents who take the survey in Texas versus those who live outside of Texas. This is a slight change from our original pre-registered moderation analysis plan (see trial history), when we anticipated being able to target the survey to Harris County residents only (we were able to restrict the sample by state, but not county).

We will examine participants ratings of the process that led to the takeover as a mediator of their ratings of the takeover.

Exploratory analyses will examine the impact of information in responses to particular questions or subquestions.
Secondary Outcomes (explanation)
To construct each index, subquestions related to each domain will be converted from a 7-point Likert scale of "Strongly Disagree" to "Strongly Agree" to numeric values of 1-7, summed, and standardized to have a mean of 0 and a standard deviation of one. Below the subquestions that contribute to each index are listed by their question numbers (see survey materials in "Supporting Documents and Materials"):

Index 1: 4 subquestions of Q6.1
Index 2: 4 subquestions of Q6.2
Index 3: 3 subquestions of Q6.3
Index 4: 5 subquestions of Q6.4
Index 5: 3 subquestions of Q6.5
In

Length of takeover secondary outcome: Binary variable based on median response to Q7.1 in control group.

Restrictiveness of conditions to end takeover secondary outcome: Binary variable based on median response to Q7.2 in control group.

Bond support secondary outcome: Binary variable based on median response to Q47 in control group.

Experimental Design

Experimental Design
This is a simple two-armed randomized survey experiment. 50% of participants will be randomized to view one block of text describing the 2023 TEA takeover of HISD (control group), and 50% will be randomized to view this text as well as additional text and figures that include information about student performance on standardized tests within different groups of Texas schools (treatment group). Both groups will then answer the same set of questions about their perceptions of the 2023 TEA takeover of HISD. Approximately 500 Texas residents will be recruited from an online survey platform to participate and approximately 1000 additional participants will be recruited from a national pool of volunteers.
Experimental Design Details
Not available
Randomization Method
The randomization will be embedded into the survey using the “Randomizer” function in Qualtrics.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1500
Sample size: planned number of observations
1500
Sample size (or number of clusters) by treatment arms
1500
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
MDES = 0.1158 SD at 80% power and alpha=0.05 with baseline covariates that explain 20% of the variance in the index; 0.1448 SD with no baseline covariates.
Supporting Documents and Materials

Documents

Document Name
Survey Instrument (Texas Sample)
Document Type
survey_instrument
Document Description
This is the survey instrument for the Texas sample (pilot data collection of 50 subjects began on 9/16/24). No data was accessed until after this revision of the pre-registration.
File
Survey Instrument (Texas Sample)

MD5: 2574ecf6371be83f6fb98c5647415251

SHA1: 36be01ae7081da23b9df27c41e5e3f3ff83f4543

Uploaded At: September 18, 2024

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Houston IRB
IRB Approval Date
2024-08-08
IRB Approval Number
STUDY00004931
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information