x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
Evaluation of the District of Columbia Public Schools Family Engagement Partnership
Last registered on June 23, 2016

Pre-Trial

Trial Information
General Information
Title
Evaluation of the District of Columbia Public Schools Family Engagement Partnership
RCT ID
AEARCTR-0001370
Initial registration date
June 23, 2016
Last updated
June 23, 2016 10:00 AM EDT
Location(s)
Primary Investigator
Affiliation
Mathematica Policy Research
Other Primary Investigator(s)
PI Affiliation
Mathematica Policy Research
Additional Trial Information
Status
Withdrawn
Start date
2014-01-15
End date
2015-12-18
Secondary IDs
Abstract
The District of Columbia Public Schools (DCPS) partnered with the Flamboyan Foundation to implement the Family Engagement Partnership (FEP), a two-year school-based intervention that aims to increase family engagement and, ultimately, student achievement. As part of this effort, DCPS selected Mathematica Policy Research, a nonpartisan research firm, to serve as the independent evaluator of FEP. Mathematica conducted a randomized controlled trial with a multi-step screening process to estimate the impacts of FEP on family engagement, student achievement, and other key outcomes. The treatment in this intent-to-treat evaluation was the invitation to apply to FEP, not necessarily the implementation of FEP. In addition to using administrative data to measure student and teacher outcomes, we planned to develop and administer surveys to assess family engagement activities and outcomes. Teacher surveys conducted in spring 2015, spring 2016, and spring 2018 would allow us to examine whether FEP activities differ from usual practices in control schools and estimate impacts on teacher satisfaction and perceptions of student-teacher relationships. We planned to conduct a parent survey in summer 2016 and summer 2018 to measure family engagement outcomes from the parents’ perspective. Findings were to be presented in a 2019 report. However, the study sponsors decided to end the evaluation early due to statistical power considerations. Preliminary findings from this evaluation and another study’s recent findings suggested that impacts of the intervention may be difficult to detect in part because they may be modest in size and not statistically significant with the sample of schools participating in the study.
External Link(s)
Registration Citation
Citation
Glazerman, Steven and Allison McKie. 2016. "Evaluation of the District of Columbia Public Schools Family Engagement Partnership." AEA RCT Registry. June 23. https://doi.org/10.1257/rct.1370-2.0.
Former Citation
Glazerman, Steven and Allison McKie. 2016. "Evaluation of the District of Columbia Public Schools Family Engagement Partnership." AEA RCT Registry. June 23. https://www.socialscienceregistry.org/trials/1370/history/8970.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
FEP is a two-year, school-based intervention that aims to increase family engagement and, ultimately, student achievement, via activities such as home visits, academic parent-teacher teams, and ongoing teacher communication with families.
Intervention Start Date
2014-04-30
Intervention End Date
2015-12-18
Primary Outcomes
Primary Outcomes (end points)
The main outcomes are students’ attachment to school, student achievement, teacher retention, teacher performance, and family engagement.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Mathematica designed a randomized controlled trial with a multi-step screening process. In Step 1, DCPS screened schools based on extant information for suitability in FEP. This step did not involve outreach to candidate schools themselves. Step 2 involved minimal outreach to schools of marginal suitability to gauge interest and capacity. The outreach was not rigorous enough to raise schools’ hopes or change their behavior. Schools passing this screen, however, had to consent to participate in the study. At the conclusion of this step, DCPS provided Mathematica with a list of schools to randomly assign.

In Step 3, the Mathematica evaluation team randomly assigned half of the schools to a treatment group and half to a control group. Mathematica first used school-level data to create pairs of schools that were similar on observable characteristics, such as average student achievement and student demographics. We then consulted with DCPS to ensure that district staff and program implementers who were familiar with the schools deemed each proposed pair of schools to be similar. Once the matched pairs were agreed upon, we randomly assigned one school in each pair to be invited to apply for FEP (treatment group); the other school was not invited to apply (control group).

FEP staff then rigorously screened the treatment schools and worked with the treatment schools that were selected for the intervention. The evaluation team planned to follow all study schools over time. In this intent-to-treat evaluation, schools assigned to treatment remain in the treatment group for the study, regardless of whether they implemented FEP.
Experimental Design Details
The original evaluation plan called for analyzing one cohort of 40 schools (20 treatment and 20 control sites). However, only 18 schools passed the initial FEP screening and entered the randomization pool in January 2014. Nine treatment schools were invited to apply to implement FEP for the 2014–2015 and 2015–2016 school years. The 18 randomly assigned schools by treatment status were as follows: • Treatment schools: Burroughs Education Campus (EC), Burrville, J.O. Wilson, Ketcham, Kimball, Langley, Maury, Savoy, and Takoma EC. • Control schools: Browne EC, Cleveland, Drew, J.C. Nalle, Ludlow-Taylor, Orr, Randle Highlands, Raymond EC, and Whittier EC. The evaluation plan was modified in March 2015 to add a second cohort of 22 schools to be randomly assigned in December 2015. The treatment schools would be invited to apply to implement FEP for the 2016–2017 and 2017–2018 school years. However, the evaluation was terminated prior to assigning the second cohort of schools.
Randomization Method
Mathematica conducted randomization within matched pairs of schools using a random number generator.
Randomization Unit
School.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
40 schools.
Sample size: planned number of observations
Approximately 10,200 students with non-missing administrative data, 400 teachers in the teacher survey sample, and 400 parents in the parent survey sample.
Sample size (or number of clusters) by treatment arms
As planned: 20 treatment schools and 20 control schools.
As implemented: 9 treatment schools and 9 control schools.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Health Media Lab
IRB Approval Date
2015-05-22
IRB Approval Number
Details not available
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers