x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Information, Accountability and Family-School Engagement: Evidence from Colombia
Last registered on October 27, 2017

Pre-Trial

Trial Information
General Information
Title
Information, Accountability and Family-School Engagement: Evidence from Colombia
RCT ID
AEARCTR-0002192
Initial registration date
April 27, 2017
Last updated
October 27, 2017 10:05 AM EDT
Location(s)
Region
Primary Investigator
Affiliation
Harvard University
Other Primary Investigator(s)
PI Affiliation
Harvard University
Additional Trial Information
Status
On going
Start date
2014-04-01
End date
2018-03-31
Secondary IDs
Abstract
The city of Manizales, capital of the Department of Caldas in central Colombia, is a mid-size Colombian city, with population close to half a million, and approximately 55,000 students in public basic education grades 1 to 11. Manizales is undertaking a major revision of its educational programs under the leadership of the mayor and in conjunction with a very active civic society. Authorities are interested in tackling the perceived low quality of education, as shown in national and international standardized tests. Our intervention aims to address the quality problem directly, and encompasses two experiments, one at the household level (information intervention) and one at the school level (family/school engagement intervention). The information intervention is designed to change the internal investment of the family as well as their relationship with the school. The family/school engagement intervention aims to increase engagement, trust, and communication between families and teachers/schools. We are interested to estimate average treatment effects and heterogeneity effects by test performance at baseline. The hypothesis is that parents from low-baseline performers have less information and may react more to treatment. We are interested in the independent impact of family treatment and school treatment, but also in the interaction between them. The unit of observation of the household experiment is household level; the unit of observation of the school experiment, will be the teacher level.
External Link(s)
Registration Citation
Citation
Barrera-Osorio, Felipe and David Deming. 2017. "Information, Accountability and Family-School Engagement: Evidence from Colombia." AEA RCT Registry. October 27. https://doi.org/10.1257/rct.2192-2.0.
Former Citation
Barrera-Osorio, Felipe, Felipe Barrera-Osorio and David Deming. 2017. "Information, Accountability and Family-School Engagement: Evidence from Colombia." AEA RCT Registry. October 27. http://www.socialscienceregistry.org/trials/2192/history/22761.
Experimental Details
Interventions
Intervention(s)
Our study combines efforts to provide information to parents with a family engagement intervention, using an experimental design. The experimental design is divided in three phases. In the first and second phase, we provide report cards on school and student performance to parents. In the third phase, we will incorporate the family engagement component.

In the first phase, we randomly assigned families into three groups. In the first group (Treatment Group 1) we provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. The information is essentially a percentile rank (e.g. 50th percentile), presented in a way that is highly salient to all families. In the second group (Treatment Group 2) we provided families with a one-page report card that shows the average score of their child’s school in comparison to the average score across all schools in Manizales. Like treatment #1, the information is a percentile rank that is translated into a reader-friendly format. In the third group (Control Group) no information was provided.

We modified the information treatments from Phase 1 to Phase 2. In the first group (Treatment Group 1) we provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. Also included with the report card was a list of suggestions for parents to engage with their children’s education. In the second group (Treatment Group 2) we provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade in the entire city. The same list of suggestions provided to parents in Treatment Group 1 was provided to parents in Treatment Group 2. In the third group (Control Group) no information was provided.

In Phase 3 of the study, we add a family engagement component designed to improve trust and communication between the family and the school. Teachers were randomly assigned to two group. In the Treatment Group, teachers received the test score information collected for the students in their class. Teachers also received a list of suggestions to promote improve family-school engagement, with two components. First, teachers received suggestions for teachers on how to improve communication with families. Second, teachers received suggestions on how to encourage families to engage with their children’s education outside of the school. This is the same list of suggestions that is provided to parents in the treatment groups in Phase 2. In the Control Teachers, teachers did not receive test score information or the list of suggestions to promote family-school engagement.
Intervention Start Date
2014-10-01
Intervention End Date
2016-12-01
Primary Outcomes
Primary Outcomes (end points)
The Early Grade Reading Assessment (EGRA) and the Early Grade Math Assessment (EGMA). Families investment in education. Gap between parents' believes and actual EGRA and EGMA.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Phase 1: Pilot Study (October 2014 – January 2015)

In April 2014, the project collected language and math test scores of students in grades 4 and 5. The first phase of the experiment consisted of two information treatments plus a control group. These students were randomly assigned into three groups: two treatment groups and one control group.

• Treatment Group 1: Individual student performance – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. The information is essentially a percentile rank (e.g. 50th percentile), presented in a way that is highly salient to all families.
• Treatment Group 2: School average performance – We provided families with a one-page report card that shows the average score of their child’s school in comparison to the average score across all schools in Manizales.
• Control Group: No information was provided.

To conduct the pilot, we randomly selected within the three groups a subsample to conduct the home visits.

Phase 2: Expanded Study (February 2015 – April 2016)

Phase 2, the expanded study, was conducted between February 2015 and April 2016. In June 2015, the project collected language and math test scores of the students who were randomly assigned to the treatment or control conditions in Phase 1. In addition, the project collected language and math test scores of students who were in grade 4 in the schools targeted in Phase 1. These new students were randomly assigned into three groups: two treatments and a control group. We modified the information treatments from Phase 1 to Phase 2:

• Treatment Group 1: Individual student performance in the school – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. Also included with the report card was a list of suggestions for parents to engage with their children’s education.
• Treatment Group 2: Individual student performance in the city – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade in the entire city. The same list of suggestions provided to parents in Treatment Group 1 was provided to parents in Treatment Group 2.
• Control Group: No information was provided.

We conducted a second home visit in November 2015. During this home visit, we provided information to parents of the students who had been assigned to either treatment group in Phase 1. The information provided was based on the modified treatment arms described above. We also provided information to the parents of the students who had been assigned to either treatment group in Phase 2. These home visits followed the same format as the home visits in Phase 1.

Phase 3: Full Study (May 2016 – August 2017)

The third phase of the study is different to the pilot study in two main senses. First, we expand the sample to all students in the grades of interest. As we described, in 2014 and 2015 students from 31 schools in grades 4, 5, and 6 were included. Beginning in 2016, we will not only follow those students that are in grades 5 and 6, but we will also include all students in grades 4, 5 and 6 that are enrolled in one of the 31 schools. We will not be following students who were in grade 5 in Phase 1 and are in grade 7 in Phase 3. Second, we will add a family engagement component.

In May 2016 we administered language and math tests to all students in grades 4, 5 and 6 within the 31 schools. This sample includes all students who were assigned to one of the three information intervention groups in Phase 1 and 2. Students who did not participate in Phase 1 or Phase 2 were not assigned to an information intervention group.

In addition to continuing to collect follow-up information for the information intervention, we included a family engagement component designed to improve trust and communication between the family and the school. In mid 2016, teachers were randomly assigned to one of two groups:

• Treatment Group: Family Engagement Intervention – Teachers received the test score information collected in May 2016 for the students in their class. Teachers also received a list of suggestions to promote improve family-school engagement, with two components. First, teachers received suggestions for teachers on how to improve communication with families. Second, teachers received suggestions on how to encourage families to engage with their children’s education outside of the school. This is the same list of suggestions that is provided to parents in the treatment groups in Phase 2.
• Control Group: No Family Engagement Intervention – Teachers will not receive test score information or the list of suggestions to promote family-school engagement.

At the end of each academic term we administered language and math tests to all students in the sample.
Experimental Design Details
Phase 1: Pilot Study (October 2014 – January 2015) In April 2014, the project collected language and math test scores of 3,652 students in grades 4 and 5, corresponding to nearly 45% of the students of these grades. The first phase of the experiment consisted of two information treatments plus a control group. After merging test score data with administrative data the final sample for this pilot phase contains 3,026 students. These students were randomly assigned into three groups: two treatment groups (each with 1,008 students) and one control group (1,010 students). • Treatment Group 1: Individual student performance – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. The information is essentially a percentile rank (e.g. 50th percentile), presented in a way that is highly salient to all families. • Treatment Group 2: School average performance – We provided families with a one-page report card that shows the average score of their child’s school in comparison to the average score across all schools in Manizales. • Control Group: No information was provided. To conduct the pilot, we randomly selected within the three groups a subsample of 2,100 students (800 in each treatment group, plus 400 in the control group) to conduct the home visits. Phase 2: Expanded Study (February 2015 – April 2016) Phase 2, the expanded study, was conducted between February 2015 and April 2016. In June 2015 , the project collected language and math test scores of the 3,026 students who were randomly assigned to the treatment or control conditions in Phase 1. In addition, the project collected language and math test scores of 1,345 students who were in grade 4 in 32 schools targeted in Phase 1. These 1,345 new students were randomly assigned into three groups: two treatments (each with 448 students) and a control group (with 449 students). We modified the information treatments from Phase 1 to Phase 2: • Treatment Group 1: Individual student performance in the school – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. Also included with the report card was a list of suggestions for parents to engage with their children’s education. • Treatment Group 2: Individual student performance in the city – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade in the entire city. The same list of suggestions provided to parents in Treatment Group 1 was provided to parents in Treatment Group 2. • Control Group: No information was provided. Data in the second phase include the 3,026 students in Phase 1 plus an additional 1,345 students in grade 4, for a total sample of 4,371. We conducted a second home visit in November 2015. During this home visit, we provided information to parents of the 2,016 students who had been assigned to either treatment group in Phase 1. The information provided was based on the modified treatment arms described above. We also provided information to the parents of the 896 students who had been assigned to either treatment group in Phase 2. These home visits followed the same format as the home visits in Phase 1. Phase 3: Full Study (May 2016 – August 2017) The third phase of the study is different to the pilot study in two main senses. First, we expand the sample to all students in the grades of interest. As we described, in 2014 and 2015 students from 31 schools in grades 4, 5, and 6 were included. Beginning in 2016, we will not only follow those students that are in grades 5 and 6, but we will also include all students in grades 4, 5 and 6 that are enrolled in one of the 31 schools. We will not be following students who were in grade 5 in Phase 1 and are in grade 7 in Phase 3. Second, we will add a family engagement component. In May 2016 we administered language and math tests to all students in grades 4, 5 and 6 within the 31 schools. This sample includes all students who were assigned to one of the three information intervention groups in Phase 1 and 2. Students who did not participate in Phase 1 or Phase 2 were not assigned to an information intervention group. In addition to continuing to collect follow-up information for the information intervention, we included a family engagement component designed to improve trust and communication between the family and the school. In mid 2016, teachers were randomly assigned to one of two groups: • Treatment Group: Family Engagement Intervention – Teachers received the test score information collected in May 2016 for the students in their class. Teachers also received a list of suggestions to promote improve family-school engagement, with two components. First, teachers received suggestions for teachers on how to improve communication with families. Second, teachers received suggestions on how to encourage families to engage with their children’s education outside of the school. This is the same list of suggestions that is provided to parents in the treatment groups in Phase 2. • Control Group: No Family Engagement Intervention – Teachers did not receive test score information or the list of suggestions to promote family-school engagement. At the end of each academic term we administered language and math tests to all students in the sample.
Randomization Method
Randomization done by computer by the researchers
Randomization Unit
For the information treatment in Phase 1 and Phase 2 the randomization unit is the individual student. For the family engagement treatment in Phase 3, the randomization unit is the teacher.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
32 schools
Sample size: planned number of observations
Phase 1 of the study includes 3,026 students. Phase 2 of the study includes 4,371 students. Phase 3 of the study includes 5,887 students.
Sample size (or number of clusters) by treatment arms
Phase 1 of the study includes 1,008 students in treatment group 1 (individual student performance), 1,008 in treatment group 2 (school average performance), and 1,010 in the control group.

Phase 2 of the study includes 1,456 students in treatment group 1 (individual performance in the school), 1,456 students in treatment group 2 (individual performance in the city), and 1,459 students in the control group.

The information intervention in Phase 3 of the study includes 921 students in treatment group 1 (individual performance in the school), 921 students in treatment group 2 (individual performance in the city), and 923 students in the control group.

The family engagement intervention in Phase 3 of the study includes 2,658 students in the treatment group and 2615 students in the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We ran power calculations for the information intervention and for the family engagement intervention separately. In the latter case, we assumed randomization among 300 teachers. Assuming a power of 0.8, the MDE for Grade 4 student arm is 0.03sd for the original sample (1,416 students) and 0.02 for the expansion (2,500 students). The MDE for Grade 5 are 0.04 sd (original sample of 1,601 students) and 0.03 (expansion sample, 2,800 students). The MDE for the family engagement intervention is 0.08 sd (Grade 4) and 0.13 (Grade 5). In short, we have sufficient power to detect plausible effects.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Committee on the Use of Human Subjects, Harvard University
IRB Approval Date
2014-09-29
IRB Approval Number
IRB14-2842
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS