Experimental Design Details
The evaluation will target low-performing third-grade students from 48 schools in metropolitan Lima that participated in either treatment or control groups in the 2012 pilot project. Within each school, half of the low-performing students will be assigned to the remedial education treatment and half to control conditions. The evaluation design will therefore stratify at the school level.
According to total enrollment levels collected from school principals at baseline in 2012, we anticipate, on average, 48 third-grade students per school distributed in two third-grade sections per school. Logistical reasons explain why we aim to target the same schools as the 2012 pilot. In the treatment schools, UCH already has contacts with the administration, which should facilitate the implementation of the program. In addition, data collection by IPA in the control schools went according to plan and there was little attrition and non-response.
IPA Peru will administer a baseline evaluation test to students in April 2014. This test will be a similar though simplified version of the science test that was used in the 2012 Science Education pilot. This test was developed to assess third-grade science proficiency according to the Peruvian National Curriculum, which includes as subdomains: “Physical World” (“Mundo Físico y conservación del ambiente”), “Human body and health” (“Cuerpo humano y conservación de la salud”) and “Animals and environment conservation” (“Animales y ambiente”). It is a timed self-administered written test that students will be given to complete on their own, without a surveyor. A surveyor will be present at the school to supervise the appropriate and timely administration of the test. Each section (corresponding to one of the three areas of the curriculum) is separately timed.
During the implementation of the program, IPA Peru will develop an end-line test of the same abilities measured for the baseline. The test will be given at the end of the school year to all the third grade students in participating schools to see if those who were placed in the treatment group were able to close the gap with the other students from their class. Both baseline and end-line will be norm-referenced, standardizing all student test scores in standard deviations above or below the control group mean.
In addition, teachers will be surveyed to collect demographic data and to gather their views on different aspects of teaching Science and Environment. Collecting this information at baseline and end-line will permit observing any changes in teacher’s perceptions of teaching science material. School directors of the sample schools will also be surveyed about their experience and characteristics of their schools, including the number of teachers, school enrollment, infrastructure, equipment and learning materials and school climate. Lastly, students will be sent home with questionnaires for parents with questions about the support and assistance that the child receives for homework, the household's socioeconomic status and other characteristics such as household size and education of parents. Both the data collected at baseline from the school directors and parents will serve to ensure balance between treatment and control groups. Additionally, the data will also be used to perform heterogeneity analysis to identify variation in effects between subgroups, such as students in homes with a working television, or students in schools with operational sanitation facilities