Remedial Science Education

Last registered on April 15, 2021

Pre-Trial

Trial Information

General Information

Title
Remedial Science Education
RCT ID
AEARCTR-0000379
Initial registration date
May 15, 2014

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 15, 2014, 2:55 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
April 15, 2021, 7:58 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Southern California

Other Primary Investigator(s)

PI Affiliation
Inter-American Development Bank
PI Affiliation
Inter-American Development Bank

Additional Trial Information

Status
Completed
Start date
2014-04-01
End date
2014-12-17
Secondary IDs
Abstract
Recent evidence suggests that remedial education can be effective at closing achievement gaps. Remedial education, by which low-performing students receive targeted, self-paced teaching aimed at mastering basic skills, has improved short- and medium-term academic performance. The proposed research project will use student-level random assignment within schools to evaluate the impact of a remedial education program for low-performing third-grade science students in 48 schools in metropolitan Lima, Peru. Key evaluation outcome measures include students’ understanding of science and the environment and achievement relative to higher performing peers.
External Link(s)

Registration Citation

Citation
Alfonso, Mariana, Emma Naslund-Hadley and Juan Esteban Saavedra. 2021. "Remedial Science Education." AEA RCT Registry. April 15. https://doi.org/10.1257/rct.379-2.0
Former Citation
Alfonso, Mariana, Emma Naslund-Hadley and Juan Esteban Saavedra. 2021. "Remedial Science Education." AEA RCT Registry. April 15. https://www.socialscienceregistry.org/trials/379/history/89642
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The proposed research project will use random assignment to evaluate the impact of a remedial education program for low-performing third-grade science students in 48 schools in metropolitan Lima, Peru.
Intervention Start Date
2014-06-05
Intervention End Date
2014-11-28

Primary Outcomes

Primary Outcomes (end points)
Key evaluation outcome measures are: i) students’ understanding of science and the environment and ii) performance on achievement tests.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Randomization will take place at the student level, with a separate lottery for each participating school. All third grade students from a given school will take the baseline exam that will identify students that place in the lowest half in each school. The evaluation will target 48 schools in metropolitan Lima that participated in either treatment or control groups in the 2012 pilot project.
Experimental Design Details
The evaluation will target low-performing third-grade students from 48 schools in metropolitan Lima that participated in either treatment or control groups in the 2012 pilot project. Within each school, half of the low-performing students will be assigned to the remedial education treatment and half to control conditions. The evaluation design will therefore stratify at the school level.

According to total enrollment levels collected from school principals at baseline in 2012, we anticipate, on average, 48 third-grade students per school distributed in two third-grade sections per school. Logistical reasons explain why we aim to target the same schools as the 2012 pilot. In the treatment schools, UCH already has contacts with the administration, which should facilitate the implementation of the program. In addition, data collection by IPA in the control schools went according to plan and there was little attrition and non-response.

IPA Peru will administer a baseline evaluation test to students in April 2014. This test will be a similar though simplified version of the science test that was used in the 2012 Science Education pilot. This test was developed to assess third-grade science proficiency according to the Peruvian National Curriculum, which includes as subdomains: “Physical World” (“Mundo Físico y conservación del ambiente”), “Human body and health” (“Cuerpo humano y conservación de la salud”) and “Animals and environment conservation” (“Animales y ambiente”). It is a timed self-administered written test that students will be given to complete on their own, without a surveyor. A surveyor will be present at the school to supervise the appropriate and timely administration of the test. Each section (corresponding to one of the three areas of the curriculum) is separately timed.

During the implementation of the program, IPA Peru will develop an end-line test of the same abilities measured for the baseline. The test will be given at the end of the school year to all the third grade students in participating schools to see if those who were placed in the treatment group were able to close the gap with the other students from their class. Both baseline and end-line will be norm-referenced, standardizing all student test scores in standard deviations above or below the control group mean.

In addition, teachers will be surveyed to collect demographic data and to gather their views on different aspects of teaching Science and Environment. Collecting this information at baseline and end-line will permit observing any changes in teacher’s perceptions of teaching science material. School directors of the sample schools will also be surveyed about their experience and characteristics of their schools, including the number of teachers, school enrollment, infrastructure, equipment and learning materials and school climate. Lastly, students will be sent home with questionnaires for parents with questions about the support and assistance that the child receives for homework, the household's socioeconomic status and other characteristics such as household size and education of parents. Both the data collected at baseline from the school directors and parents will serve to ensure balance between treatment and control groups. Additionally, the data will also be used to perform heterogeneity analysis to identify variation in effects between subgroups, such as students in homes with a working television, or students in schools with operational sanitation facilities
Randomization Method
Randomization done in office by a computer, stratified by gender.
Randomization Unit
Randomization units are individual third-grade students who are low-performing according to baseline test. Randomization will be stratified, with one lottery per school (48 schools= 48 lotteries).
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Randomization is at the student level, stratified at the school level. There are 48 schools participating in this trial
Sample size: planned number of observations
48 schools, 1100 students, 100 teachers.
Sample size (or number of clusters) by treatment arms
48 schools, 1100 students, 100 teachers.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
As the Targeted Science Education program will be working with the same schools in the metropolitan Lima area as did the 2012 science education pilot, power calculations rely on the 2012 sample. Specifically, we expect that among the 48 participating schools in metropolitan Lima, there will be approximately 1100 students that place in the lowest half of the baseline achievement distribution. Among these 1100 students, half will be randomly assigned to treatment and the other half will be assigned to control conditions. The attrition rate during the 2012 Science Education pilot was 13 percent, and a more conservative 15 percent is used for these power size calculations. A correlation of 0.576 between baseline and end line was calculated, based on the 2012 sample of students. Assuming a five percent level of significance and 80 percent statistical power, the expected minimum detectable treatment effect is (MDE) 0.16 standard deviations. As the randomization will be stratified by school, this reduces the MDE holding all other parameters constant, making this MDE likely a conservative estimate. We find it is reasonable to conduct an empirical study in which the MDE is 0.16 given the results of previous remedial and tracking programs, ranging from .14 to .28 standard deviations.
IRB

Institutional Review Boards (IRBs)

IRB Name
Human Subjects Committee for Innovations for Poverty Action IRB-USA
IRB Approval Date
2014-04-16
IRB Approval Number
00006083

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
December 17, 2014, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
November 30, 2014, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Individual level randomization within school*gender strata (48 schools).
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
1219 3rd grade students in 48 publics schools in Metropolitan Lima, Peru
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
We present results from the first randomized experiment of a remedial inquiry-based science education program for low-performing elementary students in a developing country. Among third-grade students in 48 low-income public elementary schools in Metropolitan Lima who score in the bottom 50% of their school baseline science distribution, half are randomly assigned to receive remedial inquiry-based science education in after-school sessions, and the remaining half to business as usual control conditions. Assignment to treatment increased endline science achievement by 3 percentiles (0.12 SD) with greater gains for students who attended at least one remedial session, and a concentration of gains among boys. We cannot reject the null hypothesis of no indirect science achievement gains among nonparticipants.
Citation
Remedial Inquiry-Based Science Education: Experimental Evidence From Peru Juan E. Saavedra, Emma Näslund-Hadley, Mariana Alfonso Educational Evaluation and Policy Analysis. Research Article https://doi.org/10.3102/0162373719867081 Volume: 41 issue: 4, page(s): 483-509 Article first published online: August 12, 2019; Issue published: December 1, 2019

Reports & Other Materials