Back to History Current Version

Evaluation of the "Schools for Change" program in Rwanda

Last registered on December 23, 2020
View Trial History

Pre-Trial

Trial Information

General Information

Title
Evaluation of the "Schools for Change" program in Rwanda
RCT ID
AEARCTR-0005867
Initial registration date
May 20, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 20, 2020, 10:46 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 23, 2020, 7:49 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
Rwanda
Region
Burera
Country
Rwanda
Region
Gatsibo

Primary Investigator

Name
Maggie Vinyard
Affiliation
Laterite
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Zia Kahn
PI Affiliation
Laterite
Contact Investigator
PI Name
Carlo Menon
PI Affiliation
Laterite
Contact Investigator

Additional Trial Information

Status
On going
Start date
2019-02-01
End date
2021-12-31
Keywords
Education
Additional Keywords
Positive discipline, early grade reading assessment
JEL code(s)
Secondary IDs
Abstract
The objective of Schools for Change is to strengthen child protection systems in schools and communities in order to improve learning outcomes for students. The project helps students in Rwanda learn to read and write in safe, child-friendly schools and supportive communities. One primary component of the intervention is to promote the use of positive discipline and reduce the use of physical and humiliating punishments.

The main research questions for this evaluation are whether Schools for Change improve child protection and reduce violence against students (particularly physical or humiliating punishments) in primary schools, community and homes; and whether Schools for Change improve learning outcomes for primary student. The research also attempts to assess the following secondary outcomes: how does Schools for Change lead to improvements in learning outcomes (e.g. reduced absenteeism, greater socioemotional development, improved referral mechanisms)? How effective is Schools for Change at improving teachers’, school leaders’, community reading club facilitators’ and parents’ knowledge, attitudes and practices related to child protection (particularly around physical or humiliating punishments)?

The evaluation design is a two-year cluster Randomized Controlled Trial (RCT). Baseline data collection took place in February - March 2019. Endline data collection was originally planned between July and October 2020, but due to changes in the school calendar for COVID will tentatively take place in July 2021. The sample is clustered at sector (an administrative spatial unit in Rwanda, smaller than the district) and class level, and the unit of analysis is the student. The unit of randomization is the sector. The sample is composed of 1260 students distributed across 105 classes in 17 sectors. Adequate econometric techniques are deployed to deal with the "small number of clusters" problem.

External Link(s)

Registration Citation

Citation
Vinyard, Maggie, Zia Kahn and Carlo Menon. 2020. "Evaluation of the "Schools for Change" program in Rwanda." AEA RCT Registry. December 23. https://doi.org/10.1257/rct.5867-3.0
Sponsors & Partners

Partner

Name
Save the Children Rwanda
Type
ngo
URL
https://rwanda.savethechildren.net/

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The objective of Schools for Change is to strengthen child protection systems in schools and communities in order to improve learning outcomes for students. The project helps students in Rwanda learn to read and write in safe, child-friendly schools and supportive communities. The Schools for Change program builds on Save the Children Rwanda’s long-standing Positive Discipline program in Rwanda. One primary component of the intervention is to promote the use of positive discipline and reduce the use of physical and humiliating punishments.

Save the Children is implementing the program in all primary schools in all ten sectors in Gatsibo and expanding the program in Burera district to an additional seven sectors (they implemented the program in the other sectors in Burera previously). Though Schools for Change is implemented in four of the 30 districts of Rwanda, this research only considers the Burera and Gatsibo districts. The program was implemented in the other districts prior to the start of the research, making them unsuitable for the trial.

The Schools for Change intervention in this trial consists of the following main components:
- Teacher training on Positive Discipline in Everyday Teaching (PDET)
- Parent training on Positive Discipline in Everyday Parenting (PDEP)
- Provision of books with child protection content to existing Mureke Dusome community reading clubs
- Community literacy volunteer (those who lead reading clubs) training on Positive Discipline
- Developing and improving child protection referral mechanisms, including:
- Strengthening referral mechanisms in schools and communities
- Raising awareness about child protection referral mechanisms in schools and communities and improving the protection of students from violence.
Intervention Start Date
2019-04-01
Intervention End Date
2020-07-31

Primary Outcomes

Primary Outcomes (end points)
Student literacy proficiency
Primary Outcomes (explanation)
The main outcome variables are indicator of Kinyarwanda reading proficiency obtained from the Early Grade Reading Assessment (EGRA). One set of outcomes is the score from each of the seven EGRA subtasks - letters, syllables, words, two passages, and two corresponding sets of reading comprehension questions. For letters, syllables, words, and passages, the score is the percentage of items read correctly. For comprehension, it is the percentage of questions answered correctly. A second set of outcomes is reading fluency: for three sub-tasks - letters, syllables, and words - this will be calculated as the number if items read correctly per minute.

Secondary Outcomes

Secondary Outcomes (end points)
The study considers several intermediate outcome variables at student level: experience with physical and humiliating punishments both individually and in the classroom; experience with positive discipline both individually and in the classroom; safety perceptions; school motivation; absenteeism and tardiness.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This evaluation is a two-year cluster Randomized Controlled Trial (RCT). The RCT methodology relies upon the random allocation of half the sample to treatment, and half the sample to control. The unit of randomization in this study is the sector. Given that some components of the intervention – such as the implementation of stronger child protection reporting mechanisms – is being implemented through the community at the sector level, sector was chosen to reduce contamination between treatment and control. It is impossible to isolate some schools (or clusters of a smaller size within a sector) from this component of the treatment. From the 17 sectors included in this trial, Laterite randomly assigned half the sectors to treatment and half to control, resulting in nine treatment sectors and eight control sectors. All schools in each sector are assigned the same treatment status.
Experimental Design Details
Randomization Method
Randomization was done with the Stata software applying a "re-randomization" method to increase balance.
Using sector as the unit of randomization, this evaluation design has only 17 randomization units. While simple randomization ensures that on average treatment and control groups have identical characteristics, with this small number of units to randomize the law of large numbers does not hold, and there is a high chance that two groups assigned using simple randomization will have different characteristics (Bruhn & Mckenzie, 2008), including a substantially different number of schools and sectors in each group.
To mitigate this risk, we applied an optimization technique to the randomization to ensure balanced treatment and control groups. We had little data available on the characteristics of the sectors and schools in the study, so we focused on ensuring balance in the number of schools in the treatment and control group. Having a balanced number of schools in each group is critical because it maximizes the power of the study. The pre-analysis plan provides a detailed description of the re-randomization techniques applied and of the implications for inference.
Randomization Unit
Sector
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
17 sectors
Sample size: planned number of observations
1260 students, 105 schools
Sample size (or number of clusters) by treatment arms
9 sectors treatment, 8 sectors control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Using a monte carlo simulation for the power calculation: For a minimum detectable effect of 0.5 standard deviation, assuming an ANCOVA regression model, the study has a power of 0.84 with clustered standard errors or 0.78 using a wild bootstrap method to estimate the standard errors.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Rwanda National Ethics Committee
IRB Approval Date
2019-01-28
IRB Approval Number
00001497 of IORG0001100
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Yes
Data Collection Completion Date
March 31, 2019, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
105 schools
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
1260 students
Final Sample Size (or Number of Clusters) by Treatment Arms
Treatment: 54 schools and 648 students Control: 51 schools and 612 students
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials