Using technology-enabled instruction to address cross-school, within-system heterogeneity in learning outcomes in Rwanda

Last registered on October 17, 2023

Pre-Trial

Trial Information

General Information

Title
Using technology-enabled instruction to address cross-school, within-system heterogeneity in learning outcomes in Rwanda
RCT ID
AEARCTR-0012245
Initial registration date
October 05, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 17, 2023, 10:52 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
University of Virginia

Other Primary Investigator(s)

PI Affiliation
RwandaEQUIP
PI Affiliation
NewGlobe

Additional Trial Information

Status
In development
Start date
2023-10-09
End date
2024-07-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Learning outcomes in low- and middle-income countries (LMICs) are consistently low (World Bank, 2018), yet curriculum expectations remain much higher than the benchmarks most pupils have met. This indicates a misalignment between pupil performance and policymaker expectations, and it is usually regarded as one of the contributing factors to the ongoing learning crisis (Pritchett and Beatty, 2015; Rodriguez-Segura and Mbiti, 2022). In addition to low average learning levels across many LMICs, researchers have also documented considerable heterogeneity in learning outcomes within classrooms (Muralidharan et al., 2019). This heterogeneity makes it difficult to deliver instruction that reaches most or all pupils in these classrooms, as teachers can deliver more targeted and effective instruction when classes are more homogenous (Duflo et al., 2011). In this sense, all else being equal, a broader baseline distribution of performance will result in instruction that only reaches a smaller number of students.
Given these pedagogical challenges posed by within-class heterogeneity, the literature has been appropriately concerned about the issue of within-class heterogeneity. Yet, on a macro-level, policymakers face a similar challenge with within-system heterogeneity when they perform activities like designing and implementing curricula, developing mastery benchmarks, and writing textbooks. They typically need to select a narrow level of learning to cater to in each grade, and it is therefore unlikely that they will effectively meet the needs of all pupils. Hence, collectively, the existing literature has not focused as much on within-system heterogeneity of foundational learning; instead, it often focuses on either overall low learning outcomes (World Bank, 2018) or discusses within-class heterogeneity (Muralidharan et al., 2019).
The present study aims to (1) quantify the extent to which within-system heterogeneity is present in a sample of 246 public schools across Rwanda and how this is mapped in conjunction with these pupils’ respective pedagogical needs toward mastering foundational literacy and numeracy, and (2) implement a randomized controlled trial where, in the treatment group, pupils’ pedagogical needs in each school*grade pair in Primary 4-6 are diagnosed at the start of the school year. Their teachers are then provided with customized teacher guides for the full year on literacy and numeracy. The control condition receives the same material across the board, regardless of their specific school*grade level, closer to the median level in each grade across the system.
External Link(s)

Registration Citation

Citation
Rodriguez Segura, Daniel, Tim Sullivan and Elisabeth Jeannette Turner. 2023. "Using technology-enabled instruction to address cross-school, within-system heterogeneity in learning outcomes in Rwanda." AEA RCT Registry. October 17. https://doi.org/10.1257/rct.12245-1.0
Experimental Details

Interventions

Intervention(s)
This study will consist of a three-part exercise: (1) At the beginning of the school year, data on foundational literacy and numeracy will be collected from a representative subsample of each school*grade pair in the study. (2) Each school*grade pair will then be randomized into either a treatment or control condition, stratifying by school size and cohort participating in the overall RwandaEQUIP program. (3) In the control condition, each class in the same grade will receive the same curriculum level across schools, similar to the approach used in most education systems worldwide, where the curriculum for a grade is standardized across all schools. However, this leveling was determined using system-wide learning data, which may result in it being closer to the median student's level in this system compared to other education systems globally. In contrast, the treatment condition will receive numeracy and literacy levels that are customized for the median student in their respective school*grade*subject. This intervention will last a full academic year, assessing whether a more tailored leveling of instruction on foundational learning skills in grades Primary 4-6 leads to higher learning outcomes.
Intervention Start Date
2023-10-09
Intervention End Date
2024-07-30

Primary Outcomes

Primary Outcomes (end points)
A uniform numeracy assessment across treatment and control schools. Oral reading fluency and reading comprehension data to assess literacy.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The treatment assignment was stratified using school size and cohort entering into the RwandaEQUIP programme. The trial will be limited to Primary 4-6 classes. Leveling decisions in the treatment group are made based on the median pupil performance in each school*grade*subject. The control condition was assigned to approximately 20% of all school*grade pairs.
Experimental Design Details
Not available
Randomization Method
Randomization performed using Stata 18 by researchers.
Randomization Unit
School*grade pairs.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
733 school*grade pairs, across 246 schools.
Sample size: planned number of observations
We aim to have administrative outcome data for at least 10 pupils in each school*grade pair, along with any additional data that is able to be collected via enumerators (budget pending).
Sample size (or number of clusters) by treatment arms
147 school*grade pairs out of the 733 pairs in the sample are part of the control condition (20%)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Using 733 clusters, 10 students per cluster, covariates explaining part of the variance via initial stratification, and 20% of the clusters assigned to treatment, we expect a minimum detectable effect of 0.15 SD.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number