Delivering Remote Learning in Developing Countries using a Low-Tech Solution

Last registered on November 30, 2022

Pre-Trial

Trial Information

General Information

Title
Delivering Remote Learning in Developing Countries using a Low-Tech Solution
RCT ID
AEARCTR-0010507
Initial registration date
November 28, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 30, 2022, 4:42 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region
Region

Primary Investigator

Affiliation
Monash University

Other Primary Investigator(s)

PI Affiliation
World Bank
PI Affiliation
University of Minnesota

Additional Trial Information

Status
On going
Start date
2022-10-01
End date
2023-10-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Learning losses resulting from school closures are among the most severe global challenges to medium and long-term recovery from COVID-19. School closures in the first two years of the pandemic lasted roughly twice as long in developing countries compared with advanced economies. With the existing structural fault lines in its educational system, Nepal and Pakistan are two such countries staring at this catastrophe today after keeping their schools closed continuously for a long period of time. This challenge is more acute among candidates taking the end-of-secondary school exams. We have developed and adapted a set of audio lessons (podcasts) to be delivered via the Interactive Voice Response (IVR) system. IVR is an automated phone system technology that allows incoming callers to access information via a voice response system of pre-recorded messages without having to speak to an attendant (tutor), as well as to utilize menu options via keypad selection. The intervention will set up an IVR-based toll-free line to deliver English and Mathematics lessons to students who will take the SEE/SSC in 2024, (who will be in ninth grade during the intervention). There will be weekly lesson plans for both subjects (Mathematics and English) that will be accessible during the intervention period. The intervention will also address issues related to students’ educational aspirations and hope. In a separate treatment arm, students will also receive over-the-phone support from a tutor in addition to IVR-based intervention.
External Link(s)

Registration Citation

Citation
Glewwe, Paul, Asad Islam and Shwetlena Sabarwal. 2022. "Delivering Remote Learning in Developing Countries using a Low-Tech Solution." AEA RCT Registry. November 30. https://doi.org/10.1257/rct.10507-1.0
Experimental Details

Interventions

Intervention(s)
In this project, a set of audio lessons (podcast) to deliver via the Interactive Voice Response (IVR) system of the telecommunication platform will be developed. This intervention contains three modules divided into 72 lessons for each of the 2 subjects (English and Mathematics), each lesson with a duration of 10 minutes. The lessons will be delivered via basic mobile phones as the basic mobile phone penetration rate in rural areas of Nepal and Pakistan. is significantly higher than other one-way technologies such as the internet and television.
IVR is an automated phone system technology that allows incoming callers to access information via a voice response system of pre-recorded messages without having to speak to an agent, as well as to utilize menu options via keypad selection. By using IVR, it is possible to provide education services at scale while maintaining some flexibility regarding learning levels and learning delivery schedules. Lessons at different proficiency levels can be stored in a telecom server, and students can learn by choosing lessons at their competence level. IVR lessons can be accessed anytime during the teaching period, allowing students to learn with convenience. The program will last for approximately nine months. The intervention will target grade nine students. In Nepal, the intervention will be implemented by "Edukhabar", which has prior experience implementing comparable interventions. In Pakistan, the "Institute for Education Development (IED) of Aga Khan University (AKU)" will implement the intervention. Both of these organizations have experience implementing large-scale educational interventions in their respective countries.
Intervention Start Date
2022-11-01
Intervention End Date
2023-06-30

Primary Outcomes

Primary Outcomes (end points)
(1) Student Learning
(2) Student’s Aspirations
(3) Student’s Hopes
(4) Student time spent on exam preparation
Primary Outcomes (explanation)
(1) Student Learning: Students’ learning will be measured using a standard assessment test based on the national curricula of the respective countries. There will be two tests, one each for English and Mathematics. To examine the effect of this treatment (IVR lessons and direct assistance) on learning outcomes, we will construct outcome variables: standardized test score, first we will normalize the raw test score by subtracting the mean for the control group and then dividing by the standard deviation of the control sample. We will also collect SEE/SSC results, both aggregate and separately by subject, from the examination board.
Unit of observation: Students
Data source(s): Assessment tests (Mathematics and English Test Scores)
(2) Student’s Aspirations: Using a method like the one employed by Beaman et al. (2012) and Bernard et al. (2014), we will estimate aspirations.
Unit of observation: Students
Data source(s): Baseline and endline surveys
(3) Student’s Hopes: We will measure the hope of the participants by using the Hope Scale, a 6-item self-report measure of students’ perceptions that their goals can be met (Snyder, 1994, Snyder, 2002). There are two categories of hope, with three specific items within each category. The two categories are agency and pathways. We will construct three outcome variables, agency, pathways and the overall hope score.
Unit of observation: Students
Data source(s): Baseline and endline surveys
(4) Student time spent on exam preparation: Students' time investment preparing for the exam will be measured by asking the following questions:
(a) How much time (hours) in various academic activities (i.e., reading, practising exam questions, taking lessons from private tutors etc.) do you spend each week? Total hours spent in various academic activities will be used as an outcome variable for student time investment in academic activities.
(b) To what extent do you spend time on academic activities (5-point Likert-scale response; ‘none’ to ‘a great deal’)
Unit of observation: Minutes spent on exam preparation by each student
Data source(s): Baseline and endline surveys

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The intervention will test whether IVR will improve the grade-level learning of participating students in Nepal and Bangladesh. By offering content of different proficiency levels to the secondary school students, the intervention will in effect offer a menu of choices so that students can continue to learn more if they have current grade level proficiency or catch up with the learning gap if they are behind their grade level. The study will conduct a randomized control trial (RCT). RCT allows to avoid endogeneity issues and identify the causal impact of the intervention. We will randomly assign 225 secondary schools into three groups of equal size:
T1: Self-help group – At the beginning of the program, participants will be briefed by field staff on how to access IVR-based lessons. Also, the field staff will explain how this program could lead to better educational outcomes.
T2: Assisted group – Students in this group will get the same treatment as T1. In addition, they will get a bi-weekly call from a tutor. These tutors will be hired from those within the local community who have the required teaching skills. Students of T1 may face two constraints. The first is lack of willpower: students may lack the motivation to use the lessons regularly. The second is incomprehension: students may fail to understand some of the contents of the one-way audio lessons. To counteract these two possible shortcomings, this additional bi-weekly phone calls should increase participants’ motivation and resolve problems of not understanding the lessons. These discussions will cover the content to be covered in our modules.
T3: Control – No intervention will be provided.
Experimental Design Details
Not available
Randomization Method
Randomization will be done by the researchers in the office by a computer using Stata’s random number generator.
Randomization Unit
A two-stage randomization procedure will be employed. In particular, we first randomly assign 225 schools to two treatment groups and one control group, and then within each school we randomly select 15 students (in a way that ensures almost equal numbers of boys and girls) from each of the 225 schools (for treatment in the treatment schools, and for data collection in the control schools).
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
225 schools (in Nepal)
225 schools (in Pakistan)
Sample size: planned number of observations
3375 students (in Nepal) 3375 students (in Pakistan)
Sample size (or number of clusters) by treatment arms
In Nepal:
T1: Self-help group: 75 schools (units of randomization), 1125 students (units of observation)
T2: Assisted group: 75 schools (units of randomization), 1125 students (units of observation)
T3: Control: 75 schools (units of randomization), 1125 students (units of observation)
In Pakistan:
T1: Self-help group: 75 schools (units of randomization), 1125 students (units of observation)
T2: Assisted group: 75 schools (units of randomization), 1125 students (units of observation)
T3: Control: 75 schools (units of randomization), 1125 students (units of observation)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We assume two alternative effect sizes of 0.18SD, 0.21SD, 80 and 90 percent power, and a type-1 error rate of 5 percent.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number