Delivering Remote Learning in Developing Countries using a Low-tech Solution – Nepal

Last registered on September 16, 2023

Pre-Trial

Trial Information

General Information

Title
Delivering Remote Learning in Developing Countries using a Low-tech Solution – Nepal
RCT ID
AEARCTR-0011078
Initial registration date
March 12, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 13, 2023, 3:33 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
September 16, 2023, 6:52 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Monash University

Other Primary Investigator(s)

PI Affiliation
University of Minnesota
PI Affiliation
University of Oxford
PI Affiliation
Institute for Social and Environmental Research (ISER-N)
PI Affiliation
Education Review Office, Kathmandu University
PI Affiliation
World Bank
PI Affiliation
Yale-NUS College

Additional Trial Information

Status
In development
Start date
2022-11-01
End date
2023-12-29
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Learning losses resulting from school closures are among the most severe global challenges
to medium and long-term recovery from COVID-19. School closures in the first two years of
the pandemic lasted roughly twice as long in developing countries compared with advanced
economies. With the existing structural fault lines in its educational system, Nepal is
staring at this catastrophe today after keeping their schools
closed continuously for a long period of time. This challenge is more acute among
candidates taking the end-of-secondary school exams. We have developed and adapted a
set of audio lessons (podcasts) to be delivered via the Interactive Voice Response (IVR)
system. IVR is an automated phone system technology that allows incoming callers to
access information via a voice response system of pre-recorded messages without having to
speak to an attendant (tutor), as well as to utilize menu options via keypad selection. The
intervention will set up an IVR-based toll-free line to deliver English and Mathematics
lessons to students who will take the SEE in 2024, (who will be in ninth grade during
the intervention). There will be weekly lesson plans for both subjects (Mathematics and
English) that will be accessible during the intervention period. The intervention will also
address issues related to students’ educational aspirations and hope. In a separate treatment
arm, students will also receive over-the-phone support from a tutor in addition to IVR-based
intervention. In this study, we examine whether providing remote learning opportunities
through IVR improves children's learning, time spent on educational activities, hopes and
aspirations.
External Link(s)

Registration Citation

Citation
Acharya, Deviram et al. 2023. "Delivering Remote Learning in Developing Countries using a Low-tech Solution – Nepal ." AEA RCT Registry. September 16. https://doi.org/10.1257/rct.11078-4.0
Experimental Details

Interventions

Intervention(s)
The intervention will test whether IVR will improve the grade-level learning of participating
students in Nepal. By offering content of different proficiency levels to the
secondary school students, the intervention will in effect offer a menu of choices so that
students can continue to learn more if they have current grade level proficiency or catch up
with the learning gap if they are behind their grade level. The study will conduct a
randomized control trial (RCT). RCT allows to avoid endogeneity issues and identify the
causal impact of the intervention. We will randomly assign 223 secondary schools into three
groups of equal size:
T1: Self-help group – At the beginning of the program, participants will be briefed by field
staff on how to access IVR-based lessons. Also, the field staff will explain how this program
could lead to better educational outcomes.
T2: Assisted group – Students in this group will get the same treatment as T1. In addition,
they will get a bi-weekly call from a tutor. These tutors will be hired from those within the local
community who have the required teaching skills. Students of T1 may face two constraints.
The first is lack of willpower: students may lack the motivation to use the lessons regularly.
The second is incomprehension: students may fail to understand some of the contents of the
one-way audio lessons. To counteract these two possible shortcomings, this additional biweekly phone calls should increase participants’ motivation and resolve problems of not
understanding the lessons. These discussions will cover the content to be covered in our
modules.
T3: Control – No intervention will be provided.
Intervention (Hidden)
Intervention Start Date
2023-04-17
Intervention End Date
2023-10-13

Primary Outcomes

Primary Outcomes (end points)
(1) Student Learning
(2) Student’s Aspirations
(3) Student’s Hopes
(4) Student time spent on exam preparation
Primary Outcomes (explanation)
a) Student Learning outcomes
Standardized test score of English exams at the endline
Standardized test score of Mathematics exams at the endline
Notes on the construction of the test score outcomes:
Students’ learning will be measured using a standard assessment test based on the national
curriculum of Nepal. There will be two tests, one each for English and Mathematics. To
examine the effect of this treatment (IVR lessons and direct assistance) on learning
outcomes, we will construct the outcome variable by standardizing the raw test score with
respect to the mean and standard deviation of the control group. We will also standardize the
baseline test scores with respect to the baseline control mean and standard deviation.
Unit of observation: Students
Data source(s): Assessment tests (Mathematics and English Test Scores)
b) Student Aspirations: Using a method like the one employed by Beaman et al. (2012)
and Bernard et al. (2014), we will estimate aspirations.
Unit of observation: Students
Data source(s): Baseline and endline surveys
c) Student hope: We will measure the hope of the participants by using the Hope Scale,
a 6-item self-report measure of students’ perceptions that their goals can be met (Snyder,
1994, Snyder, 2002). There are two categories of hope, with three specific items within each
category. The two categories are agency and pathways. We will construct three outcome
variables, agency, pathways and the overall hope score.
Unit of observation: Students
Data source(s): Baseline and endline surveys
d) Student time spent on exam preparation: Students' time investment preparing for the
exam will be measured by asking the following questions:
(a) How much time (hours) in various academic activities (i.e., reading, practising exam
questions, taking lessons from private tutors etc.) do you spend each week? Total hours
spent in various academic activities will be used as an outcome variable for student time
investment in academic activities.
(b) To what extent do you spend time on academic activities (5-point Likert-scale response;
‘none’ to ‘a great deal’)
Unit of observation: Minutes spent on exam preparation by each student
Data source(s): Baseline and endline surveys

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This study uses an RCT to assess whether there is a causal link between the program and
changes in outcomes. The evaluation of this employs a three-arm clustered RCT design (two
treatment groups and one control group).
Randomization is done in 2 stages: We first pick 223 schools from a pre-existing list
provided by our partner organizations. We then randomly distribute these 223 schools into
the three study arms: T1 (self-help), T2 (assisted), and C (control).
T1 (self-help) and T2 (assisted) will have 74 schools each and Control will have 75 schools.
From each school, we will randomly select 15 students on average to participate in the
program. We will ensure that about half of the students are female when we randomly select
the students to be treated from each treatment school.
The intervention contains three study arms:
a) T1: Self-help group (74 schools, 1057 students selected to receive treatment) –will only be provided with information on
how to access IVR-based lessons and how this program could result in improved
educational outcomes (such as higher marks on the SSC exams). Participants will receive
regular text messages from the IVR system.
b) T2: Assisted group (74 schools, 1060 students selected to receive treatment) – Will receive the same information as
students in T1. Additional treatment will be provided in the form of biweekly phone calls from
a tutor. Participants will also receive regular text messages from the IVR system.
c) T3: Control (75 schools, 2170 students) - This group of students will not be receiving any
intervention.
Experimental Design Details
Randomization Method
Randomization done in office by a computer using Stata/BE 17.0
Randomization Unit
A two-stage randomization procedure will be employed. In particular, we first randomly
assign 223 schools to two treatment groups and one control group, and then within each
school we randomly select 15 students on average (in a way that ensures almost equal
numbers of boys
and girls) from each of the 223 schools (for treatment in the treatment schools, and for data
collection in the control schools).
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
223 Schools
Sample size: planned number of observations
6427 Students
Sample size (or number of clusters) by treatment arms
T1: Self-help group: 74 schools (units of randomization), 1057 students selected for treatment (units of observation)
T2: Assisted group: 74 schools (units of randomization), 1060 students selected for treatment (units of observation)
Control: 75 schools (units of randomization), 2170 students (units of observation)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We consider a 0.23 SD effect size, alpha levels of 0.05, and a 0.2 intra-cluster correlation (ICC) to achieve 80% statistical power.
IRB

Institutional Review Boards (IRBs)

IRB Name
Monash University Human Research Ethics Committee
IRB Approval Date
2022-08-01
IRB Approval Number
34569
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials