Foregone and Forgotten Learning: COVID-related Learning Losses from National Assessments in Bangladesh and experimental evaluation of a learning takeup program

Last registered on December 26, 2022

Pre-Trial

Trial Information

General Information

Title
Foregone and Forgotten Learning: COVID-related Learning Losses from National Assessments in Bangladesh and experimental evaluation of a learning takeup program
RCT ID
AEARCTR-0010184
Initial registration date
October 11, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 17, 2022, 4:00 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 26, 2022, 12:32 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
The World Bank

Other Primary Investigator(s)

PI Affiliation
New York University
PI Affiliation
The World Bank
PI Affiliation
University of Oxford
PI Affiliation
The World Bank

Additional Trial Information

Status
On going
Start date
2022-05-14
End date
2023-01-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Bangladesh witnessed an overall 1.5 years of school closures due to the COVID-19 pandemic. Research from several countries provides evidence that school closures have generated steep learning losses. We conducted a large-scale learning assessment in Bangladesh comprising 7,218 grade 9 students to determine the current learning levels of students after 14.5 academic months of school closures. This was based on the National Assessment of Secondary Students (NASS) 2019 framework and covered a nationally representative sample of 300 schools. This study tracks 4997 students who took NASS 2019 and administered psychometrically linked tests in Math and Language (Bangla). The study includes Grade 8 and NASS 2019 grade 6 questions to estimate forgotten and forgone learning. The initial results show that for grade 9 students, 14.5 months of COVID-related school closures led to 26 months of learning loss among secondary students. Around 69 percent of learning losses were students failing to learn new skills (learning forgone), but 31 percent of learning losses were students forgetting skills they had already acquired (learning forgotten). There was also a sharp increase in learning inequality. The poorest students lost 50% more learning than the most affluent students.

The study also comprised a phone-based Interactive Voice Response (IVR) lesson to assess the demand for additional instruction. The students were randomly assigned to three groups (one control and two treatment arms) and provided a dedicated call number. The control group students received no message. Treatment 1 - The students were informed about the returns to education. Treatment 2 - The parents were informed about the returns to education. The lesson's content was carefully selected to be suitable for delivery over IVR.
External Link(s)

Registration Citation

Citation
Angrist, Noam et al. 2022. "Foregone and Forgotten Learning: COVID-related Learning Losses from National Assessments in Bangladesh and experimental evaluation of a learning takeup program." AEA RCT Registry. December 26. https://doi.org/10.1257/rct.10184-1.1
Experimental Details

Interventions

Intervention(s)
The study is a large-scale learning assessment in Bangladesh comprising 7,218 grade 9 students to determine the current learning levels of students after 14.5 academic months of school closures. This was based on the National Assessment of Secondary Students (NASS) 2019 framework and covered a nationally representative sample of 300 schools. This study tracks 4997 students who took NASS 2019 in grade 7 and administered psychometrically linked math and language tests (Bangla). The question booklet included LASI 2017 and NASS 2019 Grade 8 questions (approximately 20 questions) and NASS 2019 grade 6 questions (approximately 20 questions). This is to estimate forgotten and forgone learning separately. Learning inequality estimates were designed by benchmarking learning losses for those with low socio-economic status and by gender using a short survey at the end of the assessments.

The study also comprised a phone-based Interactive Voice Response (IVR) lesson to assess the demand for additional instruction. The students were randomly assigned to three groups (one control and two treatment) and provided a dedicated number to call. The control group students received no message. Treatment 1 - The students were informed about the returns to education. Treatment 2 - The parents were informed about the returns to education. The lesson's content was carefully selected to be suitable for delivery over IVR.
Intervention Start Date
2022-05-14
Intervention End Date
2022-09-30

Primary Outcomes

Primary Outcomes (end points)
1) The learning takeup of treatment
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
The learning takeup disaggregated by gender, baseline learning levels, and family economic background.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The novel data collection was from 300 schools in Bangladesh (sampled randomly from 1000 schools that were part of NASS 2019). Learning level data was collected from 7218 grade 9 students (4997 students had taken the NASS 2019, and 2221 new students were selected randomly).
Further, after the exam, the students were considered for a phone-based Interactive Voice Response (IVR) lesson to assess the demand for additional instruction. The students were randomly assigned to three groups (one control and two treatment) and provided a dedicated number to call. The control group students received no message. Treatment 1 - The students were informed about the returns to education. Treatment 2 - The parents were informed about the returns to education. The lesson's content was carefully selected to be suitable for delivery over IVR.
Experimental Design Details
Randomization Method
Stata Randomization
Randomization Unit
School
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
300 schools
Sample size: planned number of observations
7218 students
Sample size (or number of clusters) by treatment arms
300 schools
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Institute of Health Economics (University of Dhaka)
IRB Approval Date
2022-04-17
IRB Approval Number
FWA00026031

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials