Ground-truthing best education practices and evidence for COVID-19 learning recovery. Does presenting scientific evidence alter teachers’ beliefs and behaviors vis-a-vis global guidance?

Last registered on October 17, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
Ground-truthing best education practices and evidence for COVID-19 learning recovery. Does presenting scientific evidence alter teachers’ beliefs and behaviors vis-a-vis global guidance?
RCT ID
AEARCTR-0010191
Initial registration date
October 11, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 17, 2022, 5:19 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Bangladesh
Region
Country
India
Region
Country
Nepal
Region

Primary Investigator

Name
Shrihari Ramachandra
Affiliation
The World Bank
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Noam Angrist
PI Affiliation
University of Oxford
Contact Investigator
PI Name
Andres Yi Chang
PI Affiliation
The World Bank
Contact Investigator
PI Name
Sharnic Djaker
PI Affiliation
New York University
Contact Investigator
PI Name
Shwetlena Sabarwal
PI Affiliation
The World Bank
Contact Investigator

Additional Trial Information

Status
On going
Start date
2022-05-20
End date
2022-12-31
Keywords
Education
Additional Keywords
Learning Loss, Learning recovery, Remedial Classes, Assessments, COVID-19, School closures
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
COVID-19 shuttered schools for over a billion children, resulting in substantial learning losses. Institutions such as the World Bank, UNESCO, and UNICEF have issued considerable policy guidance on evidence-based best practices to recover learning losses. These include extending instructional time, conducting remedial education classes, and regular learning assessments. How realistic is it to implement these best practices? This study provides some of the first empirical evidence of teachers’ awareness, willingness, and implementation of learning loss recovery programs. We collect ground truth estimates by surveying teachers across three countries: Bangladesh, India, and Nepal. Further, we evaluate the impacts of delivering scientific evidence on the effectiveness of the global guidance on learning loss recovery using a randomized control trial by randomly assigning the teachers into four groups. The control group is only informed about the global policy in focus and is not given any evidence about its effectiveness. The three treatment groups are informed that the policy in focus is being recommended based on high-quality experimental evidence on the effectiveness from a similar or different context (Treatment 1 - Global Evidence and solution, Treatment 2 - Local Evidence, and Treatment 3 - Local evidence and solution).
External Link(s)

Registration Citation

Citation
Angrist, Noam et al. 2022. "Ground-truthing best education practices and evidence for COVID-19 learning recovery. Does presenting scientific evidence alter teachers’ beliefs and behaviors vis-a-vis global guidance?." AEA RCT Registry. October 17. https://doi.org/10.1257/rct.10191-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We collect data through standardized phone surveys in three countries - India, Nepal, and Bangladesh surveying 928 teachers (primary and secondary). The experiment elicits teachers' willingness to contribute (in minutes) to conducting extra learning recovery activities in addition to regular schooling for (a) developing and administering tests and (b) conducting remedial classes. The telephonic teacher survey comprised questions on teacher characteristics and focused on three policy modules – 1) Regular class assessments, 2) Remedial classes, and 3) Teacher school attendance in case of COVID cases, in addition to questions on non-cognitive traits and salary information. Further, the teachers are randomly assigned to either of the four groups: - 1 control and three treatment groups. In the control group, teachers are informed about the global policy in focus but are not given any evidence about its effectiveness. In the three treatment groups, T1 - The teachers are informed about the global policy and provide global evidence about the effectiveness. T2 - The teachers are informed about the policy in the local context. T3 - The teachers are informed about the local policy and also given local evidence about the effectiveness. The study aims to evaluate the impacts of delivering scientific evidence of the efficacy of global guidance on learning loss recovery.
Intervention Start Date
2022-12-30
Intervention End Date
2022-12-31

Primary Outcomes

Primary Outcomes (end points)
1) Currently conducting remedial classes (baseline) 2) Currently conducting regular formative assessments
3) Willingness to conduct extra classes. 4) Willingness to conduct regular formative assessments.
5) Willingness to spend additional time in school (in minutes) to conduct a) Extra classes b) regular formative assessments
6) Willingness to spend additional time in school (in minutes) to conduct a) Extra classes b) regular formative assessments after the treatment.
Primary Outcomes (explanation)
The Regular class assessments and Remedial classes are disaggregated into 1) Baseline (currently doing), Belief (Normative), and Behavioral intention (Willingness to spend extra time). For each activity (conducting extra classes and regular formative assessments), teachers are asked about their willingness to spend additional time doing it [in declining order from 120 minutes to 0, in 10 min increments].

Secondary Outcomes

Secondary Outcomes (end points)
1) Willingness to attend during low COVID cases 2) Willingness to attend during high COVID cases
3) Number of days willing to attend in case of low-COVID cases ( for 30 working days) 4) Number of days willing to attend in case of high-COVID cases (for 30 working days)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experiment is a telephonic survey comprising 928 teachers (primary and secondary school) in 3 countries - India, Bangladesh, and Nepal. The questions are on teacher characteristics and focused on three policy modules –
1) Regular class assessments
2) Remedial classes, and
3) Teacher school attendance in case of COVID cases.
Further, the teachers are randomly assigned to four groups
1) Control - Teachers are only provided with global policy information.
2) Treatment 1 - Teachers are provided with global policy information and a global evidence-based solution
3) Treatment 2 - Teachers are provided with local policy information
4) Treatment 3 - Teachers are provided with local policy information and a local evidence-based solution
Experimental Design Details
In addition, there are also questions on non-cognitive traits and salary information.
The Regular class assessments and Remedial classes modules were further disaggregated into three dimensions.
1) Baseline (currently doing) - (Example) Did you conduct any class tests (other than regular exams) to understand students’ learning progress? If “Yes,” how many in a month?
2) Belief (Normative) - (Example) Should teachers be required to conduct extra class tests to diagnose current learning levels and keep track of students’ progress? If “Yes,” how many in a month?
3) Behavioral intention (Willingness to spend extra time) - (Example) Are you willing to spend any extra time per day to develop, administer, and grade exams? If “Yes,” how much time (in minutes)?
Randomization Method
Stata randomization
Randomization Unit
Individual
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
928 teachers. 3 countries
Sample size: planned number of observations
928 teachers
Sample size (or number of clusters) by treatment arms
928 teachers. 3 countries
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
HML Institutional Review Board
IRB Approval Date
2022-03-25
IRB Approval Number
1118TWBG22
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials