Measuring the impact of “hybrid” digital learning on children in rural Indian communities.

Last registered on August 28, 2023

Pre-Trial

Trial Information

General Information

Title
Measuring the impact of “hybrid” digital learning on children in rural Indian communities.
RCT ID
AEARCTR-0008330
Initial registration date
October 04, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 05, 2021, 2:16 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
August 28, 2023, 9:28 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
J-PAL Europe

Other Primary Investigator(s)

PI Affiliation
MIT
PI Affiliation
MIT

Additional Trial Information

Status
On going
Start date
2021-01-01
End date
2023-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project proposes a randomized evaluation of a “hybrid” digital learning program targeted at children in Grades 5-8 in Indian villages. With the COVID-19 pandemic widening the pre-existing learning gap, we hypothesize that programs with appropriate high-quality digital content made available on easily accessible devices and supported by appropriate social support structures can improve learning outcomes and related skills when implemented directly in communities with groups of children. Therefore, this program provides a hopeful solution to making remote learning more accessible and motivating both during school closures and when schools are open. The evaluation will primarily aim to measure the impact of the program on Math and English learning outcomes. The non-profit Pratham has developed and piloted the key elements of this intervention and will implement the program to be evaluated in 276 villages in India.
External Link(s)

Registration Citation

Citation
Banerjee, Abhijit, Esther Duflo and Adrien Pawlik. 2023. "Measuring the impact of “hybrid” digital learning on children in rural Indian communities.." AEA RCT Registry. August 28. https://doi.org/10.1257/rct.8330-1.1
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Pratham, the implementation partner, has been developing a ’Hybrid’ program targeted at children in grades 5-8 where high-quality digital content relating to Math, English, and Science is made available on easily accessible devices and provided to groups of children in communities and supported by appropriate social support structures. This ’Hybrid’ model attempts to address the shortcomings of typical digital content, the traditional focus of learning in Indian schools, and lack of community engagement and ownership of learning.
Intervention (Hidden)
The digital content has been developed, curated, and piloted by a central team at Pratham while their field teams are involved in setting up and implementing the program. Owing to COVID-19 some part of the implementation will happen through online and virtual meetings until the situation eases.

1. Pratham field staff has conducted activities to engage the target children and created groups of children previously. These groups will act as a unit and will complete the activities that are prescribed. These groups are formed based on the proximity of households. End-of-course (to be undertaken independently) allows children to understand their progress.

2. Once the groups are formed, Pratham field staff will identify volunteers from the community called 'coaches' of children who will serve as 'custodians' of the digital devices (here tablets). The custodians will be in possession of the tablets and only hand them over to children when they meet to complete the content. The children shall access the content either through the tablets or throughs apps on their parent’s or coach’s mobile devices.

3. The children are expected to meet a few times a week and work together on reviewing the content.

4. The volunteers or 'coaches' will be trained on how to facilitate the engagement of the groups and on implementing the course in the village. The role of the coach would be to introduce the course, register groups for courses, motivate groups to complete course and assessment. However, due to COVID-19, Pratham's Cluster Resource Leaders (CRLs) will provide mentoring support to the coaches via virtual meetings and/or phone calls. As coaches belong to the same village, movement within the village is possible.

5. During the course of the program, Pratham staff had planned to conduct periodic visits to villages and monitor the implementation. They had also planned to provide mentoring support to the coaches and along with them conduct periodic events in the community that will include assessments and showcasing of project work. Pratham staff will also encourage other adults to engage in the program by being a part of a committee that assesses the learning of children. However, during the lockdown period most of the activity will be online and the assessments will be conducted through assessment app by parents and coaches or the students.

6. Pratham staff will conduct regular check-ins with students during COVID-19 times via different modes of communication like SMS and Whatsapp etc.
Intervention Start Date
2021-10-17
Intervention End Date
2023-03-31

Primary Outcomes

Primary Outcomes (end points)
Test scores in Maths, English, and Reading.
Primary Outcomes (explanation)
Outcome measures of interest are learning gains in foundational skills such as functional English and Maths (everyday math as well as school-like math) and vernacular reading. Well-tested and piloted assessments developed by ASER/Pratham and J-PAL (for other research projects) will be primarily used to measure these outcomes, with some adaption of appropriate standard international tools if required.

Secondary Outcomes

Secondary Outcomes (end points)
Confidence, Motivation to learn, and Collaboration.
Secondary Outcomes (explanation)
Additionally, if feasible, there will also be an attempt to measure confidence, collaboration, and motivation to learn. We will also examine the presence of differential impact across gender, age, and initial ability.

Experimental Design

Experimental Design
The evaluation will test the combined effectiveness of the “social infrastructure”, group learning methodology and the “digital trigger” content in improving learning outcomes in English and Math. A further ‘bonus’ (if found to be feasible) may be testing of social-emotional skill development (confidence, creativity, motivation, etc.).
Experimental Design Details
In consultation with our implementation partner Pratham, the research team has identified villages across the Dausa district in Rajasthan where they would like to evaluate the program being implemented through a Randomized Control Trial. This will be a two-year evaluation across 276 villages, with testing at baseline, after Year 1 and after Year 2 and some light-touch intervention monitoring. The 276 villages will be randomly assigned to:
- Control group (92 villages): No intervention;
- Treatment group 1 (92 villages): Full Hybrid model with remote support as detailed above (groups of children, digital devices and content, societal infrastructure comprised of coaches and assessment committees). Once the COVID-19 situation eases, the implementation to be switched to fully in-person form with regular visits by CRLs to the village and active village assessment committees.
- Treatment group 2 (92 villages): Hybrid model with the children’s groups but without societal infrastructure such as no community volunteers, assessment committees etc. The impact of the program will be measured by the difference in learning outcomes between the treatment and control groups. We plan to undertake baseline, midline and end-line assessments.
Randomization Method
Randomization done in office by a computer (Stata).
Randomization Unit
Village
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
276
Sample size: planned number of observations
5,520 children (20 per village).
Sample size (or number of clusters) by treatment arms
92 villages control, 92 villages T1, 92 villages T2.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Our sample size of two treatment groups of 92 villages and one control of 92 gives us a power of 80 percent to detect a difference in 0.21 standard deviation in test scores between any specific treatment.
IRB

Institutional Review Boards (IRBs)

IRB Name
MIT Committee on the Use of Humans as Experimental Subjects (COUHES)
IRB Approval Date
2020-12-19
IRB Approval Number
N/A
IRB Name
IFMR Human Subjects Committee
IRB Approval Date
2020-01-17
IRB Approval Number
N/A
Analysis Plan

Analysis Plan Documents

Pre-Analysis Plan Hybrid Digital Learning Program

MD5:

SHA1:

Uploaded At: August 28, 2023

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials