Measuring the impact of “hybrid” digital learning on children in rural Indian communities.

Pratham, the implementation partner, has been developing a ’Hybrid’ program targeted at children in grades 5-8 where high-quality digital content relating to Math, English, and Science is made available on easily accessible devices and provided to groups of children in communities and supported by appropriate social support structures. This ’Hybrid’ model attempts to address the shortcomings of typical digital content, the traditional focus of learning in Indian schools, and lack of community engagement and ownership of learning.

2021-10-17

2023-03-31

Test scores in Maths, English, and Reading.

Outcome measures of interest are learning gains in foundational skills such as functional English and Maths (everyday math as well as school-like math) and vernacular reading. Well-tested and piloted assessments developed by ASER/Pratham and J-PAL (for other research projects) will be primarily used to measure these outcomes, with some adaption of appropriate standard international tools if required.

Confidence, Motivation to learn, and Collaboration.

Additionally, if feasible, there will also be an attempt to measure confidence, collaboration, and motivation to learn. We will also examine the presence of differential impact across gender, age, and initial ability.

The evaluation will test the combined effectiveness of the “social infrastructure”, group learning methodology and the “digital trigger” content in improving learning outcomes in English and Math. A further ‘bonus’ (if found to be feasible) may be testing of social-emotional skill development (confidence, creativity, motivation, etc.).

In consultation with our implementation partner Pratham, the research team has identified villages across the Dausa district in Rajasthan where they would like to evaluate the program being implemented through a Randomized Control Trial. This will be a two-year evaluation across 276 villages, with testing at baseline, after Year 1 and after Year 2 and some light-touch intervention monitoring. The 276 villages will be randomly assigned to:
- Control group (92 villages): No intervention;
- Treatment group 1 (92 villages): Full Hybrid model with remote support as detailed above (groups of children, digital devices and content, societal infrastructure comprised of coaches and assessment committees). Once the COVID-19 situation eases, the implementation to be switched to fully in-person form with regular visits by CRLs to the village and active village assessment committees.
- Treatment group 2 (92 villages): Hybrid model with the children’s groups but without societal infrastructure such as no community volunteers, assessment committees etc. The impact of the program will be measured by the difference in learning outcomes between the treatment and control groups. We plan to undertake baseline, midline and end-line assessments.

Randomization done in office by a computer (Stata).

Village

Yes

276

5,520 children (20 per village).

92 villages control, 92 villages T1, 92 villages T2.

Our sample size of two treatment groups of 92 villages and one control of 92 gives us a power of 80 percent to detect a difference in 0.21 standard deviation in test scores between any specific treatment.

Pre-Analysis Plan Hybrid Digital Learning Program