Using technology to deliver preschool services at scale: Experimental evidence from India

We evaluate two iterations of such a technology-aided program designed to improve child development in India’s public preschools. Developed by the nonprofit, Rocket Learning (RL), the intervention incorporates key insights from home-visitation programs into a scalable, digital format. It delivers structured cognitive-development activities to parents and preschool (anganwadi centers, or AWCs), via WhatsApp groups that include parents and workers at each center. Parents are encouraged to post photos or videos of completed activities on the WhatsApp group, and automated behavioral nudges (managed by a centralized technology back-end) were sent to encourage participation. The content and approach draw on successful home-visitation models emphasizing group interactions and feedback (e.g., Andrew et al. (2020)).

The intensive version of the program introduced two key modifications. First, building on evidence from school-age children that engaging teachers in tech-enabled instruction improved learning outcomes, while bypassing them can be counterproductive (Beg et al., 2022), there was increased support for anganwadi workers. Workers in the intensive group received more content, guidance, and peer support activities; participated in more frequent training sessions; and were encouraged to boost parental engagement through phone calls and parent-teacher meetings. Second, it increased the frequency and personalization of messages to parents, as well as nudges and encouragement to boost engagement on the WhatsApp groups.

2021-11-01

2023-11-01

The primary outcome of interest is students’ learning outcomes. In particular, mathematics, executive function, and the local language, Marathi.

We use Item Response Theory (IRT) to convert the raw scores on the oral tests into four latent ability measures: math, language, executive function, and overall cognitive ability.

Regular Rocket Learning Program
The study universe comprises 13 administrative blocks (referred to as projects) within the district of Amravati. Each block was subdivided into clusters of anganwadi centers, referred to as “beats”, as per the ICDS hierarchical structure. The 13 projects included 88 beats, but we excluded those with fewer than 3 anganwadi centers, leaving 85 in total. The randomization for the regular Rocket Learning intervention, which commenced in November 2021, was conducted at the beat level and stratified at the project level. A total of 45 beats were assigned to the treatment group, which received the regular Rocket Learning program, and 40 to the control group. All anganwadi centers in each beat were either treated or left as controls. We randomly sampled three anganwadi centers within each beat to be surveyed. As a result, our sample for the first year of the experiment consists of 135 treated anganwadi centers across 45 beats, and the control group consists of 120 anganwadi centers across 40 beats.

Intensive Rocket Learning Program

The randomization for the intensive intervention was conducted at the anganwadi level. Anganwadi centers in the intensive variant were all treated in the first experiment and thus received the regular variant of the program from November 2021 to April 2023, and the intensive variant afterward. The randomization protocol was as follows: First, among the 135 anganwadis treated in the first experiment and sampled for our survey, 90 were randomly assigned to receive the intensive program, while the remaining 45 continued to receive the regular program. Randomization was clustered at the beat level, with two AWCs per beat assigned to the intensive program and one AWC assigned to the regular program.

An additional 90 AWCs were randomly selected from the universe of AWCs within the 45 treatment beats (among those not surveyed in the first year of the experiment) to maintain power to study the effects of the regular program vis-`a-vis the control and the intensive arm. This sampling was stratified at the beat level, and two AWCs were selected per beat. These 90 AWCs received regular treatment during the two years of the experiment. The 120 AWCs across the 40 control beats remained in our sample and as controls (as well as all the other AWCs in those 40 beats).

In summary, for the second year of the experiment, our sample consists of: 1) 135 anganwadis in the regular treatment arm — 45 from the first year sample and a new sample of 90 AWCs, all of which were treated with the regular treatment for two years; 2) 90 anganwadis in the intensive arm, which were also in our first year sample — these were treated with the regular treatment from November 2021 to April 2023, and with the intensive treatment afterward; and 120 anganwadis in the control group, which were also in the first year sample, and were never treated in either year.

Done using STATA on a laptop.

Unit of randomization for regular Rocket Learning intervention: group of preschool centers (anganwadis) called beats
Unit of randomization for intensive Rocket Learning intervention: Preschool centers (anganwadis)

Yes

Clusters for regular Rocket Learning intervention: 13 projects in the district of Amravati

Clusters for intensive Rocket Learning intervention: 45 beats

Planned Number of Observations for regular Rocket Learning intervention: 3,569 households Planned Number of Observations for intensive Rocket Learning intervention: 8,410 households

Regular Rocket Learning intervention: 135 treated anganwadi centers across 45 beats, and the control group consists of 120 anganwadi centers across 40 beats.

Intensive Rocket Learning intervention: 1) 135 anganwadis in the regular treatment arm — 45 from the first year sample and a new sample of 90 AWCs, all of which were treated with the regular treatment for two years; 2) 90 anganwadis in the intensive arm, which were also in our first year sample — these were treated with the regular treatment from November 2021 to April 2023, and with the intensive treatment afterward; and 120 anganwadis in the control group, which were also in the first year sample, and were never treated in either year.