Play to Learn - Improving Foundational Learning with Technology Aided Formative Teaching

Computed aided learning (CAL) has tremendous potential in bridging learning gaps in developing countries. Bulman and Fairlie (2016) and Escueta et al. (2020) provide comprehensive reviews of this literature. Most current studies focus on summative learning models where student progress is assessed at the end of CAL learning. Muralidharan et al. (2019) test an adaptive student differentiated learning echo system deployed at learning centers and find significant improvements for children in higher grades. In such models, instructors can track the student’s performance and make their pedagogical plans responsive to the progression. The efficacy of such a model for younger children has not been tested thus far.

Our study will leverage the deployment of a free educational app named Chimple, available in local vernacular language (Hindi in our case) geared towards foundational learning (literacy and numeracy) harnessing game-based approaches. Chimple can be installed on low-end smartphones and works offline with its library of 70 unique games - 50 for literacy and 20 for numeracy. The Literacy ladder takes the child from letter recognition to sentence formation and comprehension. The Numeracy ladder takes the child from number sense to basic multiplication. Chimple has a dashboard that tracks the child’s participation and progress. Chimple corresponds to the syllabi for the grade proposed by the government (K-2). These features make it very attractive to marginalized and vulnerable populations as well. The game-based approach circumvents the issue of short attention spans of young children.

XPRIZE and UNESCO have tested Chimple in a pilot evaluation in Tanzania. This program lasted 15 months (December 2017 -March 2019) with 500 children aged 7-11 across 30 villages in the Tanga region of Tanzania. The product was accessed via tablets. Learning outcomes were assessed via EGRA/EGMA test (RTI) and Social-Emotional assessment (UNESCO) as well. Large effects of over 0.5 standard deviations were observed for both literacy and numeracy.

Chimple in India: Implementation in India was initiated in a computer lab setting in 15 schools in Mumbai, Maharashtra, in partnership with the Brihan Mumbai Municipal Corporation, although this has been paused due to COVID. Around 500 children from low-income households accessed Chimple via Door Step School Mumbai community centers. But the efficacy of Chimple in a teacher-driven at-home environment has not been tested.

The intervention will be deployed in 34 schools (the Satya Bharti schools) located in 5 districts in the state of Haryana. Bharti Foundation will spearhead the deployment. Teachers in grades 1 and 2 will integrate Chimple in their online course plans as a supplemental tool. Each week specific topics will be assigned to the students. The teachers will be able to track students’ progress via the software dashboard. Due to COVID surges and inclement weather, schools are often closed. Whatsapp, a messaging tool, is being used as a classroom instructional tool aide primarily for content dissemination. The teachers will ask parents to install the Chimple app on their phones and enable children’s engagement at home. On average, a recently concluded survey indicates that the vast majority of households in Satya Bharti Schools had at least one WhatsApp-enabled phone.

2022-07-15

2023-04-30

educational software usage, math and language test scores

Educational software usage. We will employ the number of completed activities and time spent in the app by each student to measure software usage. This information will be collected from the Chimple dashboard.
Math and language tests will be similar to those conducted by Bharti Foundation and other organizations such as Pratham.

Our intervention to examine the impact of interactive app-based learning on children's test scores will be deployed in 34 primary schools across 5 districts in Haryana run by Bharti Foundation. Total enrollment across all schools is approximately 1000 students per grade. More than 80% of children enrolled are of disadvantaged castes, and 50% of households earn less than USD$1,300 per year. Approximately 90% of households have access to at least one smartphone that can run Chimple.

Teachers in grades 1 and 2 will integrate Chimple into their course plans as a supplemental tool. Each week teachers will assign homework via Chimple and track students’ progress via the dashboard. The intervention will leverage Whatsapp, a messaging tool, as a platform for teachers to ask parents to install Chimple on their phones. Schools will run a parent information session to highlight the potential value of Chimple for their children’s learning. We will also use these WhatsApp groups to nudge parents to support Chimple assignment completion at varying frequencies.

We will assess the impact of Chimple on children's test scores using a stratified randomized control trial, where a class is the unit of randomization. The sample includes 34 schools with 1 class each in grades 1 and 2, Chimple’s targeted age group. We will use baseline average test scores to group schools in 8 strata (7 groups of 4 schools and one group of six schools). In each stratum, 50% of schools will be randomly chosen to offer the Chimple program in grade 1 (e.g., Schools A and B); the other schools will offer the Chimple program in grade 2 (e.g., Schools C and D). Grade 1 in Schools A and B will serve as a comparison for Grade 1 in Schools C and D, and similarly, Grade 2 in Schools C and D will serve as a comparison for Grade 2 in Schools A and B.

Students in treated classrooms will be randomly assigned to a low and high-intensity WhatsApp group to nudge parents. This randomization will be stratified by classroom and parental access to smartphones (access or no access). In other words, we will randomize the WhatsApp group for each classroom-by-smartphone access cell.

We will use the Stata software for our randomization exercise. We will sort the schools by the average test score of students in grades 1 and 2, set the seed, and generate a random uniform distribution in the 0-1 interval within each stratum. Schools for which the random variable is below the stratum median, will participate in the Chimple intervention for first graders. The rest of the schools will participate in Chimple for second grade. By design, there will be 17 schools treated in first grade and 17 treated in second grade.
Regarding the randomization of Whatsapp groups, first, students within each classroom-by-smartphone access cell will be sorted by admission date. After setting a seed, a uniform random number will be generated within each cell, and those having values above the median will be assigned to a high-intensity group. The rest will be assigned to the low-intensity group.

We will employ a stratified randomized control trial, where a class is the unit of randomization for the Chimple intervention. The Whatsapp randomization will be at the student level.

Yes

Our sample will include 68 classrooms (34 first- and second-grade classrooms).

Our sample will include 68 classrooms, comprising approximately 2000 students in grades 1 and 2.

34 control classrooms, 34 treated classrooms, half of the students in treated schools high-intensity nudging, and the other half low-intensity nudging.

Using data from past learning assessments conducted by Bharti Foundation and a bootstrapping procedure to account for the within-school randomization, we estimate that the minimal detectable effect (MDE) in the classroom-level randomization ranges from 0.2 to 0.3 SD. That is, we have a reasonable chance (80% power) of statistically detecting a significant difference (5% significance) in test scores between children in treatment classes and children in comparison classes if the true effect of Chimple on test scores is 0.2 - 0.3 SD. An effect of 0.3 SD would be considered a large effect in the education literature; at the same time, the pilot evaluation of Chimple found impacts of approximately 0.5 SD. We used past learning assessments and dashboard data on weekly Chimple engagement (in minutes) to estimate the MDE in the student-level randomization of WhatsApp nudges. It is essential to mention that we used usage information from weeks before we started to nudge families. Different methodologies provide similar estimates. We obtained MDEs in the ranges of 0.14-0.15 SD and 0.19-0.2 SD for test scores and weekly engagement, respectively.