Understanding the adoption and diffusion of educational technology: Evidence from Peru

This project explores the adoption and diffusion of an online math curriculum (Connecta Ideas) designed to support math learning among 4th to 6th grade students. In collaboration with the Ministry of Education in Lima, 188 primary schools from the second lowest quintile of academic performance in Lima, Peru, are randomly assigned to four treatment arms:
T1: An intensive teacher intervention that pairs some teachers with coordinators to assist with take-up and use,
T2: A low-cost teacher intervention that provides some teachers with virtual workshops to encourage take-up and use,
T3: A school-wide intervention providing virtual workshops to all teachers,
C: A control group that receives no treatment.

2021-08-12

2021-12-17

We will focus on one main outcome for our measure of adoption/diffusion of the CI platform:
1) Teacher created at least one student account

We will also focus on one main outcome for a measure of potential learning associated with the CI platform:
2) Number of CI exercises completed per student

Both of the primary outcomes are derived from administrative data collected from CI platform.
1) Teacher created at least one student account - this is a measure of CI adoption on the extensive margin that is solely under the teacher's control.
2) Number of CI exercises completed per student - this is a measure of student engagement with the CI platform that is likely to be correlated with learning.

While we have focused on two primary outcomes, we view them as part of a sequential process of CI adoption and diffusion, reflected in the following outcomes:
- Teacher registered to CI
- Teacher created at least one CI student account
- Number of CI student accounts created by teacher
- Share of student ever connected to CI
- Number of CI exercises completed by students
- Number of CI exercises per student connected

In addition, we are collecting a number of additional outcomes that capture use of educational technology, time spent on different instructional activities, and communication with families.

The experimental sample for this study includes 188 public primary schools in Lima, administered by the regional education authority, known as the Dirección Regional de Educación de Lima Metropolitana (DRELM). These are schools in Metropolitan Lima from the second lowest quintile of the average national test scores from 2016 to 2019.

The randomization consists of two stages. In the first stage, schools are assigned to the different treatment arms as follows:
- 47 schools in the control group
- 47 schools in the intensive (T1) teacher intervention
- 47 schools in the low-cost (T2) teacher intervention
- 47 schools in the low-cost (T3) school-wide intervention
The randomization is stratified according to average test scores from 2016 to 2019.

In the second stage, one teacher from grade 4 and one teacher from grade 6 are randomly assigned to treatment within T1 and T2 schools. The randomization of teachers to treatment within schools serves several purposes. Most important, this will enable us to explore the diffusion of adoption within grades 4 and 6, and across to grade 5. If the diffusion within schools is limited, it will also increase our ability to discern impacts on learning outcomes by estimating effects from within-school comparisons. If the diffusion within schools is extensive, we will still be able to estimate the impacts from across-school comparisons. In addition, the randomization within schools reduces the cost and facilitates the implementation of the intensive (T1) teacher intervention.

Randomization is done in office by a computer.

The first stage of randomization will be at the school level (188 units randomized into four groups of 47)
In two of the treatment arms (T1 and T2), treatment will be randomized across teachers within school/grade.

Yes

First stage of randomization: 188 schools (into four groups of 47)
Second stage of randomization: 2 treated teachers per school (out of 47*2=94 schools)

Number of schools: 188 Number of teachers: approx. 1800

For the first stage of randomization:
- 47 schools in the intensive (T1) teacher intervention
- 47 schools in the low-cost (T2) teacher intervention
- 47 schools in the low-cost (T3) school-wide intervention
- 47 schools in the control group

For the second stage of randomization:
- 2 treated teachers
- 2 untreated teachers (but varies by school)