Evaluation of Conecta Ideas in Chile

Last registered on June 06, 2022

Pre-Trial

Trial Information

General Information

Title
Evaluation of Conecta Ideas in Chile
RCT ID
AEARCTR-0009545
Initial registration date
June 03, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 06, 2022, 5:59 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
J-PAL

Other Primary Investigator(s)

PI Affiliation
ITAM
PI Affiliation
University of California, Berkeley

Additional Trial Information

Status
On going
Start date
2022-04-28
End date
2022-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We propose a large-scale evaluation of a learning platform designed in 2002 by AutoMind, a multidisciplinary team where the leadership is now at the University of Chile. The evaluation design will allow us to test the effect of Conecta Ideas vis-a-vis a control group. Moreover, the experimental design will feature three treatment groups (in addition to the control group). First, a group where students will earn purely symbolic incentives at the individual level. These incentives will be given for using the platform and for getting correct answers to exercises. Second, a group where these symbolic incentives translate to tangible prizes (e.g., books) for students. Third, a group where prizes are given as a function of the performance of the classroom. In addition to testing the effect of Conecta Ideas, this experimental design will allow us to test: a) individual vs group incentives, and b) tangible, but non-monetary incentives vs. purely symbolic incentives.
External Link(s)

Registration Citation

Citation
Gertler, Paul, Cristian Munoz and Mauricio Romero. 2022. "Evaluation of Conecta Ideas in Chile." AEA RCT Registry. June 06. https://doi.org/10.1257/rct.9545
Experimental Details

Interventions

Intervention(s)
Conecta Ideas (CI) sessions will be carried out in the computer labs of the schools. In a typical CI session, students work to solve the same set of 20 to 30 exercises assigned to them, which are aligned to the topics covered in regular math instruction that week. Students receive automatic feedback regarding whether their answers are correct. Lab coordinators, hired and supervised by the Conecta Ideas team, are
responsible for conducting the learning sessions at the computer lab in collaboration with regular classroom teachers. Lab coordinators will receive a day of training and ongoing supervision from the implementation team. While teachers will not receive formal training, the program promotes learning-bydoing. The software provides virtual stickers and prizes to students who do well in each CI session. It also keeps track of student’s progress (in the current session and across sessions) and offers each student personalized reports in the form of graphs to motive study effort.
Intervention Start Date
2022-06-06
Intervention End Date
2022-11-30

Primary Outcomes

Primary Outcomes (end points)
Mathematics' test scores of student
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The evaluation design will allow us to test the effect of Conecta Ideas vis-a-vis a control group, and at the same time, it will allow us to measure the effect of varying the implementation details of Conecta Ideas (in particular, by testing different incentive designs). We are able to study multiple questions by relying on standardized tests taken by student.
Experimental Design Details
Not available
Randomization Method
We conducted the randomization in Stata in a two-stage process. First, we randomized at the school level, stratifying by whether schools have one or more sections in Grade 4 and the quartile of an index that summarized the average IRT score and the standard deviation of scores within a school. The probability of being assigned to control was 0.4, while the probability of being assigned to each of the three treatments (plain vanilla Conecta Ideas, CI with individual incentives, and CI with group incentives) was 0.2. After schools were assigned to one of the four experimental groups, from each treatment school, we randomly selected a single section (in case there were multiple sections in Grade 4) for treatment.
Randomization Unit
We randomized at the school level, stratifying by whether schools have one or more sections in Grade 4 and the quartile of an index that summarized the average IRT score and the standard deviation of scores within a school
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
235 Schools
Sample size: planned number of observations
10000 students
Sample size (or number of clusters) by treatment arms
We have three treatment groups: plain vanilla Conecta Ideas (CI), Conecta Ideas with individual incentives (II), and Conecta Ideas with group incentives (GI). We assigned 50 schools to each of these groups, and we have 85 schools left that serve as controls.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Based on the baseline test, we assume that ρ =.145. We assume (conservatively) that the baseline covariates (e.g., the baseline test score, age, and gender) explain 30% the variance, and thus that σ = 0.84 ≈√1 − 0.3. Finally, taking into account attrition at the student level, we assume n =24. Therefore, we can detect an effect size of at least (MDE) 0.21σ with power 90% (β) using a 5% test of level (α) when comparing any of the treatments to the control. When comparing any two treatment groups, we have an MDE of 0.23σ
IRB

Institutional Review Boards (IRBs)

IRB Name
Instituto Tecnologico Autonomo de Mexico
IRB Approval Date
2022-04-28
IRB Approval Number
N/A