Abstract
This study asks whether machine learning can improve the targeting and personalization of low-cost digital "nudges" — push-notification messages sent to parents to encourage their children to use an educational app. The setting is Conecta Ideas, a free smartphone-based mathematics platform used by primary-school students across Peru. Sustaining voluntary use of such platforms is a central challenge, and short motivational messages are a cheap, scalable lever. We study two distinct decisions a platform faces: (1) personalization — WHICH message to send to each person, choosing among several alternative messages the one predicted to work best for that individual; and (2) targeting — WHETHER to message a given person at all, given a single common message, concentrating effort on those predicted to respond most.
The study works with a population of roughly 100,000 parents who had used the app at least once in the prior year. Each parent receives push notifications drawn from four behavioral message types — a teacher recommendation (social norms), peer/usage norms, the parent's role in their child's learning (identity), and the future opportunities that learning math creates (present bias). Treated parents receive two notifications per week (Mondays and Thursdays) over three weeks; the control group receives none. The primary outcome is whether the student logs into the platform in the weeks following the messaging; secondary outcomes are time spent on the platform and the number of exercises completed.
The design has two phases. In Phase 1, parents are randomly assigned (at the individual level, stratified by whether the student logged in the previous week) across the candidate messages and a no-message control. Using the Phase 1 outcome data, we estimate both the average effect of each message and how those effects vary across individuals, using two machine-learning estimators — a causal forest and a k-nearest-neighbor matcher — fed a common set of pre-treatment characteristics (grade, location, prior platform use, baseline achievement, and school characteristics). In Phase 2, the same population is re-randomized into four arms of roughly equal size: a uniform "best" arm (everyone receives the single highest-average-effect message from Phase 1); two "personalized" arms (each person receives the message that the causal forest, or the nearest-neighbor model, predicts is best for them); and a "random" arm that serves as a no-personalization benchmark.
Phase 2 is therefore an out-of-sample experimental test of the personalized and targeted assignment rules learned in Phase 1: the rules are fixed using one experiment's data and then evaluated on a fresh draw of the same population. The study is run twice in contrasting engagement environments. The first implementation takes place during the school year (October–December 2023), when baseline weekly login rates are high. A second implementation (added as an addendum; see below) replicates the same two-phase design during the summer vacation (January–February 2024), when schools are out of session and baseline engagement is far lower, so that the value of personalization and targeting can be compared across a high-engagement and a low-engagement context.
Addendum disclosure: the summer implementation was not pre-registered in advance. It was designed and fielded after the school-year Phase 1 results were known and is documented here for completeness and transparency.