Identifying and Nurturing Math Talent at Scale: An Experiment with Art of Problem Solving Self-Paced Classes in Tamil Nadu, India

In our main treatment, students will receive one of four interventions.
a) Access to AoPS Pre-Algebra 1 course for six months, with various technological support, including WhatsApp groups and tablet + SIM access (n = 800)
b) Identical access as in (a), and weekly group tutoring sessions (n = 600)
c) Identical access as in (b), and weekly 1:1 tutoring sessions (n = 200)
d) Control group used as a baseline for comparison. (n = 800)

Treated students were also cross-randomized with two additional treatments.

1. 400 students in treatment groups were randomly assigned to receive support through school-provided computer labs. Working with school personnel, students were allowed after-school access to their school computer lab to work on AoPS.
2. Students were allocated to one of 3 "incentive" treatments
a) 533 students received no incentives.
b) 533 students were told that if they did not have adequate progress in the course, their tablet access would be revoked (though could be “earned back” if they kept up progress in the course). These incentives were enforced if a student had less than 80% of expected progess.
c) 533 students received the same information as group (b), but incentives were unenforced.

All treatments are randomized at the school level.

Students were selected for the program using one of several criterion, randomized at the school level. Selection will be based on four criteria:
a) (240 students) Performance on existing in-school tests
b) (240 students) Engagement with the internet-linked content and uploaded problems
c) (240 students) Performance on our virtual test
d) (240 students) Measures of student grit and persistence
The remaining 1440 students will be selected using combinations of the above criteria.
e) (240 students) an equal-weighted combination of (a) and (b)
f) (240 students) an equal-weighted combination of (b) and (c)
g) (480 students) an equal-weighted combination of (a) and (c)
h) (480 students) an equal-weighted combination of (a), (b), and (c)

2024-10-01

2025-07-01

1. Retention, grades, and engagement in AoPS class (only for the treatment groups)
2. Test scores on regular TN examinations (administrative data)
3. Test scores on a customized test created to measure skills in the standard math curriculum, the AoPS curriculum, and general mathematical reasoning, with a higher ceiling than standardized tests, administered at the end of the RCT.
4. Performance at the grade 10 board examinations (a high-stakes exam administered to all children in Tamil Nadu).
5. Admission into Tamil Nadu elite high schools during grades 9-12, and performance on any relevant entrance exams. For this to be a reliable outcome, we may encourage our entire sample to apply and take the entrance exams.
6. Performance on any state-wide or nation-wide exams, such as the NMMS.
7. Later-life outcomes, such as high school graduation, college matriculation, college selectiveness, and choice of a STEM major/career.

1. Qualitative evidence on students’ interest in STEM, belief in math abilities, attitude to towards math, plans, comfort and use of technology, and other behavioral outcomes, collected at the end of 10th and 12th grade.
2. Parental beliefs about student abilities, confidence, and aspirations for their children.
3. Impacts on students' non-mathematics achievement, including Science, English, and Tamil.

Students will be selected for this intervention using a combination of test scores on government-administered public school examinations and a screening mechanism in which high-performing students will receive access to a sample of AoPS material.

After identifying students, they will be randomized to one of the four primary interventions, and potentially cross-randomized to additional interventions, as described above.

Not available

Randomization performed in Stata with a saved seed.

Clustered at school level.

Yes

1100 schools

2400 students

Main Treatment: 800 students in control, 800 with AoPS only, 600 with group tutoring, 200 with 1:1 tutoring.

For subtreatments:
Incentive: 533 students with no incentives, 533 students with enforced incentives, 533 students with unenforced incentives. (All belong to treatment groups in main design.)
Comptuer Labs: 1200 untreated students, 400 treated students. (All belong to treatment groups in main design.)

0.3 standard deviations on outcomes with mean 0 and standard deviation 1