Optimizing tutoring programs for scale: Evidence from multiple iterative randomized A/B tests

Last registered on August 01, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
Optimizing tutoring programs for scale: Evidence from multiple iterative randomized A/B tests
RCT ID
AEARCTR-0016323
Initial registration date
July 02, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 01, 2025, 10:04 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Botswana
Region

Primary Investigator

Name
Claire Cullen
Affiliation
University of Oxford
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Noam Angrist
PI Affiliation
University of Oxford
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2020-10-01
End date
2025-01-25
Keywords
Education
Additional Keywords
JEL code(s)
I21, O15, C93, D61
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study uses a series of iterative randomized A/B tests to optimize the cost-effectiveness of a phone-based tutoring program in Botswana. Across multiple experimental rounds, we test modifications aimed at reducing implementation costs or enhancing learning outcomes, such as reducing scheduling frictions and increasing caregiver engagement. We test whether low-cost modifications can significantly improve program efficiency, offering scalable solutions to address the foundational learning crisis in low-resource settings.
External Link(s)

Registration Citation

Citation
Angrist, Noam and Claire Cullen. 2025. "Optimizing tutoring programs for scale: Evidence from multiple iterative randomized A/B tests." AEA RCT Registry. August 01. https://doi.org/10.1257/rct.16323-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention (Hidden)
Intervention Start Date
2020-11-01
Intervention End Date
2025-01-10

Primary Outcomes

Primary Outcomes (end points)
Student learning level (measuring using a validated phone-adapted ASER tool, measuring proficiency in addition (1), subtraction (2), multiplication (3) and division (4)).
Primary Outcomes (explanation)
The learning outcomes will be constructed as standard deviations, with mean zero in the status quo/ control group. Following the same outcome measurement used in Angrist et al., 2020 & Angrist et al., 2023.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This study consists of multiple sequential randomized A/B tests designed to test the impact of modifications of a phone-based tutoring program. Each round randomly assigns students to one of two variants of the program: either the status quo model or a modified version designed to improve cost-effectiveness.
Experimental Design Details
This study consists of a series of sequential A/B tests, each aimed at optimizing the design and delivery of a phone-based tutoring program. Each test compares two versions of the program:
Group A: The status quo model as it exists at the time of the test, which may already incorporate improvements from previous rounds.
Group B: A modified version of the program intended to improve cost-effectiveness, scalability, or learning outcomes.

After each round of testing, the results are analyzed, and the program is updated based on which version performed better. A new A/B test is then launched. This iterative design allows the program to adapt and improve over time. Importantly, while each test maintains a clear control (Group A) and treatment (Group B) arm, the definition of the "control" evolves as the program itself is refined.
Randomization Method
randomization done in office by a computer in stata
Randomization Unit
Household level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Each round includes approximately 1000 households. We will include multiple sequential tests in analysis.
Sample size: planned number of observations
Approximately 1000 households across 12 tests: 12000 households total
Sample size (or number of clusters) by treatment arms
500 households in A and 500 households in B, each test.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials