It Takes a Village (Plus AI): A Randomized Controlled Trial of Hybrid Human-AI Math Tutoring at Scale

Last registered on September 09, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
It Takes a Village (Plus AI): A Randomized Controlled Trial of Hybrid Human-AI Math Tutoring at Scale
RCT ID
AEARCTR-0019556
Initial registration date
August 31, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 09, 2026, 4:46 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Danielle Thomas
Affiliation
Carnegie Mellon University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Kalena Cortes
PI Affiliation
Texas A & M University
Contact Investigator
PI Name
Elizabeth Huffaker
PI Affiliation
University of Florida
Contact Investigator
PI Name
Kenneth R. Koedinger
PI Affiliation
Carnegie Mellon University
Contact Investigator
PI Name
Shivang Gupta
PI Affiliation
Carnegie Mellon University
Contact Investigator
PI Name
Conrad Borchers
PI Affiliation
Vanderbilt University
Contact Investigator

Additional Trial Information

Status
In development
Start date
2026-08-31
End date
2027-10-30
Keywords
Education
Additional Keywords
High-impact tutoring, K-12 math, middle school students, field experiments, student persistence, AI in education, learning analytics, teacher, teaching, computer-assisted learning, human-AI tutoring
JEL code(s)
C93, I21, I24
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project is a large-scale randomized controlled trial (RCT) to evaluate Personalized Learning Squared (PLUS), a hybrid human-AI tutoring program aimed at accelerating math learning among middle school students, particularly those from underserved communities. Tutoring boosts achievement, especially for students furthest behind, but it is costly and difficult to scale. Adaptive math software can deliver personalized practice at scale but often yields uneven engagement and results. PLUS integrates the strengths of both approaches by pairing remote human tutors with an AI-powered dashboard that synthesizes real-time and historical data from widely used platforms (e.g., iReady, IXL, MATHia, MobyMax). The dashboard helps tutors set, share, and track students' weekly math software practice, goals and target students most in need. The PLUS app housing the dashboard also provides AI-driven professional training to tutors to improve their tutoring practice. Finally, teachers are supported through regular e-mail reports on their students' engagement in PLUS tutoring and goal achievement (as set by PLUS tutors and their tutees in the PLUS app), which allows them to reward goal achievement. Eligible math or intervention course sections will be randomly assigned 1:1 within school × grade strata (or school x grade x teacher strata where teachers have more than one section of an eligible class within a grade) to this treatment or to business-as-usual (BaU) control, defined as either Tier I math instruction or a Tier II intervention class. The confirmatory research question asks whether PLUS improves student achievement on state math assessments compared with BaU instruction and support. Exploratory analyses will examine impacts on formative assessments, dosage, effort (e.g., time on task), and software engagement and progress. The study includes 20–25 schools across multiple states including California, Pennsylvania, Florida, and Oregon, involving approximately 210 classrooms and 4500 students. Findings will provide rigorous causal evidence on the scalability of hybrid human-AI tutoring and inform cost-effective strategies to reduce opportunity gaps in math learning.
External Link(s)

Registration Citation

Citation
Borchers, Conrad et al. 2026. "It Takes a Village (Plus AI): A Randomized Controlled Trial of Hybrid Human-AI Math Tutoring at Scale ." AEA RCT Registry. September 09. https://doi.org/10.1257/rct.19556-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Personalized Learning Squared (PLUS) is a hybrid human–AI tutoring program in middle school mathematics. Students in treatment classrooms receive remote tutoring from PLUS tutors during the school day, planned at typically two sessions per week of approximately 40 minutes each, with session length varying by school schedule, across the 2026–27 school year. Tutors work through the PLUS app, whose dashboard synthesizes real-time and historical learning data from the math software platforms in use at each school (e.g., iReady, IXL, MATHia, MobyMax). The dashboard helps tutors set, share, and track student weekly math software practice goals and target students most in need based on their recent effort and progress in each math software. The app also provides AI-driven professional training to tutors to improve their tutoring practice. Teachers are supported through weekly e-mail reports on their students' engagement in PLUS tutoring and goal achievement (as set by PLUS tutors and their tutees in the PLUS app), which allows them to reward goal achievement. The intervention aims to accelerate math learning among middle school students, particularly those from underserved communities. Control classrooms continue either business-as-usual Tier I instruction or their Tier II intervention. Students in control classes have, at minimum, access to math software, in general, though it may not be assigned during the BaU control classrooms.
Intervention Start Date
2026-09-14
Intervention End Date
2027-04-30

Primary Outcomes

Primary Outcomes (end points)
End-of-year state mathematics assessment scores (spring 2027 administration), standardized: CAASPP (California), PSSA (Pennsylvania), FAST (Florida), OSAS (Oregon).
Primary Outcomes (explanation)
Scores are standardized to z-scores (mean 0, SD 1) within state × grade and pooled across states for the confirmatory analysis. Standardization uses statewide means and standard deviations where published, and the study sample distribution otherwise.

Secondary Outcomes

Secondary Outcomes (end points)
Formative assessment results (e.g., NWEA MAP, educational technology vendor diagnostic tests), grades, in-software learning measures, and school attendance.
Secondary Outcomes (explanation)
-Formative assessment results are mathematics scores from district-administered assessments (e.g., NWEA MAP RIT scores, vendor diagnostic scale scores such as iReady Diagnostic), standardized within assessment × grade × administration window. Instruments vary by district and are analyzed where administered in both conditions.

-Grades include end-of-course mathematics grades from district records. Exploratory spillover analysis may also consider non-mathematics grades where available.

-In-software learning measures: engagement and progress in the math software platforms available in both conditions (e.g., time on task, problems or lessons completed, skills mastered, skill accuracy), from vendor records. Exploratory spillover analysis may also consider non-mathematics software data where available.

-Tutoring dosage in terms of sessions and minutes as captured in Zoom video conferencing logs as well as tutoring attendance as logged by PLUS tutors in the PLUS app.

-School attendance: attendance rate (days attended over days enrolled) from district records.

Experimental Design

Experimental Design
The study is a two-arm, stratified classroom-randomized controlled trial. Eligible mathematics course sections in participating schools are assigned 1:1 to treatment (PLUS hybrid human–AI tutoring) or control (business-as-usual instruction — Tier I or the school's existing Tier II intervention — with access to math software at minimum). Random assignment is stratified primarily by school × grade. Where teachers teach two or more eligible sections within a grade, teacher ID is incorporated into the block, such that such teachers will have at least one class in each of the control and treatment. And, where schools identify section-level or track (e.g., advanced, co-taught, math intervention), strata are further defined within type. Strata containing a single eligible section are pooled at the district × grade level, and any section still unmatched is assigned by an independent 50/50 draw. The target sample is approximately 210 classrooms and 4,500 students across 20–25 schools in California, Pennsylvania, Florida, and Oregon. Districts and schools are recruited by PLUS. Eligible sections are identified by the school and shared — along with class rosters and baseline student data — with the research team.

Randomization is planned to proceed in two to four waves, based on the timing of district roster submission and planned tutor start dates. Tutor capacity is tracked across waves as treated sections per tutoring time slot. Sections meeting in slots with no remaining capacity are removed from the eligible roster in the next wave. If a slot's remaining capacity is positive but insufficient for 1:1 assignment among its eligible sections, those sections are randomized in a separate cell with a fixed number of treatment assignments equal to the remaining capacity. No assignment is altered after randomization. The intent-to-treat sample is defined by section enrollment at the randomization cutoff; students are analyzed under their original assignment regardless of subsequent section changes.
Experimental Design Details
Not available
Randomization Method
Random assignment is performed by computer (Stata) using a prewritten assignment program. Before each wave's draw, the random-number seed and the sort order on stable section identifiers are documented. Within each wave, draws are subject to a prespecified balance criterion (acceptance sampling; Morgan and Rubin 2012).
Randomization Unit
Classroom
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
210
Sample size: planned number of observations
4500
Sample size (or number of clusters) by treatment arms
105 classrooms per arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The minimum detectable effect size (MDES) for the primary outcome is 0.16 student-level standard deviations at alpha = 0.05 (two-sided) and 80% power, with a single primary outcome and no multiplicity adjustment. The primary outcome is the end-of-year state mathematics assessment score, standardized within state x grade, so the outcome has a standard deviation of 1 by construction and effects are expressed in student-level SD units. The calculation reflects the experimental design, random assignment at the classroom level (ICC ~ 0.15 across classrooms), 1:1 allocation within school x grade (or school x grade x teacher) strata. The sample includes approximately 210 classrooms with an average of about 20 students each (~4,200 students), and student-level baseline covariates (prior-year state assessment scores and demographics) assumed to explain 50% of student-level outcome variance. MDES is computed with the PUMP package in R.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
It Takes a Village (Plus AI): A Randomized Controlled Trial of Hybrid Human-AI Math Tutoring at Scale
IRB Approval Date
2025-12-12
IRB Approval Number
IRB00000603; FWA00004206
Analysis Plan