Testing the efficacy of AI tutoring in secondary mathematics: A scaled RCT in UK classrooms

One-to-one tutoring is widely regarded as the gold standard for personalised education, yet it remains prohibitively expensive to deliver at scale. Recent advances in generative AI have prompted growing interest in whether large language models (LLMs) can approximate the pedagogical effectiveness of expert human tutors. However, empirical evidence remains scarce. Most published evaluations of AI tutoring systems rely on user satisfaction metrics or short-duration laboratory tasks rather than rigorous field experimentation with validated learning outcome measures.

An exploratory randomised controlled trial (RCT) conducted in 2025 provided initial evidence that AI tutoring can support student mathematics learning at levels similar to expert human tutoring (N = 165; arxiv:2512.23633). In that trial, an AI tutor integrated into the Eedi mathematics platform and supervised by human tutors produced a 5.5 percentage-point increase in students’ likelihood of correctly answering novel questions relative to expert human tutoring alone. Supervising tutors approved 76% of the AI tutor’s drafted messages without substantive edits, and communicated general enthusiasm for the pedagogical quality and performance of the AI tutor.
The present trial scales and extends the design of the exploratory trial, with the aim of testing whether the initial findings generalise to a larger cohort, accumulate over a sustained intervention period, and translate to gains on an independent standardised assessment. We will recruit and randomize 1,200 students across UK secondary schools to four arms over a full school term. Students in the control arm will receive Eedi's standard remediation (static content and practice). Students in two AI tutoring arms will receive interactive tutoring from an AI system operating under expert supervision, with the arms differing in the contextual information provided to the AI: one will receive only session-level pedagogical context, while the other will receive longitudinal student information incorporating historical performance, prior misconceptions, and curriculum position. Students in the fourth arm will receive support from human expert tutors. Our primary outcome will be change on STAR Maths (Renaissance), a computer-adaptive standardised assessment administered at baseline and endline. Secondary outcomes will include: near-term measures of learning gains; self-report measures of student self-efficacy, motivation, and attitudes toward learning from errors; and tutor affect and experience across conditions.