The Impact of Customized AI Tutoring on Group Learning

Last registered on January 06, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
The Impact of Customized AI Tutoring on Group Learning
RCT ID
AEARCTR-0017406
Initial registration date
December 29, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 06, 2026, 7:12 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Pavitra Govindan
Affiliation
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Fulya Ersoy
PI Affiliation
University of Chicago
Contact Investigator
PI Name
Peilu Zhang
PI Affiliation
University of Utah
Contact Investigator
PI Name
Matthew Brown
PI Affiliation
University of Chicago
Contact Investigator
PI Name
Tomer Yehoshua-Sandak
PI Affiliation
University of Chicago
Contact Investigator

Additional Trial Information

Status
In development
Start date
2026-01-05
End date
2028-06-01
Keywords
Behavior, Education
Additional Keywords
Field Experiment, AI, Group learning
JEL code(s)
A22; C90; C93; D02; D04; D91
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Group work plays a vital role in college students’ learning, and research has long emphasized the cognitive and social benefits of cooperative learning (Johnson, Johnson and Smith, 2014). As artificial intelligence (AI) becomes increasingly integrated into educational settings (Zhang and Aslan, 2021), understanding its role in cooperative learning environments is critical, as dynamics in group work such as peer influence, shared responsibility, and social presence may interact with AI support in complex ways. Studies on AI in cooperative learning have focused on online learning platforms (Tan, Lee and Lee, 2022).

This study investigates whether incorporating a customized AI assistant—trained on course materials and designed to prompt critical thinking—into group work affects student outcomes.

References
Johnson, D. W., Johnson, R. T., & Smith, K. A. (2014). Cooperative learning: Improving university instruction by basing practice on validated theory. Journal on Excellence in College Teaching, 25(3&4), 85–118.

Tan, Seng Chee, Alwyn Vwen Yen Lee, and Min Lee. "A systematic review of artificial intelligence techniques for collaborative learning over the past two decades." Computers and Education: Artificial Intelligence 3 (2022): 100097.

Zhang, Ke, and Ayse Begum Aslan. "AI technologies for education: Recent research & future directions." Computers and education: Artificial intelligence 2 (2021): 100025.
External Link(s)

Registration Citation

Citation
Brown, Matthew et al. 2026. "The Impact of Customized AI Tutoring on Group Learning." AEA RCT Registry. January 06. https://doi.org/10.1257/rct.17406-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Students in discussion sections work on practice problems. In treatment groups, students are allowed to use an AI tool to assist with problem-solving; in control sessions, students are not allowed to use the AI tool. All students can use course materials and ask questions to teaching assistants. After the practice period, students complete a 10-minute quiz without any aids, followed by a short online survey.
Intervention Start Date
2026-01-05
Intervention End Date
2026-05-31

Primary Outcomes

Primary Outcomes (end points)
Weekly quiz grades and midterm exam grades.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Stress level, measure of group interaction, perception of the discussion section, AI usage, attitudes toward group work and attitudes towards AI use.
Secondary Outcomes (explanation)
These outcomes are measured in weekly surveys, midterm survey, and/or final survey.

Experimental Design

Experimental Design
Unit of Randomization: Individual student
Stratification: By instructor-section
Groups: Students are randomly assigned to groups of five within each section, stratified by treatment status. Groups are re-randomized each session.
Timeline: Six discussion sections over eight weeks (three before midterm, three after).
Randomization Procedure:
Pre-midterm (Sections 1–3): Students randomly assigned to treatment or control (1/2 in control and 1/2 in treatment).
Post-midterm (Sections 4–6): Assignment conditional on prior exposure:

Students with treatment in all first three sessions → control for remaining.
Students with no prior treatment → treatment for remaining.
Others → randomized.

Communication: Assignment revealed at the start of each discussion section.
Experimental Design Details
Not available
Randomization Method
Randomization done in office by a computer for treatment assignment. Groups are assigned using https://www.random.org/lists/ list randomizer to randomly order students in each list (treatment and control). Sort students into 5-people groups based on the ordered list.
Randomization Unit
Individual student level randomization
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
The treatment is assigned at the individual student level and the number of individual students are the number of clusters.
Sample size: planned number of observations
600 students X 6 observations/student = 3600 observations for quizzes and variables measured in the weekly surveys 600 observations for variables measured in the midterm survey 600 observations for midterm grade
Sample size (or number of clusters) by treatment arms
Two treatment arms each with 300 students.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We have two primary outcomes: quizzes and midterm exams. For 600 students taking 6 quizzes, assuming 80% power at a 5% significance level (two-sided) the minimal detectable effect size is 0.093 SD, where SD is the within-student standard deviation of quiz scores. Assuming 600 completed midterm exams, the minimal detectable effect per additional exposure (going from 0 to 1, 1 to 2, etc., assuming the effect is linear) for 80% power, two-sided test, α=0.05, is 0.132 SD of the residualized exam score. Power for quizzes: For 600 students taking 6 quizzes, assuming 80% power at a 5% significance level (two-sided) the minimal detectable effect is 0.093 SD, where SD is the within-student standard deviation of quiz scores. Power for midterm exams: Assuming 600 completed midterm exams, the minimal detectable effect per additional exposure (going from 0 to 1, 1 to 2, etc., assuming the effect is linear) for 80% power, two-sided test, α=0.05, 600 students, no controls, is 0.132 SD of the residualized exam score.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
The Social & Behavioral Sciences IRB, University of Chicago
IRB Approval Date
2025-10-30
IRB Approval Number
IRB25-1223
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information