Can a Tutor AI improve learning and other educational outcomes in post-secondary education?

Last registered on September 22, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
Can a Tutor AI improve learning and other educational outcomes in post-secondary education?
RCT ID
AEARCTR-0016639
Initial registration date
September 19, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 22, 2025, 6:47 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Minoru Higa
Affiliation
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Pierre Mouganie
PI Affiliation
Simon Fraser University

Additional Trial Information

Status
In development
Start date
2025-09-22
End date
2026-05-31
Keywords
Education
Additional Keywords
Artificial Intelligence
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This trial evaluates the impact of providing access to a Tutor AI on learning and related educational outcomes in undergraduate programming courses in Colombia. The intervention consists of two treatment arms. In the Tutor AI Only arm, students receive access to an AI tutor for asynchronous academic activities such as homework, self-study, and projects. In the Tutor AI Plus arm, students receive the same asynchronous access, in addition to periodic in-class workshops where they solve exercises with AI support under the supervision of the instructor. Randomization occurs at the section level. Primary outcomes include academic performance (grades), absenteeism, course withdrawal, and hiring external tutors. Secondary outcomes include self-reported measures of confidence, motivation, and perceived usefulness of programming for the future. Data sources include baseline and follow-up surveys, focus groups, and administrative academic records.
External Link(s)

Registration Citation

Citation
Higa, Minoru and Pierre Mouganie. 2025. "Can a Tutor AI improve learning and other educational outcomes in post-secondary education?." AEA RCT Registry. September 22. https://doi.org/10.1257/rct.16639-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2025-09-22
Intervention End Date
2026-05-31

Primary Outcomes

Primary Outcomes (end points)
Grades
Absenteeism
Course Withdrawal/Drop out
Hiring external tutors
Self-reported confidence to learn programming
Self-reported motivation to learn programming
Self-reported perception that learning programming today will be useful for the future.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
In the pilot phase, course sections were randomly assigned to one of two treatment arms:
Tutor AI Only – Students in this group receive access to the AI tutor as an external assistant for asynchronous academic activities, including homework, self-study, and course projects.
Tutor AI Plus – Students in this group receive the same access to the AI tutor for asynchronous activities as in the Tutor AI Only arm, but additionally participate in periodic in-class workshops throughout the semester. In these workshops, students solve exercises with the support of the AI tutor under the guidance of the course instructor.
Prior to granting access to the AI tutor, the principal investigators (PIs) administer a baseline survey to measure students’ prior experience with AI, as well as their self-reported confidence, motivation, and perceived usefulness of learning programming.
At the end of the first semester, the PIs administer a follow-up survey to collect information on students’ experiences using the AI tutor and to reassess their confidence, motivation, and perceptions of the usefulness of learning programming.
To complement the quantitative data, the PIs also conduct a focus group with students at the end of the semester. Additionally, administrative records are collected to obtain students’ sociodemographic characteristics and academic information.


Experimental Design Details
Not available
Randomization Method
Randomization performed in the office by computer

Randomization Unit
section
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
First semester (pilot): 8 sections
Instructor 1: 2 sections control
Instructor 2: 3 sections, 1 treated with Tutor AI Only, 1 treated with Tutor AI Plus, 1 control
Instructor 3: 3 sections, 1 treated with Tutor AI Only, 1 treated with Tutor AI Plus, 1 control
Given the relatively small number of clusters, we will implement the wild cluster bootstrap-t procedure to obtain valid inference.
Sample size: planned number of observations
First semester (pilot): 364 students Second semester: 360 students Total: 724 students
Sample size (or number of clusters) by treatment arms
Tutor AI Only: 96 students
Tutor AI Plus: 98 students
Control: 170 students

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Universidad de los Andes Comité de Ética
IRB Approval Date
2025-09-10
IRB Approval Number
N/A
Analysis Plan