A Field Experiment on Adaptive Learning Applying a Machine Learning Algorithm

Last registered on May 26, 2021


Trial Information

General Information

A Field Experiment on Adaptive Learning Applying a Machine Learning Algorithm
Initial registration date
May 26, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 26, 2021, 10:28 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.



Primary Investigator

Johannes Gutenberg University Mainz

Other Primary Investigator(s)

PI Affiliation
Johannes Gutenberg University Mainz
PI Affiliation
Johannes Gutenberg University Mainz
PI Affiliation
Johannes Gutenberg University Mainz

Additional Trial Information

In development
Start date
End date
Secondary IDs
It remains an open question how to adapt and individualize learning contents. To tackle this in a digital learning context, we developed an algorithm based on a convolutional neural network that assigns tasks to the learners. Our application is a large online learning platform in which we run a randomized controlled trial. Participants are randomized into three groups: two treatment groups – a group-based adaptive treatment group and an individualized adaptive treatment group – and one control group. We analyze the difference between the three groups with respect to effort learners provide within the platform, their performance within the platform, formative assessments within the platform and their final high-stake standardized exams (summative assessment).
External Link(s)

Registration Citation

Klausmann, Tim et al. 2021. "A Field Experiment on Adaptive Learning Applying a Machine Learning Algorithm." AEA RCT Registry. May 26. https://doi.org/10.1257/rct.7637-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details


There is a control group, a group-based adaptive treatment group, and an individualized adaptive treatment group. The control group receives tasks randomly drawn from the pool of tasks available in the learning platform. Both treatment groups are affected by our “intervention”. Tasks are assigned to learners based on pre-tested distributions that optimize task difficulty and learning effort, which is the number of future tasks that learners will complete in the platform. A machine learning algorithm predicts task difficulty and future effort. The groups differ in the degree of individualization of assigned tasks. The group-based adaptive treatment group receives tasks that are adapted to the mean learner in the platform. The individualized adaptive treatment group receives tasks that are adapted to each individual learner.
Intervention Start Date
Intervention End Date

Primary Outcomes

Primary Outcomes (end points)
Number of attempted tasks
Primary Outcomes (explanation)
Cumulative number of tasks attempted on the paltform throughout the intervention period
Measure for effort

Secondary Outcomes

Secondary Outcomes (end points)
Test score in formative assessments (mock exams) on the platform
Test score in summative assessments (standardized high-stake final exams)
Secondary Outcomes (explanation)
Measure for performance

Experimental Design

Experimental Design
The setting of our experiment is a college-level online learning platform that students use outside the classroom (for more information of the platform see Sponsors & Partner section). Learning in the platform is unmonitored by their teachers. Originally, the platform consists of learning videos and tasks that are matched to the videos and that follow the purpose of deepening the understanding of the video contents. We added a section to the platform that combines learning materials to quizzes of ten tasks. Our intervention algorithm assigns tasks to the learners whenever they choose to take a quiz.
Experimental Design Details
Not available
Randomization Method
Computer-based randomization by script on a server of the cooperation partner
Randomization Unit
Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters
Sample size: planned number of observations
Sample size (or number of clusters) by treatment arms
3,300 group-based adaptive treatment group
3,300 individualized adaptive treatment group
3,300 control group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Based on a previous study in the same platform, we estimate that we have 80 percent power at an effect size of 0.057 standard deviations (SD). Equivalently, we reach 99 percent power at an effect size of 0.095 SD. As inputs in our power calculation, we chose alpha = 0.05, N = 10,000 and R²=0.005.

Institutional Review Boards (IRBs)

IRB Name
Joint Ethics Committee of the Faculty of Economics and Business Administration of Goethe University Frankfurt and the Gutenberg School of Management & Economics of the Faculty of Law, Management and Economics of Johannes Gutenberg University Mainz
IRB Approval Date
IRB Approval Number