x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
AI-based Experimentation on MOOC
Last registered on May 27, 2020

Pre-Trial

Trial Information
General Information
Title
AI-based Experimentation on MOOC
RCT ID
AEARCTR-0005916
Initial registration date
May 27, 2020
Last updated
May 27, 2020 12:18 PM EDT
Location(s)
Region
Primary Investigator
Affiliation
Tsinghua University
Other Primary Investigator(s)
PI Affiliation
University of Pittsburgh
PI Affiliation
University of California at Berkeley
Additional Trial Information
Status
In development
Start date
2020-05-28
End date
2021-07-31
Secondary IDs
Abstract
Low-engagement is a central challenging for Massive Open Online Course users. In particular, most students are silent and do not ask questions. Deploying Xiaomu, the AI teaching assistant on XuetangX, which is the largest MOOC platform in China, we examine which social information is more effective for encouraging students to ask questions and consequently improve their learning performance.
External Link(s)
Registration Citation
Citation
Liu, Tracy, Ulrike Malmendier and Stephanie Wang . 2020. "AI-based Experimentation on MOOC ." AEA RCT Registry. May 27. https://doi.org/10.1257/rct.5916-1.0.
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2020-05-28
Intervention End Date
2021-01-31
Primary Outcomes
Primary Outcomes (end points)
(1)A user's frequency for asking questions to Xiaomu; (2) A user's learning activities including the time spent on watching videos and her grades.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
A user for a course will randomly see different greeting message once she clicks Xiaomu.
1. Control: I am the AI TA, Xiaomu. I will learn and make progress with you together.

2. Prosocial Message: I am the AI TA, Xiaomu. I will learn and make progress with you together. The more question you ask will benefit students who have the same/similar doubts.

3. Authority Figure Message:I am the AI TA, Xiaomu. I will learn and make progress with you together. The instructor expects you to ask more questions and this shows that you are paying attention.

4. Instrumental benefits Message: I am the AI TA, Xiaomu. I will learn and make progress with you together. Asking more questions could improve your grade in the class.

5. Upward comparison/role model+instrumental benefits : I am the AI TA, Xiaomu. I will learn and make progress with you together. Tsinghua students who ask more questions perform better in the class. You can too.
Experimental Design Details
Not available
Randomization Method
randomization done in office by a computer
Randomization Unit
individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
9588 users from three different courses we are going to implement the experiment
Sample size: planned number of observations
same as number of clusters
Sample size (or number of clusters) by treatment arms
1916 in control; 1917 in Prosocial Message: 1917 in Authority Figure Message; 1918 in Instrumental benefits Message; 1920 in Upward comparison/role model+instrumental benefits
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number