AI-based Experimentation on MOOC

Last registered on May 27, 2020

View Trial History

Pre-Trial

Trial Information

General Information

Title

AI-based Experimentation on MOOC

RCT ID

AEARCTR-0005916

Initial registration date

May 27, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

May 27, 2020, 12:18 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

China

Region

Primary Investigator

Name

Tracy Liu

Affiliation

Tsinghua University

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Stephanie Wang

PI Affiliation

University of Pittsburgh

Contact Investigator

PI Name

Ulrike Malmendier

PI Affiliation

University of California at Berkeley

Contact Investigator

Additional Trial Information

Status

In development

Start date

2020-05-28

End date

2021-07-31

Keywords

Education

Additional Keywords

JEL code(s)

Secondary IDs

Abstract

Low-engagement is a central challenging for Massive Open Online Course users. In particular, most students are silent and do not ask questions. Deploying Xiaomu, the AI teaching assistant on XuetangX, which is the largest MOOC platform in China, we examine which social information is more effective for encouraging students to ask questions and consequently improve their learning performance.

External Link(s)

Registration Citation

Citation

Liu, Tracy, Ulrike Malmendier and Stephanie Wang . 2020. "AI-based Experimentation on MOOC ." AEA RCT Registry. May 27. https://doi.org/10.1257/rct.5916-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Intervention Start Date

2020-05-28

Intervention End Date

2021-01-31

Primary Outcomes

Primary Outcomes (end points)

(1)A user's frequency for asking questions to Xiaomu; (2) A user's learning activities including the time spent on watching videos and her grades.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

A user for a course will randomly see different greeting message once she clicks Xiaomu.
1. Control: I am the AI TA, Xiaomu. I will learn and make progress with you together.

2. Prosocial Message: I am the AI TA, Xiaomu. I will learn and make progress with you together. The more question you ask will benefit students who have the same/similar doubts.

3. Authority Figure Message：I am the AI TA, Xiaomu. I will learn and make progress with you together. The instructor expects you to ask more questions and this shows that you are paying attention.

4. Instrumental benefits Message： I am the AI TA, Xiaomu. I will learn and make progress with you together. Asking more questions could improve your grade in the class.

5. Upward comparison/role model+instrumental benefits ： I am the AI TA, Xiaomu. I will learn and make progress with you together. Tsinghua students who ask more questions perform better in the class. You can too.

Experimental Design Details

Randomization Method

randomization done in office by a computer

Randomization Unit

individual

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

9588 users from three different courses we are going to implement the experiment

Sample size: planned number of observations

same as number of clusters

Sample size (or number of clusters) by treatment arms

1916 in control; 1917 in Prosocial Message: 1917 in Authority Figure Message; 1918 in Instrumental benefits Message; 1920 in Upward comparison/role model+instrumental benefits

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Supporting Documents and Materials

IRB