Incentive Design on MOOC: A Field Experiment

Last registered on May 13, 2019

Pre-Trial

Trial Information

General Information

Title
Incentive Design on MOOC: A Field Experiment
RCT ID
AEARCTR-0004201
Initial registration date
May 13, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 13, 2019, 11:58 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Tsinghua University

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2015-02-01
End date
2016-02-29
Secondary IDs
Abstract
In this study, we examine the impact of monetary incentives on learner engagement and learning outcomes in massive open online courses (MOOCs). While MOOCs have innovated education by offering high-quality interactive educational resources to users worldwide, maintaining student learning enthusiasm in these courses is a challenge. To address this issue, we conduct a field experiment in which users are given monetary incentives to engage in online learning. Our results show that those given a monetary incentive are more likely to submit homework and to gain higher grades. The effect largely reflects the continued involvement of regular users whose activity would otherwise decrease over time. We further find that the effect persists even after we remove the monetary incentives and that it spills over into learning behavior in other courses in the same and subsequent semester. Lastly, we find that females and users from regions with fewer educational institutions show stronger treatment effects. Overall, our findings suggest that monetary incentives counteract the decay of learning engagement and may help online education users form persistent learning habits.
External Link(s)

Registration Citation

Citation
Liu, Tracy. 2019. "Incentive Design on MOOC: A Field Experiment." AEA RCT Registry. May 13. https://doi.org/10.1257/rct.4201-1.0
Former Citation
Liu, Tracy. 2019. "Incentive Design on MOOC: A Field Experiment." AEA RCT Registry. May 13. https://www.socialscienceregistry.org/trials/4201/history/46495
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2015-04-06
Intervention End Date
2015-07-01

Primary Outcomes

Primary Outcomes (end points)
assignment submission and grade
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
To investigate the effect of monetary incentives on learner engagement and performance, we use a 3x2 factorial design, with our control group receiving no incentive. In treatment groups, we vary the incentive size and framing. On the size dimension, we offer three levels of incentives---small, medium and large---to investigate the degree to which incentive size affects engagement and performance. On the framing dimension, we investigate whether positive versus negative framing (i.e, gain versus loss) leads to different effects for a given size of incentive.
Experimental Design Details
Randomization Method
randomization done in office by a computer
Randomization Unit
individual
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
2 courses
Sample size: planned number of observations
800 students (individuals)
Sample size (or number of clusters) by treatment arms
110 students for each of the six treatments, and 110 for control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
July 01, 2015, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials