Using an Artificially Intelligent, Mobile-Based App to Improve the English Skills and Labor Market Outcomes of Youth in China

Last registered on October 19, 2021

Pre-Trial

Trial Information

General Information

Title
Using an Artificially Intelligent, Mobile-Based App to Improve the English Skills and Labor Market Outcomes of Youth in China
RCT ID
AEARCTR-0008378
Initial registration date
October 14, 2021
Last updated
October 19, 2021, 6:25 PM EDT

Locations

Region

Primary Investigator

Affiliation
Stanford University

Other Primary Investigator(s)

PI Affiliation
Stanford University
PI Affiliation
Stanford University
PI Affiliation
University of California at Santa Cruz

Additional Trial Information

Status
On going
Start date
2021-05-01
End date
2021-10-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
One of the most important skills that students must learn in China’s rapidly growing, increasingly globalized, and modern economy is English. While some students are learning English (including reading, speaking, listening, and writing) at a rapid pace, other students are lagging far behind. Failing to acquire English skills, students may have more difficulty finding jobs and earning higher wages.
With few opportunities to acquire quality English language training, what can be done to support students (especially those talented students that make it to college) in China today? One exciting possibility is the emergence of educational technologies that provide students with teacher-less, readily accessible, and high-quality English language learning opportunities unavailable in the past. How educational technologies can best engage these students to maximize the learning of English is, however, an open question.
The purpose of this study is to examine the impacts of providing a popular and highly advanced educational technology (a mobile app that utilizes artificial intelligence in teaching and improving English skills), on the outcomes of college students in China. Specifically, we will examine (a) the impacts of the app on the English skill levels and subsequent related labor market outcomes of the average college youth in China; (b) the impacts of the app on the English skills and labor market outcomes of students from different backgrounds (by poverty level, gender, and achievement level); (c) ways to improve the use of the app so as to maximize impacts for the average student and for students from different backgrounds. To fulfill this purpose, we will partner with an artificial intelligence (AI) company in China that creates and delivers products and services to popularize English learning. Its proprietary AI teacher utilizes cutting-edge deep learning and adaptive learning technologies, big data, well-established education pedagogies and the mobile internet. The company provides its products and services on demand via its mobile apps, primarily its flagship English mobile app. On the company’s platform, AI technologies are seamlessly integrated with diverse learning content incorporating well-established language learning pedagogies, gamified features and strong social elements to deliver an engaging, adaptive learning experience. The app offers a fun, interactive learning environment to motivate and engage its users.
We will conduct an individual-level block RCT. The RCT will involve approximately 900 college students. Among the students, 450 will randomly receive the mobile app and 450 who will not (the control group).
External Link(s)

Registration Citation

Citation
Fairlie, Robert et al. 2021. "Using an Artificially Intelligent, Mobile-Based App to Improve the English Skills and Labor Market Outcomes of Youth in China." AEA RCT Registry. October 19. https://doi.org/10.1257/rct.8378-1.1
Experimental Details

Interventions

Intervention(s)
Please see the pre-analysis plan document
Intervention Start Date
2021-05-15
Intervention End Date
2021-10-30

Primary Outcomes

Primary Outcomes (end points)
English achievement (z-scored), are collected using a standardized English test in the endline survey.
Primary Outcomes (explanation)
Please see the pre-analysis plan document

Secondary Outcomes

Secondary Outcomes (end points)
The secondary outcome measures (attitudes towards learning and expected labor market outcomes) are also collected in the endline survey.
Secondary Outcomes (explanation)
Please see the pre-analysis plan document

Experimental Design

Experimental Design
We will partner with an artificial intelligence (AI) company in China that creates and delivers products and services to popularize English learning. Its proprietary AI teacher utilizes cutting-edge deep learning and adaptive learning technologies, big data, well-established education pedagogies and the mobile internet. The company provides its products and services on demand via its mobile apps, primarily its flagship English mobile app. On the company’s platform, AI technologies are seamlessly integrated with diverse learning content incorporating well-established language learning pedagogies, gamified features and strong social elements to deliver an engaging, adaptive learning experience. The app offers a fun, interactive learning environment to motivate and engage its users.
We will conduct an individual-level block RCT. The RCT will involve approximately 900 college students. Among the students, 450 will randomly receive the mobile app and 450 who will not (the control group).
Experimental Design Details
Randomization Method
Randomization done in office by a computer
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
NA
Sample size: planned number of observations
Approximately 900 college students
Sample size (or number of clusters) by treatment arms
A.Treatment group: 450 students
B. Control group: 450 students
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The sample size of 450 students per treatment arm was chosen based on a power calculation for individual-level RCTs. Namely, if we assume an alpha of 0.05, an R-squared of 0.45 (between the pre-treatment baseline measures and post-treatment follow-up measures), and a minimum detectable effect size of 0.14 SDs, we would need a sample of 440 participants per treatment arm to be powered at the 80% level
IRB

Institutional Review Boards (IRBs)

IRB Name
Stanford University
IRB Approval Date
2020-12-16
IRB Approval Number
53885
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials