The Impact of Generative AI on Employee Performance: A Field Experiment in a Multinational Education Technology Company

Last registered on April 03, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
The Impact of Generative AI on Employee Performance: A Field Experiment in a Multinational Education Technology Company
RCT ID
AEARCTR-0015560
Initial registration date
March 27, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 03, 2025, 11:09 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
China
Region

Primary Investigator

Name
Hugh Xiaolong Wu
Affiliation
Washington University in St. Louis Olin Business School
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Huayu Xu
PI Affiliation
Peking University
Contact Investigator

Additional Trial Information

Status
In development
Start date
2025-03-01
End date
2025-08-01
Keywords
Firms & Productivity, Labor
Additional Keywords
generative AI, productivity, personnel management, employee performance
JEL code(s)
J24, L80, M12, M50
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study examines whether promoting the use of Generative AI can improve job performance among course counselors (professional sales) working at a leading education technology company in China, and seeks to understand the underlying reasons for any improvements in job productivity. Approximately 500 counselors will participate in the study. All participants will have access to an internal database powered by ChatGPT, but about one-third (the treatment group) will receive additional encouragement to actively integrate Generative AI into their everyday tasks. The remaining counselors (the control group) will not receive this extra support.

By comparing job performance between the two groups, the study aims to determine if counselors in the treatment group achieve better productivity outcomes—particularly by reducing student attrition between course modules. Specifically, after completing the current module, parents decide whether to enroll their children in the next one. The study assesses whether counselors supported by Generative AI are more effective at encouraging parents to continue enrolling their children, thereby lowering dropout rates. The research will further explore the mechanisms driving any observed improvements in performance.
External Link(s)

Registration Citation

Citation
Wu, Hugh Xiaolong and Huayu Xu. 2025. "The Impact of Generative AI on Employee Performance: A Field Experiment in a Multinational Education Technology Company." AEA RCT Registry. April 03. https://doi.org/10.1257/rct.15560-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
All participating counselors will have access to an internal Generative AI database powered by ChatGPT. Counselors assigned to the treatment group (approximately one-third of participants) will receive additional encouragement to actively integrate Generative AI into their daily work activities. This support includes demonstrations, regular reminders, and resources on effectively using Generative AI tools to enhance interactions with students' parents, aiming to improve their job performance and reduce student attrition between course modules. The control group counselors will receive access to the same Generative AI resources but without additional promotion.
Intervention (Hidden)
Intervention Start Date
2025-04-01
Intervention End Date
2025-07-01

Primary Outcomes

Primary Outcomes (end points)
customer attrition
Primary Outcomes (explanation)
The primary outcome is counselor productivity, measured specifically by student attrition rates. Attrition is assessed by tracking whether parents discontinue their relationship with the company by not purchasing additional lessons. Lower attrition rates indicate higher counselor productivity and effectiveness in retaining students.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This study uses a randomized controlled trial (RCT) design. Approximately 500 course counselors from a leading education technology company in China will be randomly assigned to either a treatment or a control group. All counselors will have access to an internal Generative AI database powered by ChatGPT. However, counselors in the treatment group (about one-third of participants) will receive additional encouragement designed to actively promote the integration of Generative AI into their daily work routines. The remaining counselors (control group) will have access to the same resources but will not receive any additional promotion. The primary outcome—student attrition rates after the current course module—will be compared between the two groups to evaluate the effectiveness of this intervention.
Experimental Design Details
Randomization Method
Randomization will be done by the company's data science team using a random number generator.
Randomization Unit
The randomization will be done at the counselor group level, with each group consisting of approximately 12 counselors.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
~ 40 counselor groups
Sample size: planned number of observations
~ 500 counselors
Sample size (or number of clusters) by treatment arms
Approximately one-third of the counselor groups were assigned to the treatment condition, while two-thirds were assigned to the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
WashU IRB
IRB Approval Date
2025-03-13
IRB Approval Number
202503053
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials