Generative AI in Outpatient Healthcare

Last registered on May 21, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
Generative AI in Outpatient Healthcare
RCT ID
AEARCTR-0015851
Initial registration date
May 20, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 21, 2025, 4:06 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
China
Region
Sichuan Province

Primary Investigator

Name
Xinyao Qiu
Affiliation
Stanford University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Yuyu Chen
PI Affiliation
Peking University
Contact Investigator
PI Name
Hongbin Li
PI Affiliation
Stanford University
Contact Investigator
PI Name
Lingsheng Meng
PI Affiliation
Stanford University
Contact Investigator
PI Name
Qingxu Yang
PI Affiliation
University of Hong Kong
Contact Investigator

Additional Trial Information

Status
In development
Start date
2025-05-12
End date
2026-05-12
Keywords
Health
Additional Keywords
Artificial Intelligence
JEL code(s)
I10, O33, D83
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study examines the impact of large language models (LLMs) on the quality of medical services through randomized controlled trials conducted in a hospital setting. We assess whether LLM assistance can enhance service quality and identify potential mechanisms underlying these improvements.
External Link(s)

Registration Citation

Citation
Chen, Yuyu et al. 2025. "Generative AI in Outpatient Healthcare." AEA RCT Registry. May 21. https://doi.org/10.1257/rct.15851-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention (Hidden)
This study examines the impact of providing patients with access to a large language model (LLM) prior to their outpatient visits. In a randomized controlled trial conducted in collaboration with a Chinese hospital, patients are assigned either to a treatment group with access to an LLM chatbot before the visit or to a control group receiving usual care. We evaluate how access to the LLM influences physician decision-making, patient–physician interactions, and the overall outpatient experience.

The intervention is conducted at a single hospital in China and targets physicians and their scheduled patients. Physicians are randomly assigned to either an exposed group or an unexposed group, stratified by key characteristics including gender, age, education, seniority, specialty, and workload. Approximately two-thirds of physicians are assigned to the exposed group, while the remaining one-third are assigned to the unexposed group.

Patients are included in the study if they schedule an online outpatient appointment with participating physicians. Patients who walk in without an online appointment (constituting roughly 30% of all visits) are excluded from the randomization. Among the remaining 70% of patients with online appointments, those scheduled with unexposed physicians are assigned to the control group. Patients scheduled with exposed physicians are further randomized into two subgroups: 50% are assigned to a treatment group receiving a link to the LLM chatbot prior to their visit, and 50% to a control group receiving the usual appointment confirmation. Patients in the treatment group are prompted to learn about their health conditions and prepare in advance for their visits.

Based on the design, physicians in the exposed group will interact with patients from both the control and treatment groups, allowing for within-physician comparisons. In contrast, physicians in the unexposed group serve as a pure control, enabling between-physician comparisons.

Patients are incentivized to complete a short survey after their visits, and physicians are instructed to complete a survey at the end of the intervention.

The primary outcomes include:
1. Detailed medical notes, visit length, use of diagnostic tools, and prescriptions.
2. Physicians’ perceptions of the visit experience.
3. Patients’ perceptions of the visit experience.
We also examine indicative measures of patient health outcomes, particularly admissions and follow-up visits.
Intervention Start Date
2025-05-26
Intervention End Date
2025-06-08

Primary Outcomes

Primary Outcomes (end points)
Short-term subjective and objective measures of medical services.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Long-term measures of medical services
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Physicians and patients are randomly assigned to either the LLM-exposed or unexposed group, respectively.
Experimental Design Details
Physicians are randomly assigned to either the exposed or unexposed group, stratified by key characteristics including gender, age, education, seniority, specialty, and workload. Among patients with online outpatient appointments, those scheduled to see physicians in the exposed group are further randomized to receive either a link to the LLM chatbot along with the appointment confirmation (treatment) or the usual appointment confirmation alone (control). Patients scheduled to see physicians in the unexposed group are all assigned to the control group.
Randomization Method
randomization done in office by a computer
Randomization Unit
Physicians and patients are randomized at the individual level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
210 physicians
Sample size: planned number of observations
A total of 210 physicians and an average of 1,200 patients per day
Sample size (or number of clusters) by treatment arms
Physicians: 70 physicians unexposed, 140 physicians exposed.
Patients (per day): 800 control group, 400 treatment group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
PKU GSM-IRB
IRB Approval Date
2025-05-07
IRB Approval Number
PKU GSM-IRB #2025-14
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials