Generative AI in Telemedicine

Last registered on June 04, 2025

View Trial History

Pre-Trial

Trial Information

General Information

Title

Generative AI in Telemedicine

RCT ID

AEARCTR-0016116

Initial registration date

May 30, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

June 04, 2025, 9:55 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

China

Region

Primary Investigator

Name

Yichen Yu

Affiliation

London Business School

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Xu Zhang

PI Affiliation

London Business School

Contact Investigator

Additional Trial Information

Status

In development

Start date

2025-01-20

End date

2025-09-08

Keywords

Health, Labor

Additional Keywords

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

This research project aims to study the impact of providing a generative AI tool to physician assistants who provide ongoing care to patients through a telemedicine platform. In collaboration with a telemedicine company, we will conduct a randomized controlled trial to examine the following research questions:

1) What is the impact of the AI tool on physician assistants’ productivity and quality of service, as well as on patients’ medical adherence and engagement with the platform?
2) How do these effects vary across physician assistants with different levels of experience and expertise, and how do they evolve over time?
3) How does the use of patient-specific information to personalize AI-generated output affect outcomes compared to using a generic AI model?

External Link(s)

Registration Citation

Citation

Yu, Yichen and Xu Zhang. 2025. "Generative AI in Telemedicine." AEA RCT Registry. June 04. https://doi.org/10.1257/rct.16116-1.0

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

The intervention involves providing physician assistants at a telemedicine company with one of two versions of a generative AI tool designed to support continuity of care. The tool generates draft responses to patients’ inquiries. Participants in the treatment groups receive one of the following:

Treatment Arm 1: A generative AI tool that produces draft responses without incorporating patient-specific information.
Treatment Arm 2: A generative AI tool that produces personalized draft responses using patient-specific information.

Intervention (Hidden)

Intervention Start Date

2025-06-09

Intervention End Date

2025-09-08

Primary Outcomes

Primary Outcomes (end points)

1) Physician assistant productivity (e.g., inquiry resolution time).
2) Physician assistant response quality (e.g., accuracy, informational and emotional support, and readability).
3) Patient medical adherence (e.g., booking and attending follow-up appointments, and completing prescribed medication orders).
4) Patient engagement with the telemedicine platform (e.g., patient sentiment in responses and platform attrition).

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

1) Physician assistant job satisfaction, work habits, and attitudes towards healthcare AI.
2) Physician assistant effort in reviewing and editing AI-generated drafts.

Secondary Outcomes (explanation)

Experimental Design

The study involves around 900 physician assistants employed at a telemedicine company that delivers continuity of care to patients by answering patient inquiries and scheduling follow-up visits.

The experiment begins with a control period, during which all physician assistants follow their usual workflow without access to the AI tool. Toward the end of this period, participants complete a baseline survey assessing their workflow, prior experience with AI tools, and attitudes toward healthcare AI.

Following the control period, physician assistants are randomized into one of three groups at the team level:
- Control Group (1/3 of teams): Continue their usual workflow without access to the AI tool.
- Treatment Group 1 (1/3 of teams): Receive access to a basic version of the generative AI tool that does not incorporate patient-specific information.
- Treatment Group 2 (1/3 of teams): Receive access to a personalized version of the AI tool that generates draft responses using patient-specific information.

At the end of the intervention, all physician assistants complete a post-treatment survey, which includes the same questions as the baseline survey, along with additional questions about their experience using the AI tool.

Experimental Design Details

Randomization Method

Randomization will be conducted in-office using a computer-based random assignment procedure.

Randomization Unit

Physician assistant teams.

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

170 teams.

Sample size: planned number of observations

900 physician assistants.

Sample size (or number of clusters) by treatment arms

Each group will have 1/3 of the total number of clusters. Assuming that the number of clusters will be approximately 170, this means, for example:
- 57 clusters (teams) in Control Group, with no access to the AI tool.
- 56 clusters (teams) in Treatment Group 1, receiving a basic version of the AI tool that generates draft responses without incorporating patient-specific information.
- 56 clusters (teams) in Treatment Group 2, receiving a personalized version of the AI tool that generates draft responses based on patient-specific information.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Supporting Documents and Materials

IRB