Human-AI collaboration – experimental evidence

Last registered on September 20, 2023

Pre-Trial

Trial Information

General Information

Title
Human-AI collaboration – experimental evidence
RCT ID
AEARCTR-0012135
Initial registration date
September 17, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 20, 2023, 10:56 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
BNU

Other Primary Investigator(s)

PI Affiliation
BNU

Additional Trial Information

Status
In development
Start date
2023-08-28
End date
2024-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We conduct an audit study in the online healthcare market to analyze the effect of information with various characteristics on physicians’ behavior and the quality of their healthcare.
External Link(s)

Registration Citation

Citation
He, Haoran and Jinwen Xia. 2023. "Human-AI collaboration – experimental evidence." AEA RCT Registry. September 20. https://doi.org/10.1257/rct.12135-1.0
Experimental Details

Interventions

Intervention(s)
N/A
Intervention Start Date
2023-10-15
Intervention End Date
2023-12-31

Primary Outcomes

Primary Outcomes (end points)
Diagnosis: whether the diagnosis of physician is correct or not.
Effort: including the number of recommended items and the number of important items, and the IRT (Item Response Theory) score.
Primary Outcomes (explanation)
The IRT score reflects the quality of items, and its construction can be seen in Das et al. (2016).

Secondary Outcomes

Secondary Outcomes (end points)
Treatment (if the dataset is available): whether the treatment given by physicians is correct or unnecessary/harmful.
Subjective trust (if the dataset is available): physicians’ subjective trust towards the AI tool.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We conduct an audit study in the online healthcare market to analyze the effect of information with various characteristics on physicians’ behavior and the quality of their healthcare.
Experimental Design Details
Not available
Randomization Method
We made a stratified randomization based on physicians’ department and title.
Randomization Unit
The unit of randomization is at the individual physician level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
300 physicians.
Sample size (or number of clusters) by treatment arms
300/6
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The statistical power of the experiment should be sufficient, please refer to the sample section above for an explanation.
IRB

Institutional Review Boards (IRBs)

IRB Name
Business School, Beijing Normal University
IRB Approval Date
2023-09-05
IRB Approval Number
BNU-BS-IRB 2023-026
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information