AI-assisted decision making – an audit study

Last registered on September 20, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
AI-assisted decision making – an audit study
RCT ID
AEARCTR-0012134
Initial registration date
September 17, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 20, 2023, 10:58 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Jinwen Xia
Affiliation
BNU
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Haoran He
PI Affiliation
BNU
Contact Investigator

Additional Trial Information

Status
In development
Start date
2023-08-28
End date
2024-12-31
Keywords
Behavior, Other
Additional Keywords
Field experiment, Artificial Intelligence
JEL code(s)
C93, I11
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We run a field experiment in the online healthcare market to study the effect of various AI-assisted diagnostic information on physicians’ behavior and the quality of their healthcare.
External Link(s)

Registration Citation

Citation
He, Haoran and Jinwen Xia. 2023. "AI-assisted decision making – an audit study." AEA RCT Registry. September 20. https://doi.org/10.1257/rct.12134-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
n/a
Intervention Start Date
2023-09-21
Intervention End Date
2023-10-15

Primary Outcomes

Primary Outcomes (end points)
Diagnosis: whether the diagnosis of physician is correct or not.
Effort: including the number of recommended items, the number of important items, and the IRT (Item Response Theory) score.
Primary Outcomes (explanation)
The IRT score reflects the comprehensive quality of items, and its construction can be seen in Das et al. (2016).

Secondary Outcomes

Secondary Outcomes (end points)
Treatment (if the dataset is available): whether the treatment given by physicians is correct or unnecessary/harmful.
Subjective trust (if the dataset is available): physicians’ subjective trust towards the AI.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We run a field experiment in the online healthcare market to study the effect of various AI-assisted diagnostic information on physicians’ behavior and the quality of their healthcare.
Experimental Design Details
Not available
Randomization Method
We made a stratified randomization based on physicians’ department and title.
Randomization Unit
The unit of randomization is at the individual physician level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
n/a
Sample size: planned number of observations
250 physicians.
Sample size (or number of clusters) by treatment arms
250/5.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The statistical power of the experiment should be sufficient, please refer to the sample section above for an explanation.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Business School, Beijing Normal University
IRB Approval Date
2023-09-05
IRB Approval Number
BNU-BS-IRB 2023-026
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information