AI for infoemation communication issues in healthcare

This study evaluates the effect of artificial intelligence-assisted diagnostic recommendations on physicians' clinical judgment. Trained field staff will visit different hospitals in China using standardized thyroid-related case scenarios. Depending on random assignment, field staff will present or refer to different types of AI-generated recommendations during the consultation and record physicians' diagnostic judgments, testing recommendations, and treatment recommendations for borderline thyroid cases. The study aims to examine whether AI-generated information affects physicians' clinical decision-making in borderline thyroid-related cases.

2025-11-01

2026-11-01

The primary outcomes are:
1. Whether the physician recommends further treatment or intervention;
2. Whether the physician recommends observation, follow-up, or conservative management;
3. The number and type of additional tests recommended by the physician;
4. The physician's assessment of disease severity or risk level;
5. Whether the physician's recommendation is aligned with the direction of the AI-generated recommendation.

Physicians' recommendations will be coded based on post-consultation recording forms. Treatment inclination will be coded as a binary variable indicating whether the physician recommends further treatment or intervention. Conservative management will be coded as whether the physician recommends observation, follow-up, or no immediate treatment. Testing recommendations will be counted and classified by test type. Alignment with AI will be coded based on whether the physician's final recommendation is consistent with the direction of the randomly assigned AI recommendation. If a composite treatment-intensity index is constructed, component variables will be oriented so that higher values indicate a stronger inclination toward treatment or intervention, standardized, and averaged.

Secondary outcomes include whether the physician asks about the source of the AI recommendation, whether the physician expresses trust or skepticism toward the AI recommendation, consultation length, whether follow-up or referral is recommended, whether medication- or surgery-related management is recommended, and the level of detail in the physician's explanation of the recommendation.

This study uses a randomized field experiment design. Trained field staff will visit different hospitals in China and consult physicians using standardized thyroid-related case scenarios. Each hospital consultation constitutes one experimental observation. The study plans to complete approximately 200 experimental observations, of which approximately 100 have already been completed. During the consultation, field staff will present different types of AI-assisted recommendations, or no AI recommendation, according to random assignment. The study will compare physicians' diagnostic judgments, testing recommendations, and treatment recommendations across experimental conditions to assess whether AI-generated information affects clinical decision-making in borderline cases.

Not available

Randomization will be conducted before each hospital visit by the research team using a computer-generated random assignment procedure. Each planned visit will be assigned to one experimental condition. Where feasible, randomization will be stratified by city, hospital level, or field staff to improve balance across experimental conditions. The assigned condition will be communicated to the corresponding field staff before the visit, and the field staff will use the corresponding case script and AI recommendation text.

The unit of randomization is the individual hospital consultation or field visit. Each visit will be randomly assigned to one experimental condition. If multiple visits occur within the same hospital, the analysis will account for potential correlation at the hospital level.

No

Planned number of clusters: Approximately 200 hospital consultations or field visits. If multiple visits occur within the same hospital, the number of distinct hospitals will be recorded separately.

Planned number of observations: Approximately 200 hospital consultations or field visits. Approximately 100 observations have already been completed, and the study plans to complete approximately 100 additional observations.

Sample size by treatment arm: If three experimental conditions are used, approximately one-third of visits will be assigned to the AI recommendation favoring further treatment condition, approximately one-third to the AI recommendation favoring observation or conservative management condition, and approximately one-third to the no-AI-recommendation control condition. Final group sizes may vary slightly due to field implementation constraints.

Based on a planned sample size of approximately 200 hospital consultations or field visits, a 5 percent significance level, 80 percent statistical power, and approximately equal allocation across three experimental conditions, the study is powered to detect medium-sized treatment effects. For a binary primary outcome, such as whether the physician recommends further treatment, assuming a control-group mean of approximately 50 percent, the minimum detectable effect for a pairwise comparison is approximately 24-25 percentage points. For a standardized continuous outcome, the minimum detectable effect for a pairwise comparison is approximately 0.48-0.50 standard deviations. The actual minimum detectable effect will depend on the final allocation across conditions, the variance of the outcome variables, and whether standard errors are clustered at the hospital level.