Overcoming Medical Overuse with AI Assistance: An Experimental Investigation

2024-04-17

2024-06-30

The key outcome variables in this experiment are the degree of physician overtreatment, and AI on accuracy rate.

We conducted a lab-in-the-field experiment at a medical school affiliated with Central South Hospital, utilizing a two-by-three factorial design. The participants, referred to as prospective physicians, were recruited from this institution, where they were trained to become medical doctors.

Incentive Schemes (Between-Subject): The experiment varied the incentive structures for participants when making choices in a multiple-choice medicine prescription task. Three types of incentives were tested: flat, progressive, and regressive, affecting how participants were compensated for their choices.

AI Assistance (Within-Subject): We also varied the decision-making process by introducing an AI assistance feature. Participants made initial choices independently and could then opt to view AI-generated recommendations before finalizing their decisions.

Control Variables: To account for individual fixed effects and other potential confounders affecting the propensity for overtreatment, we implemented pre- and post-experiment assessments. These assessments collected data on doctors’ professional abilities, cognitive reflection (CRT), IQ, algorithm literacy, trust in algorithms, awareness of algorithms, and perceptions of algorithm fairness.

1.Medicine Prescription Task
Participants answered a set of 20/817 multiple-choice questions derived from a standard practice book. They were informed that each question had only one correct answer but were allowed to select up to two answers to simulate potential overtreatment scenarios. An example of such a question is provided below:
Female, 54 years old, with a history of hyperthyroidism. Recently, due to overwork and emotional stress, she has experienced insomnia, and heart and chest discomfort. Physical examination: Heart rate 160 beats per minute, electrocardiogram shows clear signs of myocardial ischemia, and sinus rhythm irregularity. The best choice would be:
Options:
A. Amiodarone
B. Quinidine
C. Procainamide
D. Propranolol
E. Lidocaine


2.Treatment Design
• Flat Treatment: Participants received a constant monetary payoff of 3 yuan for each task completed, regardless of their choice.
• Progressive Treatment: Participants received a higher payoff for selecting two options (4 yuan) versus one (2 yuan).
• Regressive Treatment: Participants received varied payoffs depending on the number of choices and accuracy of their answers: 4 yuan for one correct choice, 3 yuan for two choices including the correct answer, 1 yuan for one incorrect choice, and 0 yuan for two incorrect choices.
Additionally, depending on participants choices, we donate to a patients regarding charity with the following rules: 8 yuan for one correct choice, 6 yuan for two choices including the correct answer, 2 yuan for one incorrect choice, and 0 yuan for two incorrect choices.


AI assistance was presented as follows:
The AI recommends option: D.
Analysis: Based on the patient’s symptoms and physical examination results, the optimal medication choice is usually D. Propranolol. This beta-blocker is commonly used to control rapid heart rates and arrhythmias, particularly in hyperthyroidism cases. It helps slow down the heart rate, reduce the cardiac workload, and alleviate myocardial ischemia symptoms. However, a specific treatment plan should be consulted with a physician.

randomization in recruitment

Individual.

Yes

We aim to recruit between 120 and 160 participants.

We aim to recruit between 120 and 160 participants.

40-60 individuals in the Group 1，Group 2 and Group 3, respectively.