Clinical Decision Support (CDS) for Radiology Imaging: A Randomized Control Trial
Last registered on March 15, 2019

Pre-Trial

Trial Information
General Information
Title
Clinical Decision Support (CDS) for Radiology Imaging: A Randomized Control Trial
RCT ID
AEARCTR-0001846
Initial registration date
December 14, 2016
Last updated
March 15, 2019 2:21 PM EDT
Location(s)
Primary Investigator
Affiliation
MIT
Other Primary Investigator(s)
PI Affiliation
MIT
PI Affiliation
MIT
PI Affiliation
Aurora Health Care
PI Affiliation
MIT
PI Affiliation
MIT
Additional Trial Information
Status
Completed
Start date
2015-10-27
End date
2019-03-14
Secondary IDs
Abstract
The goal of the study is to determine whether clinical decision support (CDS) for radiology affects the number, type, or appropriateness of targeted high-cost radiology images (i.e. MR, CT, NM, PET) ordered. The CDS will be delivered to physicians in the Aurora Health Care system in Wisconsin. It will be delivered in Epic, an industry-standard electronic medical record software, through ACR Select, which is a leading decision support tool based on the American College of Radiology (ACR) Appropriateness Criteria (see http://www.acr.org/Quality-Safety/Appropriateness-Criteria), and presents the ACR appropriateness scores for each image on a scale of 1-9 with 1-3 labelled as ‘usually not appropriate’, 4-6 ‘May be appropriate’, and 7-9 ‘usually appropriate’.
External Link(s)
Registration Citation
Citation
Abraham, Sarah et al. 2019. "Clinical Decision Support (CDS) for Radiology Imaging: A Randomized Control Trial." AEA RCT Registry. March 15. https://www.socialscienceregistry.org/trials/1846/history/43473
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Clinical Decision Support Treatment:
A best practices alert (BPA) pop-up screen providing CDS will appear at physician sign-off for all scans scored 1-6, and scans scored 7-8 for which an alternative scan scored 8-9 exists.
This screen will show the appropriateness score of the original scan order, and will display up to 7 alternative scans that are scored >4 and greater than or equal to the original score for the same indications and patient characteristics. It will also display a link to relevant ACR documentation relevant to the selected scan and indication.
Intervention Start Date
2016-12-15
Intervention End Date
2017-12-15
Primary Outcomes
Primary Outcomes (end points)
Primary Outcome Measure:

1. Number of scans ordered that would be subject to the CDS best practice alert
[Time Frame: first 365 days after CDS is turned on for the treatment group]
Definition: Scans where the CDS would be triggered based on the criteria listed in the intervention description.

Secondary Outcome Measures:

2. Number of high-cost scans (CT, MR, NM and PET) ordered
[Time Frame: first 365 days after CDS is turned on for the treatment group]
Definition: includes CT, MR, NM and PET scans ordered and performed.

3. Number of scans ordered that would be subject to the CDS best practice alert and that ACR Select rates 1-3 ("usually not appropriate")
[Time Frame: first 365 days after CDS is turned on for the treatment group]
Definition: Scans where the CDS would be triggered and scored 1-3 based on the criteria listed in the intervention description.

4. Number of scans ordered that would be subject to the CDS best practice alert and that ACR Select rates 4-6 ("may be appropriate")
[Time Frame: first 365 days after CDS is turned on for the treatment group]
Definition: Scans where the CDS would be triggered and scored 4-6 based on the criteria listed in the intervention description.

Other pre-specified outcomes

1. Number of low-cost scans ordered
[Time Frame: first 365 days after CDS is turned on for the treatment group]
Definition: includes X-rays, ultra sounds, mammograms, fluoroscopy and bone densitometry
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Inclusion criteria:
• Medical provider actively employed at Aurora with a valid Epic log-in.
• Is a Medical Doctor (MD), Doctors of Osteopathic Medicine (DO), podiatrist (DPM), nurse practitioner (NP), physician assistant (PA), or certified nurse midwife (CNM)
• Has with imaging order permissions at Aurora Health Care.
• Providers had at least one high- or low-cost imaging order in the year from November 1, 2015 or a medical resident who joined Aurora since that time.

Exclusion criteria:
• Opted out of the study prior to November 24, 2016

Providers were randomly allocated equally into treatment or control groups with no stratification.
Experimental Design Details
Randomization Method
Randomization done in office by a computer (Stata)
Randomization Unit
Health care provider (individual)
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
3511 providers
Sample size: planned number of observations
3511 providers
Sample size (or number of clusters) by treatment arms
1,756 control, 1,755 treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Using historical data from the electronic medical record at Aurora Health Care spanning November 1, 2015 to December 6, 2016 we performed power calculations using parametric and nonparametric (simulation) methods: repeatedly simulating data under an assumed model of the effects of CDS on targeted high-cost (HC) scans, estimating a statistical model on the simulated data, and computing the fraction of simulations in which we were able to reject the null hypothesis of no effect. Results are very similar regardless of the method. Using a 7-month intervention window and a six-month pre-period to calculate the lag of the dependent variable, we calculate a minimum detectable effect (MDE) of -1.5 scan orders that would trigger the BPA (a 13% reduction compared to the mean of 11.1 imaging orders with a standard deviation of 23.8). For high-cost scans our MDE is -3 (7% of the mean of 47 scans and a standard deviation of 93), -0.6 imaging orders scored 1-3 (19% compared to a mean of 3.1 and a standard deviation of 8.0). This compares favorably to our review of observational studies finds that when CDS was implemented in comparable outpatient settings for MR and CT scans, the roll-out of CDS correlated with 20-70% reduction in the rate of “inappropriate” scanning by 20-70% and a 20 to 30% reduction in high cost scan volume (Blackmore et al., 2011; Ip et al., 2014, 2013; Lehnert et al., 2010, Solberg et al., 2010).
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Aurora Health Care, Inc. Institutional Review Board
IRB Approval Date
2016-03-09
IRB Approval Number
15-138E
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
Yes
Intervention Completion Date
December 15, 2017, 12:00 AM +00:00
Is data collection complete?
Yes
Data Collection Completion Date
March 15, 2018, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
3,511
Final Sample Size (or Number of Clusters) by Treatment Arms
1,756 health care providers control, 1,755 health care providers
Data Publication
Data Publication
Is public data available?
Yes
Program Files
Program Files
Yes
Reports and Papers
Preliminary Reports
Relevant Papers
Abstract
There is widespread concern over the health risks and healthcare costs from potentially inappropriate high-cost imaging. As a result, the Centers for Medicare and Medicaid Services (CMS) will soon require high-cost imaging orders to be accompanied by Clinical Decision Support (CDS): software that provides appropriateness information at the time orders are placed via a best practice alert for targeted (i.e. likely inappropriate) imaging orders, although the impacts of CDS in this context are unclear. In this randomized trial of 3,511 healthcare providers at Aurora Health Care, we study the impacts of CDS on the ordering behavior of providers. We find that CDS reduced targeted imaging orders by a statistically significant 6%, however there was no statistically significant change in the total number of high-cost scans or of low-cost scans. The results suggest that the impending CMS mandate requiring healthcare systems to adopt CDS may modestly increase the appropriateness of high-cost imaging.
Citation
Doyle J, Abraham S, Feeney L, Reimer S, Finkelstein A (2019) Clinical decision support for high-cost imaging: A randomized clinical trial. PLoS ONE 14(3): e0213373. https://doi.org/10.1371/journal.pone.0213373