NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Evaluating a Patient-Centered Tool to Help Medicare Beneficiaries Choose Prescription Drug Plans (CHOICE)
Initial registration date
July 18, 2019
July 19, 2019 12:06 PM EDT
Stanford University School of Medicine
Other Primary Investigator(s)
University of California San Diego
Additional Trial Information
Patient-Centered Outcomes Research Institute (PCORI) Award (CDR-1306-03598); ClinicalTrials.gov Registry (NCT02895295)
The objective of this study is to determine whether providing Medicare beneficiaries with a web-based patient-centered decision tool to help them choose among prescription medication coverage plans improves outcomes for patients including a greater likelihood of changing a plan, better coverage for prescribed drugs, less decisional conflict when choosing plans, and greater satisfaction with the choice process relative to current practice. In this study, the investigators tested the effectiveness of two versions of a web-based tool (called CHOICE) to help people choose among Medicare Part D plans (Treatments A and B) relative to standard care (Control). Both treatment arms incorporated simplified design and automated importation of an individual's prescription drugs relative to standard care. The treatment arms varied based on whether they provided expert guidance on recommended plans. In the control arm, study participants were directed to the existing, publicly available Medicare.gov website and received instructions on how to download their drugs from the Palo Alto Medical Foundation (PAMF) patient-facing online personal health portal (myhealthonline). The study sample included PAMF patients who were enrolled in Part D plans (not Medicare Advantage) during the 2016 enrollment period. Prior to the 2017 open enrollment period (October 15 to December 7, 2016), we invited a subset of PAMF patients not covered by either MediCal or a Medicare Advantage plan, aged 66-85, residing in 4 counties served by PAMF, and with at least one active medication order to participate in a study examining the effectiveness of decision tools that provide personalized information on the financial implications of enrolling in different Part D plans. The primary study outcomes included 1) Plan switching, 2) Decisional conflict 3) Satisfaction with the choice process, and 4) Change in generosity of coverage of prescription drugs. The investigators measured the primary study outcomes using a combination of administrative data and a post open enrollment survey. The investigators also collected information on individual characteristics at the time of enrollment in the study and implemented a survey examining use of the intervention tool to assess patient experience at the time of use.
Bundorf, M. Kate and Ming Tai-Seale. 2019. "Evaluating a Patient-Centered Tool to Help Medicare Beneficiaries Choose Prescription Drug Plans (CHOICE)." AEA RCT Registry. July 19.
Study subjects randomized to the control arm will receive information on how to download their prescription drug information from their electronic medical record and provided with a list of resources available in the community to help them choose a prescription drug plan.In the "Expert Recommendation" arm, participants will have access to a decision support tool that provides personalized information on the financial implications of enrolling in different plans and expert recommendations of particular plans. In the "Individual Analysis" arm, participants will have access to a decision support tool that provides personalized information on the financial implications of enrolling in different plans.
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
Plan switching; Decisional Conflict; Satisfaction With the Choice Process; Change in Estimated Prescription Drug Spending
Primary Outcomes (explanation)
Secondary Outcomes (end points)
Time Spent Choosing a Plan; Enrolled in Expert-Recommended plan in 2017; Medicare Part D Knowledge
Secondary Outcomes (explanation)
Interventional study with participants randomly assigned to one of three arms, two treatment arms and one control arm.
Experimental Design Details
randomization done in office by computer
Was the treatment clustered?
Sample size: planned number of clusters
Sample size: planned number of observations
Sample size (or number of clusters) by treatment arms
305 in each arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
See Clinicaltrials dot gov registration for detailed information on sample size calculation
INSTITUTIONAL REVIEW BOARDS (IRBs)
Sutter Health Institutional Review Board
IRB Approval Date
IRB Approval Number
Post Trial Information
Is the intervention completed?
Intervention Completion Date
December 07, 2016, 12:00 AM +00:00
Is data collection complete?
Data Collection Completion Date
January 20, 2017, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
313 individuals control, 316 individuals expert recommendation; 299 individuals individual analysis
Reports, Papers & Other Materials
Algorithms increasingly assist consumers in making their purchase decisions across a variety of markets;
yet little is known about how humans interact with algorithmic advice. We examine how algorithmic,
personalized information affects consumer choice among complex financial products using data from a randomized,
controlled trial of decision support software for choosing health insurance plans. The intervention
significantly increased plan switching, cost savings, time spent choosing a plan, and choice process satisfaction,
particularly when individuals were exposed to an algorithmic expert recommendation. We document
systematic selection - individuals who would have responded to treatment the most were the least likely to
participate. A model of consumer decision-making suggests that our intervention affected consumers’ signals
about both product features (learning) and utility weights (interpretation).
M. Kate Bundorf, Maria Polyakova, Ming Tai-Seale, "How do Humans Interact with Algorithms: Evidence from Health Insurance", NBER Working Paper #21497
Choosing a health insurance plan is difficult for many people, and patient-centered decision support may help consumers make these choices. We tested whether providing a patient-centered decision-support tool—with or without machine-based, personalized expert recommendations—influenced decision outcomes for Medicare Part D enrollees. We found that providing an online patient-centered decision-support tool increased older adults’ satisfaction with the process of choosing a prescription drug plan and the amount of time they spent choosing a plan. Providing personalized expert recommendations as well increased rates of plan switching. Many people who could have accessed the tool chose not to, and the characteristics of people who used the tool differed from those who did not. We conclude that a patient-centered decision-support tool providing personalized expert recommendations can help people choose a plan, but different approaches may be necessary to encourage more people to periodically reevaluate their options.
M. Kate Bundorf, Maria Polyakova, Cheryl Stults, Amy Meehan, Roman Klimke, Ting Pun, Albert Solomon Chan, and Ming Tai-Seale, 2019. "Machine-Based Expert Recommendations And Insurance Choices Among Medicare Part D Enrollees", Health Affairs, Vol. 38, No.3
REPORTS & OTHER MATERIALS