Evaluating Interventions to Reduce Choice Errors in California's Affordable Care Act Marketplace

Last registered on October 05, 2022


Trial Information

General Information

Evaluating Interventions to Reduce Choice Errors in California's Affordable Care Act Marketplace
Initial registration date
October 03, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 05, 2022, 11:13 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.


Primary Investigator

Covered California

Other Primary Investigator(s)

Additional Trial Information

On going
Start date
End date
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
The American Rescue Plan Act of 2021 dramatically expanded the Affordable Care Act’s (ACA) subsidies, but many low-income enrollees may not know that they can switch to a more generous Cost-Sharing Reduction (CSR) Silver plan with no increase in their monthly premium. During the 2022 Open Enrollment Period in California’s ACA marketplace, we tested two interventions designed to increase CSR Silver take-up among low-income households enrolled in bronze plans. The first intervention used a behaviorally-informed letter paired with three email reminders. The second intervention automatically crosswalked eligible households from bronze to $0 CSR Silver plans with the same health insurer at the beginning of the renewal period. Amid the availability of enhanced marketplace subsidies through the end of 2025, our findings inform current health policy debates on the relative effectiveness of different approaches to improve plan choice quality for low-income households.
External Link(s)

Registration Citation

Feher, Andrew. 2022. "Evaluating Interventions to Reduce Choice Errors in California's Affordable Care Act Marketplace." AEA RCT Registry. October 05. https://doi.org/10.1257/rct.10165-1.0
Experimental Details


The intervention was deployed in California's ACA marketplace among low-income households eligible for CSR Silver 94 coverage but enrolled in bronze plans. The intervention for those assigned to the treatment group included a behaviorally-informed letter paired with three email reminders.
Intervention Start Date
Intervention End Date

Primary Outcomes

Primary Outcomes (end points)
We use Covered California administrative data to create our primary outcome: an indicator for whether the household was enrolled in a Cost Sharing Reduction Silver plan at the end of the 2022 Open Enrollment Period.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
In September 2021, we identified 9,288 CSR Silver 94-eligible households with an email address enrolled in bronze plans for the 2021 enrollment year. To understand the extent to which informational nudges – delivered via direct mail and email – could induce these enrollees in bronze plans to switch to CSR Silver plans, we used Stata to randomly assign 30 percent of households to a control group (n = 2,777) and the remaining 70 percent to a letter and email treatment group (n = 6,511). Beyond the required renewal notice, households in the control group did not receive any messages in October 2021, but from November to January they received the standard messages that Covered California sends to renewing consumers during open enrollment, which encourage members to shop and compare plans and provide information on the differences between metal tiers but do not make explicit recommendations. Households in the treatment group received the required renewal notice and were assigned to receive a CSR-specific letter in mid-October followed by three email reminders sent in November and December 2021. These outreach materials included out-of-pocket savings when a consumer visits the doctor or fills a prescription, a screenshot of what the CSR Silver plan would look like when they entered the shopping portal, peer comparison language that noted 9 out of 10 consumers like them choose CSR Silver plans and different ways to switch plans (e.g. self-serving or working with an enrollment assister). Households that had indicated a written language preference of Spanish received outreach in Spanish; all other consumers received outreach in English.
Experimental Design Details
Randomization Method
We used Stata to carry out the randomization.
Randomization Unit
Random assignment occurs at the household-level.
Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters
Sample size: planned number of observations
9,288 households
Sample size (or number of clusters) by treatment arms
6,511 households in the treatment arm and 2,777 households in the control arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Institutional Review Boards (IRBs)

IRB Name
California Health and Human Services Agency Committee for the Protection of Human Subjects
IRB Approval Date
IRB Approval Number


Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information


Is the intervention completed?
Data Collection Complete
Data Publication

Data Publication

Is public data available?

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials