|
Field
Abstract
|
Before
This study seeks to understand the role of intermediaries in the Medicare market. Specifically, we investigate how quality of insurance advice is correlated with consumer and agent characteristics.
|
After
This study seeks to understand the role of intermediaries in the Medicare market. Specifically, we investigate how quality of insurance advice is correlated with consumer and agent characteristics.
|
|
Field
Trial End Date
|
Before
March 31, 2026
|
After
October 15, 2026
|
|
Field
Last Published
|
Before
October 14, 2024 04:54 PM
|
After
June 18, 2025 07:14 PM
|
|
Field
Intervention End Date
|
Before
March 31, 2026
|
After
October 15, 2026
|
|
Field
Primary Outcomes (End Points)
|
Before
The key outcome of interest is the quality of the insurance plan recommendation from an insurance producer (or insurance agent). Our primary outcome evaluates the quality of an agent's recommendation against what a fully-informed expert would recommend. We are primarily interested in the heterogeneity in recommendations and response rates by insurance agent characteristics based on publicly available data and randomly assigned consumer characteristics such as gender and age. Finally, we test whether randomly assigning consumers to signal that they are soliciting and comparing multiple recommendations improves the quality of agent recommendations.
|
After
The key outcome of interest is the quality of the Medicare plan recommendations from the insurance producer (or insurance agent). We will examine three primary outcomes: i) whether the agent recommends Medicare Advantage (MA), ii) the difference between the recommended Medicare Supplement (Medigap) plan premium and the lowest Medigap premium available to the consumer, and iii) the difference between the recommended Medigap premium and the lowest Medigap premium offered by insurance companies the agent is currently appointed with. More specifics on this are included in our pre-analysis plan.
|
|
Field
Primary Outcomes (Explanation)
|
Before
Insurance agent characteristics we focus on include number of appointments, firm affiliation, number of states licensed in, tenure, and advertising presence. We will also use text analysis from recordings to study variations in the language used by caller/consumer and insurance producer characteristics. We measure recommendation quality as whether or not the agent recommends Medicare Advantage, the average plan price of the recommendation, and the agent’s accuracy when answering factual questions.
|
After
These are the primary outcomes because they are the most direct measures of recommendation quality for our scenario.
|
|
Field
Experimental Design (Public)
|
Before
We will conduct a correspondence study that randomly assigns caller and client characteristics to a randomly selected group of insurance agents. To identify insurance agents, we will assemble a list of the most common insurance agents that appear from Google and other public recommendation lists (including government databases, industry affiliation lists, social media websites, and business review and consumer recommendation websites). We will conduct these searches from multiple IP addresses. We will also use text analysis from recordings to study variations in the language used by caller/consumer and insurance producer characteristics.
|
After
We will conduct a correspondence study that randomly assigns seniors to call insurance agents for a plan recommendation. We will evaluate whether recommendation quality varies with caller and agent characteristics. Finally, we evaluate whether advice quality is elastic to a randomly assigned “competition” treatment.
|
|
Field
Randomization Method
|
Before
Consumers will be randomly assigned to insurance producers by a computer.
|
After
Insurance agents are randomly assigned to treatment or control status by a computer. Consumers are randomly assigned by the secret shopper firm.
|
|
Field
Randomization Unit
|
Before
Consumers will be randomly assigned to insurance agents (and therefore insurance agent characteristics) by a computer at the individual level. Variation in competition signaling and consumer characteristics such as gender and age will also be randomly assigned.
|
After
Consumers will be randomly assigned to insurance agents at the individual level. Variation in competition signaling and consumer characteristics such as gender and age will also be randomly assigned.
|
|
Field
Planned Number of Clusters
|
Before
We will collect approximately 400-800 observations, depending on our ability to obtain additional funding and when our current sources of research funding run out. We will exclude incomplete observations, off-topic observations, and responses from individuals who are not licensed insurance producers. (Complete responses from individuals who are not licensed can be reported separately.) We anticipate having funding for a minimum of 400 observations, however there is a chance we will have fewer than 400 observations if response rates are lower than expected or if several observations do not meet our screening criteria.
|
After
With our current research funding, we expect to be able to collect between 450-850 observations. For additional detail, please see our pre-analysis plan.
|
|
Field
Planned Number of Observations
|
Before
We will collect approximately 400-800 observations, depending on our ability to obtain additional funding and when our current sources of research funding run out. We will exclude incomplete observations, off-topic observations, and responses from individuals who are not licensed (complete responses from individuals who are not licensed can be reported separately). We anticipate having funding for a minimum of 400 observations, however there is a chance we will have fewer than 400 observations if response rates are lower than expected or if several observations do not meet our screening criteria. The maximum number of observations will also depend on take-up, screening, and whether we are able to obtain additional funding.
|
After
With our current research funding, we expect to be able to collect between 450-850 observations. For additional detail, please see our pre-analysis plan.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
We will aim to have a 50/50 balance of the main agent and consumer characteristics of interest (e.g. the gender and age of the consumers) and whether or not consumers signal competition. We do not expect the sample to be perfectly balanced due to anticipated differences in response rates among agents and because the randomization does not ensure exact balance by insurance agent characteristics such as number of states licensed in, number of appointments, etc.
|
After
We will aim to have an equal number of observations in our treatment and control arms.
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
|
After
We are powered to detect differences of 10% in the Medicare Advantage recommendation rate and differences of $75 per year in annual Medigap premium recommendations.
|
|
Field
Additional Keyword(s)
|
Before
Medicare, Brokers, Agents, Intermediaries
|
After
Medicare, Health Insurance, Brokers, Agents, Intermediaries
|
|
Field
Intervention (Hidden)
|
Before
We will conduct a correspondence study that randomly assigns caller/client characteristics to a randomly selected group of insurance producers (or insurance agents). We will study variation in recommendation quality across insurance agents to determine which agent licensure, demographic, and geographic characteristics are correlated with better or worse recommendations. We plan to focus on insurance agent characteristics from publicly available data. Finally, we also test whether randomly assigning consumers to signal that they are soliciting and comparing multiple recommendations improves the quality of agent recommendations.
|
After
We will conduct a correspondence study that randomly assigns Medicare-eligible seniors to call insurance producers (or insurance agents) for plan recommendations. We will study variation in recommendation quality across consumer and agent characteristics. We will also test whether randomly assigning consumers to signal that they will solicit and compare multiple recommendations improves recommendation quality.
|
|
Field
Secondary Outcomes (End Points)
|
Before
We will also look at the number of incorrect/misleading statements an agent makes as a measure of recommendation quality. Our primary outcome looks at the quality of the agent's recommendation against an expert benchmark, but we also use consumer choice, other agent recommendations, and random choice as benchmarks. We will also look at the impact of state-level differences in the regulatory and market environment.
|
After
We will look at whether agents use certain sales tactics and accurately explain important insurance concepts to seniors. More specifics on this are included in our pre-analysis plan.
|
|
Field
Secondary Outcomes (Explanation)
|
Before
|
After
These outcomes are secondary because they seek to explain mechanisms or are less direct measures of quality.
|