|
Field
Trial Start Date
|
Before
September 20, 2024
|
After
October 16, 2024
|
|
Field
Last Published
|
Before
September 19, 2024 10:18 PM
|
After
October 14, 2024 04:54 PM
|
|
Field
Intervention Start Date
|
Before
September 26, 2024
|
After
October 16, 2024
|
|
Field
Primary Outcomes (End Points)
|
Before
The key outcome of interest is the quality of the insurance plan recommendation from an insurance producer (or insurance agent). Our primary outcome evaluates the quality of an agent's recommendation against what an expert would recommend. We are primarily interested in the heterogeneity in recommendations and response rates by insurance agent characteristics based on publicly available data and randomly assigned consumer characteristics. Insurance agent characteristics we focus on include number of appointments, firm affiliation, number of states licensed in, tenure, and advertising presence. We will also use text analysis from recordings to study variations in the language used by caller/consumer and insurance producer characteristics.
|
After
The key outcome of interest is the quality of the insurance plan recommendation from an insurance producer (or insurance agent). Our primary outcome evaluates the quality of an agent's recommendation against what a fully-informed expert would recommend. We are primarily interested in the heterogeneity in recommendations and response rates by insurance agent characteristics based on publicly available data and randomly assigned consumer characteristics such as gender and age. Finally, we test whether randomly assigning consumers to signal that they are soliciting and comparing multiple recommendations improves the quality of agent recommendations.
|
|
Field
Primary Outcomes (Explanation)
|
Before
We measure recommendation quality as whether or not the agent recommends Medicare Advantage and the average plan price of the recommendation.
|
After
Insurance agent characteristics we focus on include number of appointments, firm affiliation, number of states licensed in, tenure, and advertising presence. We will also use text analysis from recordings to study variations in the language used by caller/consumer and insurance producer characteristics. We measure recommendation quality as whether or not the agent recommends Medicare Advantage, the average plan price of the recommendation, and the agent’s accuracy when answering factual questions.
|
|
Field
Experimental Design (Public)
|
Before
This study seeks to understand the role of intermediaries in the Medicare market. We investigate how insurance plan recommendations are correlated with consumer and agent characteristics. The key outcome of interest is the quality of an insurance plan recommendation from an insurance producer (or insurance agent). We are also interested in the heterogeneity in recommendations by insurance agent characteristics based on publicly available data.
|
After
We will conduct a correspondence study that randomly assigns caller and client characteristics to a randomly selected group of insurance agents. To identify insurance agents, we will assemble a list of the most common insurance agents that appear from Google and other public recommendation lists (including government databases, industry affiliation lists, social media websites, and business review and consumer recommendation websites). We will conduct these searches from multiple IP addresses. We will also use text analysis from recordings to study variations in the language used by caller/consumer and insurance producer characteristics.
|
|
Field
Randomization Unit
|
Before
Consumers will be randomly assigned to insurance agents (and therefore insurance agent characteristics) by a computer. Variation in consumer characteristics such as age will also be randomly assigned.
|
After
Consumers will be randomly assigned to insurance agents (and therefore insurance agent characteristics) by a computer at the individual level. Variation in competition signaling and consumer characteristics such as gender and age will also be randomly assigned.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
We will aim to have a 50/50 balance of the main agent characteristics of interest, but we do not expect the sample to be perfectly balanced due to the expected differences in response rates among agents. The randomization does not ensure exact balance by insurance agent characteristics such as number of states licensed in, number of appointments, etc. We will also aim to have a balanced sample of consumer characteristics such as gender.
|
After
We will aim to have a 50/50 balance of the main agent and consumer characteristics of interest (e.g. the gender and age of the consumers) and whether or not consumers signal competition. We do not expect the sample to be perfectly balanced due to anticipated differences in response rates among agents and because the randomization does not ensure exact balance by insurance agent characteristics such as number of states licensed in, number of appointments, etc.
|
|
Field
Intervention (Hidden)
|
Before
We will conduct a correspondence study that randomly assigns caller/client characteristics to a randomly selected group of insurance producers (or insurance agents). We will study variation in recommendation quality across insurance agents to determine which agents licensure, demographic, and geographical characteristics are correlated with better or worse recommendations. We plan to focus on insurance agent characteristics from publicly available data.
|
After
We will conduct a correspondence study that randomly assigns caller/client characteristics to a randomly selected group of insurance producers (or insurance agents). We will study variation in recommendation quality across insurance agents to determine which agent licensure, demographic, and geographic characteristics are correlated with better or worse recommendations. We plan to focus on insurance agent characteristics from publicly available data. Finally, we also test whether randomly assigning consumers to signal that they are soliciting and comparing multiple recommendations improves the quality of agent recommendations.
|
|
Field
Secondary Outcomes (End Points)
|
Before
We will also look the the number of incorrect/misleading statements an agent makes as a measure of recommendation quality. Our primary outcome looks at the quality of the agent's recommendation against an expert benchmark, but we also use consumer choice, other agent recommendations, and random choice as benchmarks. We will also look at the impact of state-level differences in the regulatory and market environment.
|
After
We will also look at the number of incorrect/misleading statements an agent makes as a measure of recommendation quality. Our primary outcome looks at the quality of the agent's recommendation against an expert benchmark, but we also use consumer choice, other agent recommendations, and random choice as benchmarks. We will also look at the impact of state-level differences in the regulatory and market environment.
|