Increasing the Efficacy & Diffusion of Covid-19 Messaging for Vaccination

Last registered on January 26, 2022

Pre-Trial

Trial Information

General Information

Title
Increasing the Efficacy & Diffusion of Covid-19 Messaging for Vaccination
RCT ID
AEARCTR-0008711
Initial registration date
December 17, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 19, 2021, 1:19 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
January 26, 2022, 2:15 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
MIT

Other Primary Investigator(s)

PI Affiliation
Harvard University
PI Affiliation
Yale University
PI Affiliation
Johns Hopkins School of Nursing
PI Affiliation
MIT
PI Affiliation
Stanford University
PI Affiliation
MIT
PI Affiliation
HKS

Additional Trial Information

Status
In development
Start date
2021-12-20
End date
2022-07-18
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study will distribute videos of health care professionals encouraging Covid-19 vaccination to a large sample of Facebook users, and will test the most effective ways to maximize diffusion of this vaccine-related content to increase vaccination rates. The study sample will be U.S. states where vaccination rates remained low in fall 2021. The experimental design is an RCT with 4 groups, randomized at the county level: 1) a control group which receives no intervention, 2) a treatment group in which Facebook users receive ads which include videos of health care professionals telling them to get vaccinated, 3) a treatment group in which Facebook users receive ads which include videos of health care professionals encouraging them to help their friends to get vaccinated, and 4) a treatment group in which Facebook users receive ads which include videos of health care professionals encouraging them to get their most influential friends to help their friends get vaccinated. In treatments 3 and 4, participants will have the option to sign up to be a "vaccine ambassador," in which case they will get notifications when the study team posts new vaccine-related content, and will receive reminders about encouraging their friends to be vaccinated. The study team is building a website to host the videos of health care professionals which answer common questions about Covid-19 vaccination. The investigators will measure engagement with vaccine-related content in each treatment group, as well as assess effects on vaccination rates. Other outcomes which may be considered include Covid-19 case and symptoms prevalence, Covid-19 hospitalization or mortality, and outcomes related to movement, travel, and closures.
External Link(s)

Registration Citation

Citation
Alsan, Marcella et al. 2022. "Increasing the Efficacy & Diffusion of Covid-19 Messaging for Vaccination." AEA RCT Registry. January 26. https://doi.org/10.1257/rct.8711
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
We have 3 main treatment groups + control.

Treatment 1: Direct Messaging
Facebook users in the area receive ads which include videos of health professionals telling them to get vaccinated. The health care professionals answer common questions about Covid-19 vaccines (e.g. are they safe / are they free).

Treatment 2: Friends Messaging
Facebook users in the area receive ads which include videos of health professionals encouraging them to help their friends to get vaccinated.

Treatment 3: Gossips Messaging
Facebook users in the area receive ads which include videos of health professionals encouraging them to nominate their most influential friends to help their friends get vaccinated.

Moreover, when individuals arrive at the study webpage, they will be randomized into different versions of the same videos:

Content Variation Treatments: "Fact" vs "Myth" Frame
Some of the health professionals' videos will be framed as responding to a question about the vaccines, while others will be framed as responding to a myth about the vaccine.
Intervention Start Date
2021-12-20
Intervention End Date
2022-01-17

Primary Outcomes

Primary Outcomes (end points)
1. Changes in county-level Covid-19 vaccination
Measured by publicly available county- or zip code-level vaccination data. The study will examine total vaccinations as well as vaccinations by age group. The study will also examine people 1) getting their first dose of vaccine, 2) completing their vaccine series, and 3) getting booster shots.

2. Social shares of vaccine-related content
Number of social media shares of vaccine-related videos and messaging that the study team has created, disaggregated at the county or zip code level.

3. Number of views of vaccine-related content
Number of views of the website and vaccine-related videos, disaggregated at the county or zip code level.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
1. COVID-19 cases
Measured by publicly available county- or zip code-level Covid-19 cases, in some areas also split by vaccinated vs unvaccinated status.

2. COVID-19 symptoms prevalence
Measured by publicly available county- or zip code-level Covid-19 symptoms, in some areas also split by vaccinated vs unvaccinated status.

3. COVID-19 hospitalization and mortality
Measured by data on patients admitted to hospitals or who have been recorded to have died from Covid-19, , in some areas also split by vaccinated vs unvaccinated status.

4. Vaccine hesitancy
Measured by publicly available county- or zip code-level surveys

5. Movement and Travel
Measured using mobile phone location data of Facebook users. Vaccinated people may be more likely to take risks, including travel.

6. School closures
Increased cases could lead to more school closures. The investigators do not expect the study to be well-powered on this outcome.

7. Number of vaccine ambassador volunteers
This will be compared between the "friends" and "gossips" treatment groups. This will include people who start to sign up to be vaccine ambassadors as well people who complete the vaccine ambassador consent process.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The main study will have three treatment arms and a control group. Each of the treatments will be randomized at the county level and will be initiated through a Facebook ad campaign.

Experimental Sample: The experimental sample includes where vaccinated rates remained low in fall 2021.

The following procedures will be used:

Treatment Group 1 ("Direct" messaging): The investigators will conduct a Facebook ad campaign to show content about COVID-19 vaccination to a large number of Facebook users. Facebook will allocate ad credits across users in treatment areas. Like for any Facebook ad, individuals can choose to whether or not to watch the video and can close the ad at any time. If individuals wish, they can also share any of the content with others.

Treatment Group 2 ("Friends" messaging): For this group, the Facebook ad campaign will recruit individuals to become "vaccine ambassadors" to help spread the word about vaccination to their friends, families and communities. Recruitment ads will be disseminated in a similar manner to the content in T1. Individuals will be able to share the ad with others. Individuals interested in learning more can click through a link in the ad to the vaccine ambassador webpage. Participants can consume or share any of the information on the webpage. Interested individuals can also enroll formally as vaccine ambassadors. They can share their contact information with the study to a) be entered into prize lotteries, b) receive reminders to share content with others, c) be contacted in the future for possible follow-up surveys. They will be asked to give their informed consent at this time and will be screened based on whether they live in a treated county and whether they are 18 years or older.

Treatment Group 3 ("Gossips" messaging): This treatment looks the same as Treatment Group 2, except the recruitment ads will differ slightly. If the individual clicks through to learn more about vaccine ambassadors, they will also be reminded to share the invitation with people they know who are often the source of important news or information.

Control: individuals in these counties will receive no intervention and will never be contacted by members of the study team.
Treatment and control status at the county level will be merged with aggregated and de-identified datasets to measure COVID vaccinations, COVID symptoms, and cases, and engagement with the vaccine ambassador content.

The study team will create a series of websites with materials that the vaccine ambassadors can share with friends, family and others in their communities. This same content will be disseminated through sponsored Facebook ads in T1. On the website, the framing of the vaccine-related videos with either a "fact" or "myth" frame will be randomized at the individual level. The purpose is to study which framing is more effective at engaging users and encouraging sharing.
Experimental Design Details
Not available
Randomization Method
Randomization was done using Stata on a computer
Randomization Unit
U.S. county for the main experiment
Individual for the variation in content framing
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
1,397 counties
Sample size: planned number of observations
1,397 observations of county-level data
Sample size (or number of clusters) by treatment arms
468 counties in the control group, 310 counties in T1 ("direct" messaging) treatment, 309 counties in T2 ("friends" messaging), and 310 counties in T3 ("gossips" messaging)

We do not know how many individuals will visit the project website. Randomization into content framing will be done by the website itself with 50% seeing the myth framing and 50% seeing the fact framing.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Massachusetts Institute of Technology
IRB Approval Date
2021-11-17
IRB Approval Number
2106000397
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information