|
Field
Trial Status
|
Before
on_going
|
After
completed
|
|
Field
Abstract
|
Before
In prior work, we have found that video messages delivered by doctors increase knowledge of COVID-19, and in some cases, lead to viewers taking actions consistent with the messaging. In a scale-up phase, we plan to use Facebook ads to show a 15 second video clip recorded by 6 MGH, Harvard and Lynn Community health center doctors to approximately 40,000,000 Facebook users. The ads will be shown between November 14-29 and focus on staying safe – limiting travel, social distancing and mask-wearing. We are randomizing exposure to the ad campaign at the ZIP code and county level to study whether the videos change mobility and Thanksgiving holiday travel and whether there are any detectable impacts on the spread of COVID-19. Using information about Facebook social network connections and mobility patterns between counties, we also plan to measure whether exposure to the ad campaign spilled over along the network.
|
After
In prior work, we have found that video messages delivered by doctors increase knowledge of COVID-19, and in some cases, lead to viewers taking actions consistent with the messaging. In a scale-up phase, we plan to use Facebook ads to show a 15 second video clip recorded by 6 MGH, Harvard and Lynn Community health center doctors to approximately 40,000,000 Facebook users. The ads will be shown between November 14-29 and focus on staying safe – limiting travel, social distancing and mask-wearing. We are randomizing exposure to the ad campaign at the ZIP code and county level to study whether the videos change mobility and Thanksgiving holiday travel and whether there are any detectable impacts on the spread of COVID-19. Using information about Facebook social network connections and mobility patterns between counties, we also plan to measure whether exposure to the ad campaign spilled over along the network.
Following the completion of the Thanksgiving intervention, we implemented a follow-on Christmas intervention with the same two-stage research design, re-randomized in the same 13 states. Videos recorded by nurses and physicians encouraging people to stay home for Christmas were again pushed as ads to Facebook users.
|
|
Field
Last Published
|
Before
November 30, 2020 11:37 AM
|
After
June 29, 2021 11:14 AM
|
|
Field
Study Withdrawn
|
Before
|
After
No
|
|
Field
Intervention Completion Date
|
Before
|
After
December 31, 2020
|
|
Field
Data Collection Complete
|
Before
|
After
Yes
|
|
Field
Final Sample Size: Number of Clusters (Unit of Randomization)
|
Before
|
After
We conducted the Thanksgiving intervention as anticipated. Our final sample included 6998 zip codes in 820 counties. We restricted the sample to only the zip codes that reported COVID-19 infections at baseline. We also ran an almost-identical experiment for the Christmas holiday, using physician- and nurse-recorded messages that pertained to Christmas travel. We re-randomized using the same sample. However, prior to the Christmas campaign, 60 fully rural counties in the top tercile of votes for Donald Trump in the 2020 election were removed from the study. This was done out of caution and to avoid adverse effects. The research team was concerned that the messaging campaign might have adverse unintended effects in very rural, heavily Republican-leaning counties given the growing polarization in December. The remaining sample had 767 counties. We included in the campaign all zip codes in the intervention in the selected counties (even if they could not be matched to COVID-19 infection data). For the COVID-19 outcomes, we have a final sample of 6716 zip codes.
|
|
Field
Was attrition correlated with treatment status?
|
Before
|
After
No
|
|
Field
Final Sample Size: Total Number of Observations
|
Before
|
After
Thanksgiving: 6998 zip codes, 820 counties.
Christmas: 6716 zip codes, 767 counties.
Recall, Facebook Movement Range mobility data is available at the county level, while COVID-19 data is at the zip code level.
|
|
Field
Final Sample Size (or Number of Clusters) by Treatment Arms
|
Before
|
After
See the Consort diagram (Figure 1) in the Supplemental Materials for a detailed break-down of sample sizes by treatment and campaign.
|
|
Field
Data Collection Completion Date
|
Before
|
After
February 11, 2021
|
|
Field
Intervention (Public)
|
Before
Facebook users will be exposed to an ad campaign featuring a set of videos about Thanksgiving travel. 6 doctors from MGH, Harvard and Lynn Community health center have each recorded one short message (15 seconds) using an identical script: “This Thanksgiving, the best way to show your love is to stay home. If you do visit, wear a mask at all times. I'm Dr. XX from XX, and I'm urging you: don't risk spreading COVID. Stay safe, stay home.”
The video messages will be posted to a project Facebook page titled "The Doctors for Coronavirus Prevention Project". Users will see the ads and the name of the Facebook group associated with the ads on their feeds. Watching the videos in the ad posts is completely optional for Facebook users.
Within treated geographies, Facebook's algorithm will be used to place the ads. We expect each treated individual to see the ads approximately 3 times over the course of the ad campaign.
|
After
Facebook users will be exposed to an ad campaign featuring a set of videos about Thanksgiving travel. 6 doctors from MGH, Harvard and Lynn Community health center have each recorded one short message (15 seconds) using an identical script: “This Thanksgiving, the best way to show your love is to stay home. If you do visit, wear a mask at all times. I'm Dr. XX from XX, and I'm urging you: don't risk spreading COVID. Stay safe, stay home.”
The video messages will be posted to a project Facebook page titled "The Doctors for Coronavirus Prevention Project". Users will see the ads and the name of the Facebook group associated with the ads on their feeds. Watching the videos in the ad posts is completely optional for Facebook users.
Within treated geographies, Facebook's algorithm will be used to place the ads. We expect each treated individual to see the ads approximately 3 times over the course of the ad campaign.
The same intervention protocols were followed again prior to the Christmas holiday with a new set of messages recorded by doctors and nurses.
|
|
Field
Randomization Unit
|
Before
In Phase 1, we use a 2-stage randomization at the county and zip code levels.
In Phase 2, randomization is at the county level. Ad targeting is based at the zip code level. Zip codes are assigned to the county with which it shares the largest area.
Individual Facebook users are targeted by Facebook's algorithms.
|
After
In Phase 1, we use a 2-stage randomization at the county and zip code levels.
In Phase 2, randomization is at the county level. Ad targeting is based at the zip code level. Zip codes are assigned to the county with which it shares the largest area.
Christmas: same as Phase I
Individual Facebook users are targeted by Facebook's algorithms.
|
|
Field
Planned Number of Clusters
|
Before
Phase I: 820 counties, 6998 zip codes in 13 states.
Phase 2: 1063 counties in 15 additional states.
|
After
Phase I: 820 counties, 6998 zip codes in 13 states.
Phase 2: 1063 counties in 15 additional states.
Christmas: 767 counties, 6716 zip codes (For this wave, we excluded rural counties with high Trump vote shares in the 2020 election out of caution due to the polarization about COVID, as we didn't want to create backlash. We also included zip codes even if they did not report COVID-19 cases.)
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
Phase I: 50% of the 820 counties are treated. 50% of the 6998 zip codes are treated.
Phase 2: 80% of the 1063 counties are treated.
|
After
Phase I: 50% of the 820 counties are treated. 50% of the 6998 zip codes are treated.
Phase 2: 80% of the 1063 counties are treated.
Christmas: same as Phase I (50% of counties and zips in the sample frame)
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
|
After
We determined that a sample of 820 counties would provide 80% power to detect effect sizes of 0.2 standard deviations for county-level outcomes, comparing intervention (H) vs. control (L), using only average county-level data. For outcomes with zip code level data, using intra-class correlations of 0.2 (0.475) assuming clusters of equal size, a sample of 6,998 zip codes would provide 80% power to detect effect sizes of 0.057 (0.072) standard deviations (using zip code level data).
|
|
Field
Keyword(s)
|
Before
Behavior, Health
|
After
Behavior, Health
|