Till Microtargeting Do Us Part: Measuring the Effects of Political Ads on Facebook

Last registered on November 09, 2020
View Trial History

Pre-Trial

Trial Information

General Information

Title
Till Microtargeting Do Us Part: Measuring the Effects of Political Ads on Facebook
RCT ID
AEARCTR-0006703
Initial registration date
November 05, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 09, 2020, 10:44 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
George Beknazar-Yuzbashev
Affiliation
Columbia University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Mateusz Stalinski
PI Affiliation
The University of Chicago
Contact Investigator

Additional Trial Information

Status
On going
Start date
2020-08-31
End date
2020-11-15
Keywords
Behavior, Electoral
Additional Keywords
JEL code(s)
D9, D8
Secondary IDs
Abstract
Microtargeted political ads create innovative opportunities for influencing voter’s behavior, which many scholars brand as a threat to democracy (Borgesius et al 2018, Ribeiro et al 2019, Witzleb et al 2019). To study the effects of microtargeted ads we conduct a field experiment on Facebook in which we ask participants to change settings on ad preferences to see fewer ads on a particular topic. Individuals are randomly assigned one of the two topics: social issues, elections, and politics (treatment group) and alcohol (control group). Furthermore, a subset of individuals installed a browser extension that collects data on ads that they receive on their Facebook feed.

Microtargeted political advertising can affect individual decision-making in a variety of ways. To capture its broad impact, we collect a rich set of outcomes including voter turnout (using voter files), voting preferences, policy views, beliefs about views of others, as well as measures of affective polarization. To inform the ongoing debate whether political ads on social media should be more tightly regulated, we also measure ads’ effect on misinformation.

External Link(s)

Registration Citation

Citation
Beknazar-Yuzbashev, George and Mateusz Stalinski. 2020. "Till Microtargeting Do Us Part: Measuring the Effects of Political Ads on Facebook." AEA RCT Registry. November 09. https://doi.org/10.1257/rct.6703-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We ask participants to change settings on ad preferences to see fewer ads on a particular topic. Individuals are randomly assigned one of the two topics: social issues, elections, and politics (treatment group) and alcohol (control group). The intervention lasted for 2+ weeks in the run up to the elections. We start collecting outcomes on 5th November 2020.
Intervention Start Date
2020-08-31
Intervention End Date
2020-11-15

Primary Outcomes

Primary Outcomes (end points)
- whether the user voted in the 2020 election;
- presidential ticket voted for in 2020;
- views on policy issues;
- affective polarization as measured by the amount donated to a supporter of the opposing party (dictator game);
- affective polarization as measured by the feelings thermometer;
- prediction about how other respondents (from the opposing party) answered the questions about views on policy issues;
- misinformation levels as measured as agreement/disagreement with several statements.

We will explore heterogeneity in treatment effects in three dimensions:
- partisanship (Democrat/Republican/Independent) and strength of partisanship;
- the number of ads shown by different campaigns;
- geographic (including analysis of offline advertising in different areas).



Primary Outcomes (explanation)
Policy issues are the following:
- We should increase the number of Supreme Court justices (D).
- Response of the Federal Government to the Covid-19 pandemic was adequate (R).
- We need to reduce police funding (D).
- Access to abortion during the first trimester should be restricted (R).
- Free health care for everyone is a good policy (D).

In particular, we are interested in whether one's views and predictions about views' of the out-group become more "extreme" (in the partisan direction as indicated by symbols R/D above). We will investigate heterogenous effects of ads on views and predictions depnding on the composition of ads shown to pariticpants (the number of ads shown by different campaigns).

Statements (misinformation) are the following:
- Joe Biden and Kamala Harris support defunding the police.
- Joe Biden lacks stamina and has poor health.
- Raising taxes is a part of Joe Biden's program.
- Joe Biden wants to increase the number of Supreme Court justices.
- Joe Biden supports policies advocated by the radical left.

For affective polarization outcomes and predictions about answers of other participants, we assign the "opposing side" in the following way. If a person identifies as Republican or Democrat, they are assigned the other main party. If they identify as an Independent, we ask them who are they closer to, and they are assign the other group.


Secondary Outcomes

Secondary Outcomes (end points)
- words used to describe supporters of the in- and out-party;
- which issue the user considers the most important for the election;
- making political donations;
- party for the Congress voted for in 2020.
Secondary Outcomes (explanation)
Words are the following:
- open-minded
- generous
- honest
- selfish
- intelligent

Experimental Design

Experimental Design
The study has the following steps:
1. Recruit people on Facebook through Facebook Ads;
2. We incentivize participants to change Facebook ad settings related to see fewer ads on social issues, elections, and politics (treatment) or alcohol (control). Topics are randomly assigned.
3.After 2+ weeks, we conduct the follow up with a survey eliciting outcomes. We will also look at voter file data to elecit whether a person voted in the elections.

Experimental Design Details
Randomization Method
Treatment randomization was done by Qualtrics.

To determine the winner of the lottery we will use the following procedure. On November 6 we will go to https://www.marketwatch.com/investing/index/djia. We will use the last digits of the averages as seen at 7 pm EST to fill a random number of the same length as the number of participants minus one digit by digit in the following order:
Dow (last digit), S&P (last digit), Nasdaq (last digit), GlobalDow (last digit), Gold (last digit), Oil (last digit), Dow (second to last digit), S&P (second to last digit), Nasdaq (second to last digit), GlobalDow (second to last digit), Gold (second to last digit), Oil (second to last digit).

If a particular digit brings the randomly generated number above the number of participants, we skip that digit and move on to the next one. For example, if there are 1000 respondents and we get 384 as the first digits and the last digit of GlobalDow on that day is 3, we skip it and move to the last digit of Gold. If it is also not equal to 0 or 1, we move onto Dow's second to last digit, and so on. If we exhaust the sequence and fail to finish the random number we will continue with the next day's sequence (keeping the numbers that we already generated).

For the avoidance of doubt, the "first" participant in this scheme is mapped to 0, the second -- to 1, and so on.
Randomization Unit
Individual.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1
Sample size: planned number of observations
Approximately 1000 users.
Sample size (or number of clusters) by treatment arms
Approximately 500 users.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
The University of Chicago IRB
IRB Approval Date
2020-08-31
IRB Approval Number
IRB20-1438
IRB Name
Columbia University IRB
IRB Approval Date
2020-08-28
IRB Approval Number
IRB-AAAT2171
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials