Social Media and Political Persuasion: A Field Experiment

Last registered on February 01, 2020

Pre-Trial

Trial Information

General Information

Title
Social Media and Political Persuasion: A Field Experiment
RCT ID
AEARCTR-0002713
Initial registration date
February 02, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 03, 2018, 9:33 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
February 01, 2020, 6:40 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Yale University

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2018-02-03
End date
2020-03-31
Secondary IDs
H0, D72, L82, L86
Abstract
Social media has increasingly become an important source of news consumption, however, limited evidence exists on its causal effect. I will conduct a natural field experiment to measure the effects of news consumption on political opinions and political behavior. The main outcomes are whether participants are persuaded by news they were randomly exposed to as a result of the intervention and whether being offered content which matches the participant’s ideology increases polarization. The results will be used to compare models explaining how individuals learn when consuming news from an outlet with an ideological slant. Specifically, I will test if consumers are affected by the slant of an outlet and whether consumers suffer from confirmation bias and are persuaded only by news matching their opinion.
External Link(s)

Registration Citation

Citation
Levy, Roee. 2020. "Social Media and Political Persuasion: A Field Experiment." AEA RCT Registry. February 01. https://doi.org/10.1257/rct.2713
Former Citation
Levy, Roee and Roee Levy. 2020. "Social Media and Political Persuasion: A Field Experiment." AEA RCT Registry. February 01. https://www.socialscienceregistry.org/trials/2713/history/61843
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2018-02-03
Intervention End Date
2018-03-01

Primary Outcomes

Primary Outcomes (end points)
Persuasion - Were subject persuaded by news they consumed
Polarization - Did subject become more polarized when consuming news matching their opinion
Primary Outcomes (explanation)
See attached plan

Secondary Outcomes

Secondary Outcomes (end points)
Selective Exposure - Which news pages were selected
Knowledge - Did news exposure increase knowledge
Secondary Outcomes (explanation)
See attached plan

Experimental Design

Experimental Design
See attached plan (registered before the trial, will be published when the trial is complete)
Experimental Design Details
Subjects will be recruited to a baseline survey using Facebook ads. Towards the end of the survey, subjects will be given an option to “like” four Facebook news pages. “Liking” a page on Facebook is similar to subscribing to a (free) newspaper, and once a page is “liked” its content may start appearing in the subjects' feed.
Subjects will be assigned to a conservative, liberal or control treatment. In the conservative treatment four pages of conservative-leaning media outlets will be offered (e.g. Fox News); and in the liberal treatment, the four pages offered will be of liberal-leaning media outlets (e.g. the New York Times). In the control treatment, subjects will not be offered any pages. This is an encouragement design and in each treatment, subjects will choose whether to “like” each of the pages.
Approximately 6-8 weeks after the baseline survey, subjects will be invited to a follow-up survey to measure any self-reported changes in their opinions and political knowledge. In addition, In addition, the subjects' Facebook posts will be used to measure the effect of the intervention on their behavior.
Randomization Method
Each participant will be assigned to a control, conservative or liberal treatment, based on a randomized block design by participants’ baseline ideology. Since the sample frame arrives over time, the entire sample will not be stratified in advance but will be split gradually into groups, according to subjects' ideology.

At the beginning of the survey, respondents will be asked where they position themselves ideologically on a 7-point ideological scale from very liberal to very conservative, with an additional option of “I haven't thought about it much.” Subjects will be assigned to a treatment based on where they position themselves on the scale and when they answered the question. Each block (stratum) will be composed of three sequential subjects who chose the exact same answer among the eight options in the ideology scale survey question. The first three subjects who gave the same answer in the ideology question will be randomly split to the conservative, liberal and control groups. The next three individuals with the same ideology will be also be split into the three groups, and the process will continue with each additional respondent. In practice, since subjects arrive gradually, the first subject in each stratum will be randomly assigned to one of the three treatments, the second subject will be randomly assigned to one of the two remaining treatments and the third subject will be assigned to the final remaining treatment. If there are any technical issues in determining the strata for a particular subject, the subject will be randomly assigned to one of the three treatments.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
NA
Sample size: planned number of observations
Approximately 12,000 individuals will take part in the experiment. Most of the analysis will focus on the subset of subjects who complete the follow-up survey. I predict that approximately 60% of subjects who complete the baseline survey will also complete the follow-up survey. The precise number of participants depends on budget constraints. Since subjects are recruited using ads, and the costs of ads is somewhat volatile, the final number can be smaller or greater, and will probably in the range of 10,000-15,000
Sample size (or number of clusters) by treatment arms
1/3 - control group
1/3 - conservative treatment
1/3 - liberal treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Yale IRB
IRB Approval Date
2017-08-14
IRB Approval Number
2000021422
Analysis Plan

Analysis Plan Documents

“Social Media and Political Persuasion: A Field Experiment” - Pre-analysis Plan, Feb 4 (fixed page names)

MD5: b309cbc818add23255fe99114eabccee

SHA1: 4dd59812c1c8725912b4dc6fbdeb215ab6e941bf

Uploaded At: February 04, 2018

“Social Media and Political Persuasion: A Field Experiment” - Updated Pre-analysis Plan Before Follow-Up Survey

MD5: 62431d8239b582fa440e9991de4db239

SHA1: ae1098cff6b86d574e182ef796b4e1506b766365

Uploaded At: April 18, 2018

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials