Social Media and Political Persuasion: A Field Experiment

Last registered on February 01, 2020

View Trial History

Pre-Trial

Trial Information

General Information

Title

Social Media and Political Persuasion: A Field Experiment

RCT ID

AEARCTR-0002713

Initial registration date

February 02, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

February 03, 2018, 9:33 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

February 01, 2020, 6:40 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

United States of America

Region

Primary Investigator

Name

Roee Levy

Affiliation

Yale University

Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status

In development

Start date

2018-02-03

End date

2020-03-31

Keywords

Electoral, Governance

Additional Keywords

persuasion, confirmation bias, media, polarizatoin

JEL code(s)

D72, L82, L86, O33

Secondary IDs

H0, D72, L82, L86

Abstract

Social media has increasingly become an important source of news consumption, however, limited evidence exists on its causal effect. I will conduct a natural field experiment to measure the effects of news consumption on political opinions and political behavior. The main outcomes are whether participants are persuaded by news they were randomly exposed to as a result of the intervention and whether being offered content which matches the participant’s ideology increases polarization. The results will be used to compare models explaining how individuals learn when consuming news from an outlet with an ideological slant. Specifically, I will test if consumers are affected by the slant of an outlet and whether consumers suffer from confirmation bias and are persuaded only by news matching their opinion.

External Link(s)

Registration Citation

Citation

Levy, Roee. 2020. "Social Media and Political Persuasion: A Field Experiment." AEA RCT Registry. February 01. https://doi.org/10.1257/rct.2713-4.4

Former Citation

Levy, Roee. 2020. "Social Media and Political Persuasion: A Field Experiment." AEA RCT Registry. February 01. https://www.socialscienceregistry.org/trials/2713/history/205598

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

Intervention (Hidden)

Intervention Start Date

2018-02-03

Intervention End Date

2018-03-01

Primary Outcomes

Primary Outcomes (end points)

Persuasion - Were subject persuaded by news they consumed
Polarization - Did subject become more polarized when consuming news matching their opinion

Primary Outcomes (explanation)

See attached plan

Secondary Outcomes

Secondary Outcomes (end points)

Selective Exposure - Which news pages were selected
Knowledge - Did news exposure increase knowledge

Secondary Outcomes (explanation)

See attached plan

Experimental Design

See attached plan (registered before the trial, will be published when the trial is complete)

Experimental Design Details

Subjects will be recruited to a baseline survey using Facebook ads. Towards the end of the survey, subjects will be given an option to “like” four Facebook news pages. “Liking” a page on Facebook is similar to subscribing to a (free) newspaper, and once a page is “liked” its content may start appearing in the subjects' feed.
Subjects will be assigned to a conservative, liberal or control treatment. In the conservative treatment four pages of conservative-leaning media outlets will be offered (e.g. Fox News); and in the liberal treatment, the four pages offered will be of liberal-leaning media outlets (e.g. the New York Times). In the control treatment, subjects will not be offered any pages. This is an encouragement design and in each treatment, subjects will choose whether to “like” each of the pages.
Approximately 6-8 weeks after the baseline survey, subjects will be invited to a follow-up survey to measure any self-reported changes in their opinions and political knowledge. In addition, In addition, the subjects' Facebook posts will be used to measure the effect of the intervention on their behavior.

Randomization Method

Each participant will be assigned to a control, conservative or liberal treatment, based on a randomized block design by participants’ baseline ideology. Since the sample frame arrives over time, the entire sample will not be stratified in advance but will be split gradually into groups, according to subjects' ideology.

At the beginning of the survey, respondents will be asked where they position themselves ideologically on a 7-point ideological scale from very liberal to very conservative, with an additional option of “I haven't thought about it much.” Subjects will be assigned to a treatment based on where they position themselves on the scale and when they answered the question. Each block (stratum) will be composed of three sequential subjects who chose the exact same answer among the eight options in the ideology scale survey question. The first three subjects who gave the same answer in the ideology question will be randomly split to the conservative, liberal and control groups. The next three individuals with the same ideology will be also be split into the three groups, and the process will continue with each additional respondent. In practice, since subjects arrive gradually, the first subject in each stratum will be randomly assigned to one of the three treatments, the second subject will be randomly assigned to one of the two remaining treatments and the third subject will be assigned to the final remaining treatment. If there are any technical issues in determining the strata for a particular subject, the subject will be randomly assigned to one of the three treatments.

Randomization Unit

Individual

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

Sample size: planned number of observations

Approximately 12,000 individuals will take part in the experiment. Most of the analysis will focus on the subset of subjects who complete the follow-up survey. I predict that approximately 60% of subjects who complete the baseline survey will also complete the follow-up survey. The precise number of participants depends on budget constraints. Since subjects are recruited using ads, and the costs of ads is somewhat volatile, the final number can be smaller or greater, and will probably in the range of 10,000-15,000

Sample size (or number of clusters) by treatment arms

1/3 - control group
1/3 - conservative treatment
1/3 - liberal treatment

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Supporting Documents and Materials

IRB