x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Social Media and Political Persuasion: A Field Experiment
Last registered on February 01, 2020

Pre-Trial

Trial Information
General Information
Title
Social Media and Political Persuasion: A Field Experiment
RCT ID
AEARCTR-0002713
Initial registration date
February 02, 2018
Last updated
February 01, 2020 6:40 PM EST
Location(s)
Region
Primary Investigator
Affiliation
Yale University
Other Primary Investigator(s)
Additional Trial Information
Status
In development
Start date
2018-02-03
End date
2020-03-31
Secondary IDs
H0, D72, L82, L86
Abstract
Social media has increasingly become an important source of news consumption, however, limited evidence exists on its causal effect. I will conduct a natural field experiment to measure the effects of news consumption on political opinions and political behavior. The main outcomes are whether participants are persuaded by news they were randomly exposed to as a result of the intervention and whether being offered content which matches the participant’s ideology increases polarization. The results will be used to compare models explaining how individuals learn when consuming news from an outlet with an ideological slant. Specifically, I will test if consumers are affected by the slant of an outlet and whether consumers suffer from confirmation bias and are persuaded only by news matching their opinion.
External Link(s)
Registration Citation
Citation
Levy, Roee. 2020. "Social Media and Political Persuasion: A Field Experiment." AEA RCT Registry. February 01. https://doi.org/10.1257/rct.2713-4.3.
Former Citation
Levy, Roee. 2020. "Social Media and Political Persuasion: A Field Experiment." AEA RCT Registry. February 01. http://www.socialscienceregistry.org/trials/2713/history/61843.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2018-02-03
Intervention End Date
2018-03-01
Primary Outcomes
Primary Outcomes (end points)
Persuasion - Were subject persuaded by news they consumed
Polarization - Did subject become more polarized when consuming news matching their opinion
Primary Outcomes (explanation)
See attached plan
Secondary Outcomes
Secondary Outcomes (end points)
Selective Exposure - Which news pages were selected
Knowledge - Did news exposure increase knowledge
Secondary Outcomes (explanation)
See attached plan
Experimental Design
Experimental Design
See attached plan (registered before the trial, will be published when the trial is complete)
Experimental Design Details
Subjects will be recruited to a baseline survey using Facebook ads. Towards the end of the survey, subjects will be given an option to “like” four Facebook news pages. “Liking” a page on Facebook is similar to subscribing to a (free) newspaper, and once a page is “liked” its content may start appearing in the subjects' feed. Subjects will be assigned to a conservative, liberal or control treatment. In the conservative treatment four pages of conservative-leaning media outlets will be offered (e.g. Fox News); and in the liberal treatment, the four pages offered will be of liberal-leaning media outlets (e.g. the New York Times). In the control treatment, subjects will not be offered any pages. This is an encouragement design and in each treatment, subjects will choose whether to “like” each of the pages. Approximately 6-8 weeks after the baseline survey, subjects will be invited to a follow-up survey to measure any self-reported changes in their opinions and political knowledge. In addition, In addition, the subjects' Facebook posts will be used to measure the effect of the intervention on their behavior.
Randomization Method
Each participant will be assigned to a control, conservative or liberal treatment, based on a randomized block design by participants’ baseline ideology. Since the sample frame arrives over time, the entire sample will not be stratified in advance but will be split gradually into groups, according to subjects' ideology.

At the beginning of the survey, respondents will be asked where they position themselves ideologically on a 7-point ideological scale from very liberal to very conservative, with an additional option of “I haven't thought about it much.” Subjects will be assigned to a treatment based on where they position themselves on the scale and when they answered the question. Each block (stratum) will be composed of three sequential subjects who chose the exact same answer among the eight options in the ideology scale survey question. The first three subjects who gave the same answer in the ideology question will be randomly split to the conservative, liberal and control groups. The next three individuals with the same ideology will be also be split into the three groups, and the process will continue with each additional respondent. In practice, since subjects arrive gradually, the first subject in each stratum will be randomly assigned to one of the three treatments, the second subject will be randomly assigned to one of the two remaining treatments and the third subject will be assigned to the final remaining treatment. If there are any technical issues in determining the strata for a particular subject, the subject will be randomly assigned to one of the three treatments.
Randomization Unit
Individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
NA
Sample size: planned number of observations
Approximately 12,000 individuals will take part in the experiment. Most of the analysis will focus on the subset of subjects who complete the follow-up survey. I predict that approximately 60% of subjects who complete the baseline survey will also complete the follow-up survey. The precise number of participants depends on budget constraints. Since subjects are recruited using ads, and the costs of ads is somewhat volatile, the final number can be smaller or greater, and will probably in the range of 10,000-15,000
Sample size (or number of clusters) by treatment arms
1/3 - control group
1/3 - conservative treatment
1/3 - liberal treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Yale IRB
IRB Approval Date
2017-08-14
IRB Approval Number
2000021422
Analysis Plan
Analysis Plan Documents
“Social Media and Political Persuasion: A Field Experiment” - Pre-analysis Plan, Feb 4 (fixed page names)

MD5: b309cbc818add23255fe99114eabccee

SHA1: 4dd59812c1c8725912b4dc6fbdeb215ab6e941bf

Uploaded At: February 04, 2018

“Social Media and Political Persuasion: A Field Experiment” - Updated Pre-analysis Plan Before Follow-Up Survey

MD5: 62431d8239b582fa440e9991de4db239

SHA1: ae1098cff6b86d574e182ef796b4e1506b766365

Uploaded At: April 18, 2018

Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS