Motivations to share true and false information online

Last registered on October 07, 2024

Pre-Trial

Trial Information

General Information

Title
Motivations to share true and false information online
RCT ID
AEARCTR-0014074
Initial registration date
September 27, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 07, 2024, 7:04 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Sciences Po

Other Primary Investigator(s)

PI Affiliation
Paris School of Economics
PI Affiliation
London Business School
PI Affiliation
Sciences Po

Additional Trial Information

Status
In development
Start date
2024-10-01
End date
2024-10-23
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study is designed to assess the effect of two alternative policy measures aimed at curbing the dissemination of false news on social media: fact-checking and priming users of social or reputational costs of sharing misinformation. The study will be based on a randomized experiment with actual users of X (formerly Twitter).

The purpose of the study is threefold:
1) Estimate the effect of two policies that have been proposed in the public debate to limit the circulation of fake news, on social media: (1) making reputational or social costs of sharing false news more salient through a nudge and (2) offering fact checking of the content.

2) Using structural estimation, we will estimate parameters of the utility function of shares of news on social media that explain the motivation behind the decision to share news (true or false). In particular, we will estimate how individuals trade off the following 3 motives: partisan persuasion, partisan signaling, and reputation concerns.

3) We will use the results of structural estimation to evaluate the channels through which the policies affect the sharing of true and false news.

External Link(s)

Registration Citation

Citation
Guriev, Sergei et al. 2024. "Motivations to share true and false information online." AEA RCT Registry. October 07. https://doi.org/10.1257/rct.14074-1.0
Experimental Details

Interventions

Intervention(s)
In the study, the participants will be offered a choice to retweet one of 4 tweets (2 true, 2 false, selected according to a well-defined procedure detailed in the pre-analysis plan) or not to retweet. The treatments will emulate two policies to reduce the circulation of fake news, namely, fact checking and priming fake news circulation. The treatments will be implemented after the presentation of the tweets and before the decision to share.

Intervention (Hidden)
Intervention Start Date
2024-10-01
Intervention End Date
2024-10-23

Primary Outcomes

Primary Outcomes (end points)
The primary outcome is whether the participant chose to share one of the tweets we show them and if so, which one of the 4 tweets they chose to share (if any).
Primary Outcomes (explanation)
The outcome is directly reported by the participant on the survey platform

Secondary Outcomes

Secondary Outcomes (end points)
For each participant, we ask them to evaluate the veracity and the partisanship of each tweet
Secondary Outcomes (explanation)
Both measures are incentivized.
Veracity is an evaluation between 0 and 100 of the likelihood that the content of the tweet is true
Partisanship is an evaluation of whether the Tweet favored Democrats or Republicans on a scale from 1 to 5. We ask participants to guess the mean partisanship reported by other participants.

Experimental Design

Experimental Design
The experiment aims to study the drivers of the sharing behavior of news. We will conduct an online survey of about 10000 American voting-age individuals on the CINT online platform, restricting our sample to X users.

The survey will have the following phases:
(a) Pre treatment phase: Collecting socio-demographic characteristics, political preferences, Twitter usage
(b) Presentation of 4 tweets in random order (see below)
(c) Treatment phase
(d) Sharing decision (not share or share one of the 4 tweets)
(e) Evaluation of veracity and partisanship of each tweet.

The 4 tweets will be selected according to the following procedure.The aim is to obtain four lists (true democrat, false democrat, true republican, false republican) covering the same distribution of topics that were central in the electoral campaign.
To do this, we first collect all recent false news fact-checked by major fact checking groups. We note that the false news favoring democrats are (much) fewer than false news favoring republicans. We drop from this list all the false news that are not related to federal politics or 2024 elections. We also drop any false news that are related to false images. (As we focus on the text). We, then, from the remaining false news, select 10 most recent false news favoring democrats that cover different topics. (We start with democrat false tweets because this is the most restricted set of false news.) We then select 10 corresponding false news favoring republicans such that we maintain the same distribution across topics as in the list of false news favoring democrats, trying to select the most similar in form and most recent, e.g., most relevant for the elections. We then reformulate these pieces of news into the form of tweets, if they are not in that form initially. We use ChatGPT to help with reformulation.
We then consider the population of the recent tweets from both liberal and conservative media with political news (defined by keywords as detailed below). Then, from this list, we select10 tweets from conservative and 10 tweets from liberal news media. In both cases we follow the same distribution across topics as in the lists of pro-democrat (and pro-republican) 10 false news.

This procedure results in four lists of 10 Tweets :
1) True tweets from conservative media
2) True tweets from liberal media
3) False tweets favoring Democrats
4) False tweets favoring Republicans

Each participant will be shown (in random order) four tweets, one drawn from each list.

Treatment groups:

T1: Control group: Participants proceed to the sharing decision right after seeing the tweets

T2: Offer fact check: After seeing the tweets, we provide participants with the opportunity to see a fact check of the false tweets, before the sharing decision

T3: Prime fake news circulation: After seeing the tweets and before sharing the decision, we warn participants that fake news circulates on social media

Participants will be randomly assigned to one of those three treatment groups.


Experimental Design Details
Here is the text from your LaTeX code that you can copy and paste:

**Step-by-step procedure:**

1. **Collect Political Tweets from Mainstream Media:**
- Use Twitter’s API to collect political tweets from liberal news outlets (YahooNews, NYTimes, HuffPost, NBCNews, Politico, CNN, WashingtonPost, USA Today, CBSNews) and conservative news outlets (FoxNews, NYPost, RCPolitics, TheIJR, WashTimes, CNBC, WSJ, Newsmax, Townhallcom). These are considered a pool of true-news tweets to be matched with false-news tweets.
- Define search parameters using specific keywords to ensure the tweets are about politics and issues relevant to the presidential campaign:
- **Political Keywords:** Biden, Trump, president, election, administration, convention, White House, Harris, Vance, Obama, debate, liberal, conservative, Republican, Democrat, Democratic, politics, political, voter, governor, congress, representative, senate, gov., rep., sen., GOP, mayor, Supreme Court.
- **Issue Keywords:** Abortion, federal ban, Roe v. Wade, Dobbs v. Jackson, immigration, border, migrants, gang, legal status, illegal, asylum, healthcare, Obamacare, Medicare Medicaid, economy, tax cuts, unemployment, inflation, trade, foreign policy, NATO, isolationist, Israel, Gaza, Ukraine, Russia, China, Taiwan, Hamas, equality, LGBTQ, woke, transgender, democracy, January 6, Capitol, dictator, electoral fraud, climate change, Paris climate deal, oil, gas, coal, working class, taxes, tariff, Chinese goods, protectionist.
- Use the most recent collected tweets to ensure relevance. Define a time window to get enough matched pairs of tweets for the experiment.

2. **Collect False News from Fact-Checks:**
- Access Snopes and PolitiFact to collect fact-checked statements characterized as false.
- Add fact checks done after the September 10 Harris-Trump debate by the New York Times.
- Use the most recent fact-checked statements to ensure relevance.
- Exclude fact-checks about videos or photos, focusing on verbal statements.
- For each fact check, determine whether it originated from Republicans or Democrats, assuming the source indicates which party the false news favors.
- Transform the fact-checked false statements into the form of tweets, based on versions produced by ChatGPT.

3. **Select the Democrat False Tweets:**
- Select 10 of the most recent fact-checked statements favoring Democrats, deemed appropriate for the experiment.
- Code the topic of each false tweet using broad categories (e.g., abortion, climate change, economy, healthcare, immigration, integrity of candidate/fraud/election manipulation, LGBTQ+/identity/minority discrimination).

4. **Select Corresponding Republican False Tweets:**
- Select by hand 10 false tweets favoring Republicans, maintaining the same distribution across topics as the pro-Democrat false tweets.

5. **Select True News:**
- Form two lists of true news: one from conservative media and one from liberal media, selecting the 1,500 most recent tweets.
- From these 1,500 tweets, select those usable for the experiment (no pictures, videos, or improper statements, focusing on general interest issues).
- From the selected tweets, pick 10 from Republican media and 10 from Democrat media, ensuring the same distribution across topics as the initial list, and picking the most recent ones.

6. **Generate Four Lists:**
- (1) True tweets from conservative media.
- (2) True tweets from liberal media.
- (3) False tweets favoring Democrats.
- (4) False tweets favoring Republicans.

7. **Assign Photos to Tweets:**
- Choose 4 different neutral photos of Harris and Trump.
- For each tweet, create 4 versions, each with one of the photos, so that each participant sees all four photos, randomly assigned to the tweets.

8. **Assign Tweets to Respondents:**
- Each respondent is presented with 4 tweets, each chosen randomly from the four lists above. All tweets will have different photos, with the photos randomly assigned.

Randomization Method
The participants to the survey will be randomly allocated to the different groups by the survey platform CINT
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
na
Sample size: planned number of observations
10000 survey participants
Sample size (or number of clusters) by treatment arms
One-third of the sample in each treatment group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
PSE IRB
IRB Approval Date
2024-06-28
IRB Approval Number
2022-024
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials