Fake news: Overprecision and detection.

Last registered on January 17, 2025

Pre-Trial

Trial Information

General Information

Title
Fake news: Overprecision and detection.
RCT ID
AEARCTR-0014786
Initial registration date
January 15, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 17, 2025, 7:12 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Universitat de Barcelona (UB) i Institut d'Economia de Barcelona (IEB)

Other Primary Investigator(s)

PI Affiliation
Technical University in Berlin
PI Affiliation
Universitat de Barcelona and IEB

Additional Trial Information

Status
In development
Start date
2024-11-17
End date
2025-03-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study investigates how individuals identify fake news in four European Union countries: Germany, Ireland, Poland, and Spain. The study consists of a two-wave survey, with all participants invited to participate in both waves. In the first wave, a representative sample of each country will be presented with 13 news headlines and asked to classify each as either true or fake. Participants are then randomly assigned to either the treatment group (two-thirds of the participants) or the control group (one-third). Participants in the treatment group receive feedback indicating whether they correctly classified more or less than half of the headlines, while those in the control group receive no feedback. All participants are then asked to classify a new set of 13 news headlines. Three weeks later, during the second wave, all participants are re-contacted and asked to classify another set of 13 news headlines as either true or fake. The primary goal of the intervention is to assess how feedback influences participants' ability to identify fake news. Additionally, the study aims to examine: (i) the long-term effects of the intervention, measured in the second wave conducted three weeks after the first; and (ii) the role of overprecision in participants’ ability to distinguish fake news.
External Link(s)

Registration Citation

Citation
Bosch-Rosa, Ciril, Daniel Montolio and Guillem RIAMBAU. 2025. "Fake news: Overprecision and detection.." AEA RCT Registry. January 17. https://doi.org/10.1257/rct.14786-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The intervention consists on providing information to participants on how well they perform on assessing the veracity of different news headlines.
Intervention Start Date
2024-11-17
Intervention End Date
2024-12-24

Primary Outcomes

Primary Outcomes (end points)
1. Percentage of correctly assessed headlines
2. Confidence level for each headline (i.e., the probability assigned that a headline is true)
3. Overprecision measure
4. Whether the headline would be shared in social media

All pre and post information shock and across waves.
Primary Outcomes (explanation)

Overprecision is measured following the Subjective Error Method (Bosch-Rosa et al., 2024). This is calculated by taking the differences between the absolute expected error and the actual error for each picture and then aggregating across all pictures in a block. E.g., suppose the average age for a given picture is 45. If the participant guesses the average age is 53 and expects her guess to be 5 years away from the correct answer, then her measure of overprecision for that picture is [(53-45) – 5] = 8-5 = 3 (she is overprecise for this picture).

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The survey will be carried out simultaneously across all four countries. We will have a representative sample of 1,200 respondents in Spain, Germany, and Poland. For Ireland, we will collect 800. All participants who finish wave 1 will be re-contacted for wave 2 exactly 3 weeks after completing wave 1.

Allocation to treatment or control is random. In treatment, participants will be told whether they correctly classified or not more than half of the news in control there is no information on performance.
Experimental Design Details
Not available
Randomization Method
Randomization will be done by the company carrying the survey, YOUGOV (link here), by means of a computational algorithm.
Randomization Unit
Randomization at the individual level within each country.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
There are 4 countries, but no clusters.
Sample size: planned number of observations
6,000 (1,600 per country with 1,200 in Ireland) for the first wave
Sample size (or number of clusters) by treatment arms
6,000 (1,600 per country with 1,200 in Ireland) for the first wave. These are divided 2/3 into treatment (positive or negative signal) and 1/3 control.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
(all computations using STATAMP 16) For cross-country average effects: power twoproportions 0.5 0.45, n1(2933) n2(1467) Study parameters: alpha = 0.0500 N = 4,400 N1 = 2,933 N2 = 1,467 N2/N1 = 0.5002 delta = -0.0500 (difference) p1 = 0.5000 p2 = 0.4500 Estimated power: 0.8794
IRB

Institutional Review Boards (IRBs)

IRB Name
Comissió de Bioètica de la UB
IRB Approval Date
2023-11-16
IRB Approval Number
N/A
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information