How does the communication of scientific uncertainty in times of changing evidence affect people's trust over time?

Last registered on February 02, 2024

Pre-Trial

Trial Information

General Information

Title
How does the communication of scientific uncertainty in times of changing evidence affect people's trust over time?
RCT ID
AEARCTR-0012927
Initial registration date
February 01, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 02, 2024, 4:26 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region
Region

Primary Investigator

Affiliation
Technical University of Munich

Other Primary Investigator(s)

PI Affiliation
Technical University of Munich
PI Affiliation
Technical University of Munich

Additional Trial Information

Status
On going
Start date
2024-01-30
End date
2024-04-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Uncertainty is an inherent part of scientific knowledge and evidence remains “tentative forever” (Popper 1961, p. 280). However, it is unclear how the public responds to transparent communication of scientific uncertainty. Communicating uncertainty becomes even more challenging in the context of emerging crises such as pandemics, economic downturns or climate risks, which often require immediate changes or adaptations in people's behaviour. Scientific findings and related evidence-based recommendations are likely to change during the (early) phases of these crises and may contribute to the public's perception that expert recommendations and claims of facts are not credible or even reflect elite attempts to advance their own agenda. A more thorough understanding of how the communication of scientific uncertainty in the context of crises and changing evidence affects people's trust is thus urgently needed. We will conduct survey experiments across three countries to assess how the explicit communication of scientific uncertainty in the context of public health recommendations in a hypothetical pandemic scenario affects participants' trust in sciences, trust in the communicator of the messages and trust in the behavioral recommendations.
External Link(s)

Registration Citation

Citation
Steinert, Janina, Henrike Sternberg and Nikkil Sudharsanan. 2024. "How does the communication of scientific uncertainty in times of changing evidence affect people's trust over time?." AEA RCT Registry. February 02. https://doi.org/10.1257/rct.12927-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The survey experiment will be set up as follows: First, all participants will receive the following introductory information:

"Suppose that it is the year 2030 and the world is faced with a new global pandemic that poses significant health risks to those infected with the virus. In the early phase of this new pandemic, the World Health Organization issues the following information:"

Participants will then be randomised into one of two health messages, one not acknowledging scientific certainty and one explicitly acknowledging scientific uncertainty.

Group 1 - Not acknowledging uncertainty: "The World Health Organization currently recommends that everyone should only drink bottled water or boil tap water for at least one minute to make it safe to drink. Sticking to these drinking water guidelines can best protect you from infection with the new virus, which transmits primarily through contaminated water."

Group 2 - Acknowledging uncertainty: "The World Health Organization currently recommends that everyone should only drink bottled water or boil tap water for at least one minute to make it safe to drink. Sticking to these drinking water guidelines can best protect you from infection with the new virus, which transmits primarily through contaminated water.\\\\It is possible that this recommendation may change because we are learning about the new pandemic in real time. This is expected since science is all about discovery, testing, learning, and adapting. We will do our best to update you with emerging evidence and what the new information means for you."

In the next step of the experiment, participants first receive the following information:

"Three months after this first recommendation, the spread of the virus is still not contained. At this point, the World Health Organization issues a new recommendation."

Following this information, we will show a second health message but with behavioural recommendations that are different from those outlined in the first health message. Again, the updated information will either not acknowledge or acknowledge scientific uncertainty. Specifically, participants will be randomised into seeing one of the following two messages:

Group 1 - Not acknowledging uncertainty: "The World Health Organization has updated their previous recommendation and currently recommends that people should wash their hands with soap and water for at least 20 seconds and use hand sanitisers that contain at least 60% of alcohol. We also recommend sanitising high-touch surfaces such as doorknobs and handrails. Practicing hand and surface hygiene can best protect you from infection with the new virus, which transmits primarily via direct contact from hands to a person’s nose or mouth and not via contaminated water."

Group 2 - Acknowledging uncertainty: "The World Health Organization has updated their previous recommendation and currently recommends that people should wash their hands with soap and water for at least 20 seconds and use hand sanitisers that contain at least 60% of alcohol. We also recommend sanitising high-touch surfaces such as doorknobs and handrails. Practicing hand and surface hygiene can best protect you from infection with the new virus. This change was due to new scientific evidence showing that the virus transmits primarily via direct contact from hands to a person’s nose or mouth and not via contaminated water. It is possible that this recommendation may change as we learn about the pandemic in real time. We will do our best to let you know if new information comes along that may lead us to change our recommendation and what the change means for you."

For each of the messages, the survey will be programmed so that participants will be unable to progress to the next screen for twenty seconds to ensure they take sufficient time to read through and process the presented information.
Intervention Start Date
2024-01-30
Intervention End Date
2024-04-30

Primary Outcomes

Primary Outcomes (end points)
The experiment has two main trust-related outcomes: (1) participants' trust in the content/information of the recommendations; and (2) participants' trust in the sender/people formulating the recommendation.
Primary Outcomes (explanation)
The variables will be measured with the following questions:

Trust in content: "How trustworthy do you think this (updated) recommendation is?"
Trust in the sender: "How trustworthy do you think the organization that formulated this (updated) recommendation is?"

Both questions will have options ranging from 1 (not at all trustworthy) to 7 (very trustworthy). We will ask these questions after each round of the experiment to measure the initial trust and change in trust (i.e. updated trust).

Secondary Outcomes

Secondary Outcomes (end points)
We will capture two secondary outcomes: (1) intent to adopt the recommended behaviours; and (2) intent to share the received information.
Secondary Outcomes (explanation)
The variables will be measured by means of the following questions:

Behavioural intent: "How likely is it that you would adopt the recommendation of drinking only bottled or boiled water?" (for first message) / "How likely is it that you would adopt the recommendation of sanitising hands and high-touch surfaces?" (for second message)

Intent to share information: "How likely is it that you would share this information with your friends and family?"

Both questions will have options ranging from 1 (Not at all likely) to 7 (very likely). Similar to the primary outcomes, we will ask these questions after each round of the experiment.

Experimental Design

Experimental Design

In the first stage of the experiment, we will randomise 2/3rds of the participants into the certainty communication arm and 1/3rds into the uncertainty communication arm. In the second stage, we will randomise those who initially received the certainty communication into either updated certainty communication or updated uncertainty communication in a 1:1 ratio. Those who initially received the uncertainty communication will all receive the updated uncertainty communication in the second stage. This leads to three experimental groups in a 1:1:1 ratio: (i) certainty in the first and certainty in the second message (the reference group), (ii) uncertainty in the first and uncertainty in the second message, and (iii) certainty in the first and uncertainty in the second message.

Experimental Design Details
Randomization Method
Computer-based randomization in survey software Qualtrics
Randomization Unit
Individual survey participants
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
NA
Sample size: planned number of observations
4000-5000 individuals per country
Sample size (or number of clusters) by treatment arms
1000-2000 individuals per experimental arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Our main outcomes are Likert measures of trust that range from 1 to 7. To be conservative, we powered our study under a dichotomised version of the Likert response with the aim of detecting a 5 percentage point effect. Thus, with a binary outcome, a control proportion of 50\% (the most conservative assumption), a two-tailed alpha=0.05, and 80% power, we would require a sample of 1565 individuals per experimental arm to detect a 5 percentage point effect. This would lead to a required total sample size of 4695 individuals per country.
IRB

Institutional Review Boards (IRBs)

IRB Name
German Association for Experimental Economic Research e.V.
IRB Approval Date
2023-10-19
IRB Approval Number
jtTxj9rD
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials