Attitudes towards science when its’ results (don’t) confirm people’s priors

Last registered on December 16, 2020

Pre-Trial

Trial Information

General Information

Title
Attitudes towards science when its’ results (don’t) confirm people’s priors
RCT ID
AEARCTR-0006904
Initial registration date
December 16, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 16, 2020, 8:18 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Technical University of Munich

Other Primary Investigator(s)

PI Affiliation
ZEW - Leibniz Centre for European Economic Research, Mannheim

Additional Trial Information

Status
In development
Start date
2021-01-01
End date
2021-09-30
Secondary IDs
Abstract
The public’s attitudes towards science are crucial for the functioning of enlightened societies, since attitudes towards science may affect support for public funding of scientific activities, policy choices, or the public’s willingness to comply with science-based behavioral rules (e.g., wearing a mask in public during the COVID-19 pandemic).
We hypothesize that public trust in science may depend on the extent to which its findings confirm people’s prior beliefs. To test this hypothesis, we conduct a randomized experiment within a large-scale, representative survey in Germany. Focusing on the controversial question how migration affects natives’ labor-market success, we first elicit all respondents’ beliefs about the impact of migration on natives’ labor-market success. Then, we randomly assign respondents to one of four experimental groups (one control group, three information-treatment groups). Treatment 1 informs that some studies find negative impacts of migration on natives’ labor-market success, treatment 2 informs that some studies find no significant impact, and treatment 3 informs about both types of findings (note that the information provided in all three treatments is truthful). After treatment administration, we measure individuals’ attitudes towards science using several survey questions, and we summarize the answers to these questions into one index, which is our main outcome of interest. In our main analysis, we plan to estimate heterogeneous information-treatment effects by respondents’ prior beliefs. This will allow us to investigate the extent to which scientific findings confirm or contradict people’s priors affects their attitudes towards science. Furthermore, we will estimate treatment effects on individuals’ posterior beliefs about the impact of migration, and on general attitudes towards migration to investigate the extent to which individuals update their beliefs and attitudes about migration in response to (contradicting) scientific information.
External Link(s)

Registration Citation

Citation
Blesse, Sebastian and Philipp Lergetporer. 2020. "Attitudes towards science when its’ results (don’t) confirm people’s priors." AEA RCT Registry. December 16. https://doi.org/10.1257/rct.6904-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
We investigate how information about scientific findings affects public attitudes towards science. We particularly focus on whether learning about scientific findings that contrast individuals’ prior beliefs lowers individuals’ attitudes towards science. Therefore, we implement an online-survey experiment among a representative sample of adults aged 16 to 75 years in Germany. First, we elicit all respondents’ beliefs about the impact of migration on natives’ labor-market success. Then, we randomly assign respondents into one of four different experimental groups (one control group, three information-treatment groups that all provide truthful information). Treatment 1 informs that some studies find negative impacts of migration on natives’ labor-market success, treatment 2 informs that some studies find no significant impact, and treatment 3 informs about both types of findings. Afterwards, we measure individual’s attitudes towards science in a series of questions that we combine into one index which is our main outcome of interest. Our main analytical focus will be on heterogeneous treatment effects by respondents’ prior beliefs to investigate the extent to which scientific findings confirm or contradict people’s priors affects their attitudes towards science. Furthermore, we will estimate treatment effects on individuals’ posterior beliefs about the impact of migration, and on general attitudes towards migration to investigate the extent to which individuals update their beliefs and attitudes about migration in response to (contradicting) scientific information.
Intervention Start Date
2021-01-01
Intervention End Date
2021-03-31

Primary Outcomes

Primary Outcomes (end points)
Our primary outcomes of interest are (i) the index on attitudes towards science, and (ii) attitudes towards migration. We will show average treatment effects and heterogeneous treatment effects by prior beliefs.
Primary Outcomes (explanation)
The index on attitudes towards science will be constructed using the method by Kling et al. (2007), combining the following outcome measures: Outcome 1: Credibility of migration studies, Outcome 2: Trust in science, Outcome 3: Demand for more scientific evidence, Outcome 4: Demand for evidence-based policy making, Outcome 5: Demand for further information on scientific studies.

Attitudes towards migration will be measured in one survey question (Outcome 6, see above).

In our main specifications, we will show average treatment effects and heterogeneous treatment effects by prior beliefs.

Secondary Outcomes

Secondary Outcomes (end points)
Effect heterogeneities on the individual items of the index; long-term treatment effects on (i) beliefs on the impact of migration on natives’ labor-market success, and on (ii) Outcome 1: Credibility of migration studies; treatment-effect heterogeneities by attitudes towards migration elicited in previous survey waves and political preferences.
Secondary Outcomes (explanation)
The estimation of long-term effects of the treatment is facilitated by the specific structure of the survey experiment. The experiment is embeded within a panel study that repeatedly surveys the same respondents, and the present study spans over two survey waves 2 months apart. Outcomes 1-3 are measured in survey wave 1. At the beginning of survey wave 2, we re-eilcit subjects’ beliefs on the impact of migration on natives’ labor market success, and (i) Outcome 1: Credibility of migration studies. Only after re-eliciting these outcomes, we remind respondents of the information provided in the first survey wave by simply showing them their treatment again (treatment assignment remains constant across survey waves). After the reminder, we elicit Outcomes 4-6.

Experimental Design

Experimental Design
We conduct the experiment in a sample of about 4400 adults aged between 15 and 75 years. The survey is conducted as an online survey within the representative, probability-based “German Internet Panel” (GIP) (see Blom et al., 2015 for details). Our experiment spans over two GIP waves with the same respondents.

Our experiment is structured as follows:
Stage 1: prior beliefs on the impact of migration on natives’ labor-market success (wave 1)

Stage 2: randomized information provision (wave 1)
Control group: No information; Treatment 1: studies find negative impacts of migration on natives’ employment, Treatment 2: studies find no impacts of migration on natives’ employment, Treatment 3: studies find both no and negative impacts of migration on natives’ employment

Stage 3: measuring outcomes
Outcome 1: Credibility of migration studies (wave 1)
Outcome 2: Trust in science (wave 1)
Outcome 3: Demand for more scientific evidence (wave 1)
Re-elicitation of prior beliefs (wave 2)
Re-elicitation of outcome 1 (wave 2)
Reminder about information treatments (wave 2): same treatment assignment as in wave 1
Outcome 4: Demand for evidence-based policy making (wave 2)
Outcome 5: Demand for further information on scientific studies (wave 2)
Outcome 6: Attitudes towards migration (wave 2)
Experimental Design Details
Randomization Method
Randomization is carried out by the survey company, using a computer.
Randomization Unit
at the individual level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
4,400 Respondents
Sample size: planned number of observations
4,400 adults aged 16 – 75 years
Sample size (or number of clusters) by treatment arms
Approx. 1,100 will be assigned to each of the treatment groups
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials