Prolific sample for: Competing Causal Interpretations: An Experimental Study

Last registered on December 06, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
Prolific sample for: Competing Causal Interpretations: An Experimental Study
RCT ID
AEARCTR-0012652
Initial registration date
December 04, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 06, 2023, 8:55 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 06, 2023, 11:40 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
United States of America
Region

Primary Investigator

Name
Sandro Ambuehl
Affiliation
University of Zurich
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Heidi Thysen
PI Affiliation
Norwegian School of Economics
Contact Investigator

Additional Trial Information

Status
In development
Start date
2023-12-05
End date
2023-12-16
Keywords
Lab
Additional Keywords
Causality, directed acyclic graph, DAG, model persuasion, narratives
JEL code(s)
C91, D01, D83
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This is a follow up to a lab study we conducted previously. The abstract of the lab study is this:

A central factor when choosing an action is its causal effect on the outcome of interest. Yet, causal information is often lacking. People instead observe correlational or historical data, along with causal interpretations and recommendations provided by experts who frequently disagree with each other. Our laboratory experiments study choice in such settings, where beliefs concern the structure of the data-generating process rather than merely magnitudes. Roughly half of our subjects attempt to determine the fit of the causal interpretations to past data, as the literature on model persuasion assumes. We characterize the limits to their ability to do so. Half the subjects’ choices are co-determined by the interpretations’ promises of future payouts, as the literature on narrative competition assumes, or by the downside these choices entail if they are mistaken. Subjects also commonly employ heuristics such as Occam’s razor. The fact that they typically prefer flexibility over parsimony insures them against bad choices in some settings but has the opposite effect in others. Our estimates predict well out of sample and closely agree across two different identification strategies in two different samples. We also study the extent to which behavior is robust to framing and the relation between subjects’ choices and their political attitudes and psychological characteristics. Our results characterize the cases in which subjects’ behavioral tendencies render them most receptive to misleading interpretations. They also inform the literatures on narrative competition and model persuasion.
External Link(s)
Slides of presentation of lab study results

Registration Citation

Citation
Ambuehl, Sandro and Heidi Thysen. 2023. "Prolific sample for: Competing Causal Interpretations: An Experimental Study." AEA RCT Registry. December 06. https://doi.org/10.1257/rct.12652-1.1
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
SETTING: See description on experimental design

INTERVENTION: We run two treatments. In one treatment, we provide help to subjects, in two forms. First, subjects have access to a description of the correlational implications of causal structures. Second, we highlight data charts for which the competing interpretations have conflicting implications. In the other treatment, subjects get no such help.
Intervention Start Date
2023-12-05
Intervention End Date
2023-12-16

Primary Outcomes

Primary Outcomes (end points)
The sequence of chocie of interpretations across all rounds of the study.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Relation of choice of interpretations to demographic factors, especially education
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
In our experiment, subjects choose an action at a cost. The action possibly affects a monetary outcome. Whether it does and by how much depends on a data-generating process (DGP) that involves the action, the outcome, and an additional quantity (the covariate). Subjects do not know the DGP, but they observe information about it. Specifically, in each round, we fit a misspecified model to large datasets generated by the DGP, along with the true model. For each model, we calculate the implied optimal action (the recommendation), and the payoff the subject can expect if the specification is correct and the subject follows that model (the promise). Subjects observe each model’s recommendation and promise. While we do not tell subjects which models are misspecified, we present them with a graphical description of each model in the form of a directed acyclic graph (DAG). In addition, subjects can observe unconditional correlations between any pair of variables in the data generated by the DGP, as well as the correlations between any pair of variables conditional on any third variable. Subjects who wish and are able to do so can therefore check which models may be misspecified. Throughout, our DGPs and misspecified models are recursive systems of linear Gaussian equations. Subjects choose which model to follow. The corresponding action recommendation is carried out and determines the subjects’ payout according to the DGP.
We frame this setting to subjects as follows. A true ‘mechanism’ determines the relation between the subject’s ‘action’ and her ‘bonus.’ Two or three ‘advisors’ each have a ‘theory’ about that mechanism. While data cannot affect an advisor’s theory, each advisor has used the data generated by the true mechanism to derive a recommendation for what action is optimal and what bonus the subject can expect if the advisor’s theory is correct. Subjects select one of the advisors whose recommendation is carried out.
Experimental Design Details
Randomization Method
Randomization done online by computer
Randomization Unit
Subject
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Total of 800 subjects, 400 for each treatment.
Sample size: planned number of observations
Each subject makes 18 main choices between conflicting interpretations
Sample size (or number of clusters) by treatment arms
Total of 800 subjects, 400 for each treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB board of the Department of Economics, University of Zurich
IRB Approval Date
2021-12-13
IRB Approval Number
OEC IRB # 2021-102
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials