Simplicity vs Precision in Eliciting Probabilistic Expectations

Last registered on September 22, 2020

Pre-Trial

Trial Information

General Information

Title
Simplicity vs Precision in Eliciting Probabilistic Expectations
RCT ID
AEARCTR-0006482
Initial registration date
September 21, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 22, 2020, 7:46 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Karlsruhe Institute of Technology

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2020-09-17
End date
2020-09-30
Secondary IDs
Abstract
The practice of directly surveying consumers' macroeconomic expectations has become more widespread in recent years, as the generated data finds broader applications in economics. In particular, growing attention is paid to probabilistic expectations, which provide informative measures of individual uncertainty. However, there is still no clear consensus on the best question wording and format, despite them being central in the process of eliciting probabilistic expectations. Existing research states that survey questions, as a form of communication should ultimately be understood and interpreted in the same way by both researchers and participants. Complex, hard-to read questions that cause high non-response rates are therefore undesirable. This applies equally to question format: validity of results using expectations data strongly rely on both respondents' willingness and ability to express expectations over uncertain events in a probabilistic form. Ideally, the question format should take both into account. To sum up, a question should combine a simple wording and design, such that the elicited expectations accurately represent the respondents beliefs about the future outcome. On the other hand, simplified wordings often involve a certain degree of ambiguity and lay the ground for diffuse interpretations such that the resulting answers are no more interpersonally comparable. I address the trade-off between simplicity and precision in survey design in the context of inflation expectations of a representative sample of German consumers.

In the current experiment, conducted as part of the Bundesbank Online Pilot on Consumer Expectations, I test the effects of variations in question wording and format on consumer response behaviour. To this end, respondents are broken down into four treatment groups and presented versions of the question about probabilistic expectations. As part of the experiment, two competing wording versions, previously known from leading consumer surveys, are considered in the treatments: expected change in (i) prices in general or (ii) the inflation rate. In addition, a question format to elicit a subjective distribution alternative to the one already widely used is also included in the treatments. So far, the practice has been for respondents to assign probabilities that the inflation rate over the next 12 months will fall within a certain range. As opposed to this, in two of the four treatments a less restrictive setting is used, where survey participants are asked to report the most likely, the minimum and the maximum outcome they expect. I hypothesize that the 'inflation-rate' treatment, while prompting people to think more about price changes on a broader scale, also generates a higher non-response rate, especially among the less educated population. I also predict that moving from a probabilistic question formulation with predefined intervals to a less restrictive one, one should observe an upward shift in expectations coupled with larger variation in responses.
External Link(s)

Registration Citation

Citation
Pavlova, Lora. 2020. "Simplicity vs Precision in Eliciting Probabilistic Expectations." AEA RCT Registry. September 22. https://doi.org/10.1257/rct.6482-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2020-09-17
Intervention End Date
2020-09-30

Primary Outcomes

Primary Outcomes (end points)
Effects of question wording and format on consumer response behaviour. In particular, patterns of assigning probability to intervals, presence of outliers, variance and IQR of the subjective distribution, disagreement among respondents, non-response rate, internal consistency of answeres and heterogeniety of the effects with respect to demographic groups.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Respondents are randomly assigned to four groups and presented versions of the question about probabilistic expectations, where both wording and format are varied. Then respondents are asked a follow-up quesion on how they intepreted the wording of the previous quesions about their expectations.
Experimental Design Details
Randomization Method
Randomization is done by a survey firm.
Randomization Unit
Individual.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
4000 respondents randomly assinged to four groups.
Sample size: planned number of observations
4000 respondents
Sample size (or number of clusters) by treatment arms
Approximately 1000 in each of the four treatment arms.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials