x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Simplicity vs Precision in Eliciting Probabilistic Expectations
Last registered on September 22, 2020

Pre-Trial

Trial Information
General Information
Title
Simplicity vs Precision in Eliciting Probabilistic Expectations
RCT ID
AEARCTR-0006482
Initial registration date
September 21, 2020
Last updated
September 22, 2020 7:46 AM EDT
Location(s)
Region
Primary Investigator
Affiliation
Karlsruhe Institute of Technology
Other Primary Investigator(s)
Additional Trial Information
Status
On going
Start date
2020-09-17
End date
2020-09-30
Secondary IDs
Abstract
The practice of directly surveying consumers' macroeconomic expectations has become more widespread in recent years, as the generated data finds broader applications in economics. In particular, growing attention is paid to probabilistic expectations, which provide informative measures of individual uncertainty. However, there is still no clear consensus on the best question wording and format, despite them being central in the process of eliciting probabilistic expectations. Existing research states that survey questions, as a form of communication should ultimately be understood and interpreted in the same way by both researchers and participants. Complex, hard-to read questions that cause high non-response rates are therefore undesirable. This applies equally to question format: validity of results using expectations data strongly rely on both respondents' willingness and ability to express expectations over uncertain events in a probabilistic form. Ideally, the question format should take both into account. To sum up, a question should combine a simple wording and design, such that the elicited expectations accurately represent the respondents beliefs about the future outcome. On the other hand, simplified wordings often involve a certain degree of ambiguity and lay the ground for diffuse interpretations such that the resulting answers are no more interpersonally comparable. I address the trade-off between simplicity and precision in survey design in the context of inflation expectations of a representative sample of German consumers.

In the current experiment, conducted as part of the Bundesbank Online Pilot on Consumer Expectations, I test the effects of variations in question wording and format on consumer response behaviour. To this end, respondents are broken down into four treatment groups and presented versions of the question about probabilistic expectations. As part of the experiment, two competing wording versions, previously known from leading consumer surveys, are considered in the treatments: expected change in (i) prices in general or (ii) the inflation rate. In addition, a question format to elicit a subjective distribution alternative to the one already widely used is also included in the treatments. So far, the practice has been for respondents to assign probabilities that the inflation rate over the next 12 months will fall within a certain range. As opposed to this, in two of the four treatments a less restrictive setting is used, where survey participants are asked to report the most likely, the minimum and the maximum outcome they expect. I hypothesize that the 'inflation-rate' treatment, while prompting people to think more about price changes on a broader scale, also generates a higher non-response rate, especially among the less educated population. I also predict that moving from a probabilistic question formulation with predefined intervals to a less restrictive one, one should observe an upward shift in expectations coupled with larger variation in responses.
External Link(s)
Registration Citation
Citation
Pavlova, Lora. 2020. "Simplicity vs Precision in Eliciting Probabilistic Expectations." AEA RCT Registry. September 22. https://doi.org/10.1257/rct.6482-1.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2020-09-17
Intervention End Date
2020-09-30
Primary Outcomes
Primary Outcomes (end points)
Effects of question wording and format on consumer response behaviour. In particular, patterns of assigning probability to intervals, presence of outliers, variance and IQR of the subjective distribution, disagreement among respondents, non-response rate, internal consistency of answeres and heterogeniety of the effects with respect to demographic groups.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Respondents are randomly assigned to four groups and presented versions of the question about probabilistic expectations, where both wording and format are varied. Then respondents are asked a follow-up quesion on how they intepreted the wording of the previous quesions about their expectations.
Experimental Design Details
Randomization Method
Randomization is done by a survey firm.
Randomization Unit
Individual.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
4000 respondents randomly assinged to four groups.
Sample size: planned number of observations
4000 respondents
Sample size (or number of clusters) by treatment arms
Approximately 1000 in each of the four treatment arms.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS