Measuring inflation expectations: How stable are density forecasts?

Last registered on January 14, 2022

Pre-Trial

Trial Information

General Information

Title
Measuring inflation expectations: How stable are density forecasts?
RCT ID
AEARCTR-0008716
Initial registration date
December 16, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 19, 2021, 1:02 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
January 14, 2022, 5:55 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region
Region

Primary Investigator

Affiliation
Heidelberg University

Other Primary Investigator(s)

PI Affiliation
Heidelberg University
PI Affiliation
Heidelberg University

Additional Trial Information

Status
Completed
Start date
2021-12-17
End date
2021-12-22
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
A common assumption of economic models is that inflation expectations matter for firms’ and households’ decisions. Central banks are interested in measuring these expectations to monitor the effectiveness of their monetary policy and to plan future interventions. Inflation expectations are commonly measured by conducting surveys on representative samples of a population. In the past, many of these surveys asked respondents either for a point prediction of the future inflation rate (that is to state a specific number), or to pick an interval out of multiple offered. More recently, central banks have started using density forecasts.

In a density forecast, survey respondents are shown a number of intervals representing possible ranges of future inflation rate (e.g. 0% to 1% might be one such an interval, 1% to 2% the next). We refer to the totality of these intervals as the scale of the density forecast. In several density forecasts, such as that of the New York Fed SCE (survey of consumer expectations), the scale is centered on a salient value, such as zero and covers both possibilities, inflation and deflation rates. Respondents are then asked to state for each interval a probability that reflects how likely they think it is that the inflation rate will fall within this interval in the future. Compared to point forecasts (which ask for a single number) or to questions asking to pick a single interval, density forecast provide much more fine-grained information about a respondents’ inflation expectations. Specifically, this approach allows to quantify the uncertainty respondents have in their expectations and to better study the disagreement between differing expectations.

Density forecasts have become increasingly common, with institutions such as such as the New York Fed or the German Bundesbank using them. In the future, we expect density forecast to become a ubiquitous tool in inflation expectations research. Like every other survey measure, data obtained from density forecasts can be misleading if the survey questions are not designed and interpreted carefully. Past research has shown that the scales on which a question is asked affect the answers given by respondents. Due to that, particular care must be taken in how the scale is constructed and how meticulous it is used in subsequent versions of the survey.

Our research project provides a thorough test of the consistency of expectations stated in density forecasts with regards to the scale used. We use the inflation expectation question utilized by the NY Fed in the SCE and the Bundesbank as a baseline, which allows our results to provide meaningful results about currently used methods by policy-relevant institutions. Based on the question, we test how the answers are affected by changes to the scale.
External Link(s)

Registration Citation

Citation
Becker, Christoph, Peter Duersch and Thomas Eife. 2022. "Measuring inflation expectations: How stable are density forecasts?." AEA RCT Registry. January 14. https://doi.org/10.1257/rct.8716-2.0
Experimental Details

Interventions

Intervention(s)
Eliciting inflation density forecasts using different scales
Intervention Start Date
2021-12-17
Intervention End Date
2021-12-22

Primary Outcomes

Primary Outcomes (end points)
(i) Individual density forecasts of the 12-month ahead inflation rate [Q1]
(ii) Individual estimate if 12 months from now the inflation rate will be positive or negative (deflation) [Q3]
(iii) Individual point forecast of the 12-month ahead inflation rate [Q5]
Primary Outcomes (explanation)
(i) Participants distribute probabilities on a scale of possible inflation rates with 6-14 bins (depending on treatment). Probabilities need to add up to 100.
(ii) Participants are asked to state whether they think inflation or deflation is more likely occur 12 months from now.
(iii) Participants are asked about a point estimation depending on their answer in (ii). If they answered “deflation”, they will be asked for a point forecast for deflation and similarly for inflation.

Secondary Outcomes

Secondary Outcomes (end points)
(i) Participants certainty regarding their answers to [Q1, Q3, Q5]. [Q2, Q4, Q6]
(ii) Demographics (age, gender, education, US state of residence) [Questionnaire]
(iii) Financial literacy [Questionnaire]
(iv) Knowledge of inflation target [Questionnaire]
Secondary Outcomes (explanation)
(i) Ordinal measurement of certainty (six-point Likert scale).
(ii) Answers will be used as control variables.
(iii) We use the three multiple-choice questions from Lusardi and Mitchell (2011).
(iv) Multiple-choice question that tests if participants know the inflation target of the Fed.

Experimental Design

Experimental Design
Participants in the survey will be asked about their density forecast for the inflation 12 months from now [Q1]. The question for this density forecast is directly taken from the NY Fed. After the density forecast, participants will state whether they expect inflation or deflation 12 months from now [Q3] and then give their inflation expectations again in a point forecast [Q5]. Both questions are also taken from the NY Fed SCE. This will allow us to check if the predictions of these questions align. After each of these questions, participants state how certain they are regarding their answer on a six-item Likert scale [Q2, Q4, Q6]. After this, the participants will answer a questionnaire including control and demographic questions [Questionnaire].

Participants in the main part of the experiment will take part in either the Baseline or one of 12 possible treatments. In the Baseline participants will answer the density forecast as it is currently used by the NY Fed and the Bundesbank. The Baseline will serve as a benchmark for our other treatments. Our treatment interventions fall into three categories: Shifting, Compression and Centralization. In the four Shifting treatments, the scale will be shifted to either the right or the left. Both shifts are implemented in two different proportions, to test how larger shifts affect the answers. In the Compression treatments, the scales will either be compressed, that is being pulled together more tightly around the midpoint, or decompressed, implying spreading the scale further around the midpoint. The latter would also include larger values previously not captured by the scale. Both compression and decompression come again in two proportions. The last set of treatments is designated Centralization. A feature of the NY Fed density forecast is that bins closer to the midpoint are smaller compared to others. Our treatments will vary the Centralization of the scale by further changing the size of the different bins (3 treatments) and by providing a scale with completely even bin sizes (1 treatment).

In order to recruit a representative sample of the US population, we will run the survey on the online research platform Prolific. All treatments will be run at the same time and participants will be assigned to one of these treatments iteratively when starting the study. This way, we make sure that any differences we obtain are not driven by timing but instead reflect a genuine effect of our treatment intervention. Participants will earn a fixed fee of £1 for their participation.
Experimental Design Details
Randomization Method
Randomization of treatments done by the experimental software.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1300 Prolific participants
Sample size: planned number of observations
1300 Prolific participants
Sample size (or number of clusters) by treatment arms
100 Prolific participants per condition (Baseline + 12 treatments):

100 Shift I, 100 Shift II, 100 Shift III, 100 Shift IV, 100 Compression I, 100 Compression II, 100 Compression III, 100 Compression IV, 100 Centralization I, 100 Centralization II, 100 Centralization III, 100 Centralization IV
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Test if the mean between any two treatments differs. Test used is two-sample t-test, assuming equal variances in both treatment groups. All our treatment groups have 100 participants. For this group size, power calculation (alpha = 0.05, beta = 0.8) informs us that the minimum detectable effect size would be a difference in means of 0.398.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
German Association for Experimental Economic Research e.V.
IRB Approval Date
2021-07-10
IRB Approval Number
apyKIJdX

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
December 19, 2021, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
December 19, 2021, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
1301 participants
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
1301 participants
Final Sample Size (or Number of Clusters) by Treatment Arms
101 Shift I, 100 Shift II, 100 Shift III, 100 Shift IV, 100 Compression I, 100 Compression II, 100 Compression III, 100 Compression IV, 100 Centralization I, 100 Centralization II, 100 Centralization III, 100 Centralization IV
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials