Question Order and Expectation Formation: Experimental Evidence from a Business Survey

The intervention is a manipulation of question order within the questionnaire of the ifo Business Survey; the informational content, question wording, and response scales are held identical across all respondents. Firms are randomly assigned, within strata defined by firm size and sector, to one of three arms that differ only in the sequence of three question blocks: an expectation block (qualitative expectations about the firm's own business situation and about the German economy, plus a quantitative turnover expectation), a substantive attribute block (operating costs, skilled-labor availability, and competitive intensity, each with a retrospective and a forward-looking item), and a neutral control block matched to the attribute block in number of items, response format, and cognitive demands but covering topics unlikely to prime business-cycle considerations (digitalization of administrative and billing processes, company-provided training, and predictability of operational processes).
In the control arm (A), respondents answer the neutral filler block first and the expectation block afterwards. In treatment arm B, respondents answer the substantive attribute block first and the expectation block at the end, so that they reflect on concrete business conditions before reporting expectations. In treatment arm C, respondents answer the expectation block at the very beginning of the questionnaire, before any other firm-related content, with the attribute block following afterwards. Within each block, the display order of topics is randomized to avoid within-block ordering effects, while the internal order of the three expectation items is fixed across all arms. The contrast between arms B and C isolates the pure question-order effect, since both answer the same questions in reverse order; the control arm serves as a diagnostic benchmark for separating content-specific priming from positioning and attention effects.

2026-08-01

2026-08-31

• Qualitative expectation about the firm's own business situation over the forecast horizon, three-point scale (better / unchanged / worse), coded symmetrically (+1 / 0 / −1).
• Qualitative expectation about the overall economic situation in Germany, same three-point scale and coding.
• Quantitative turnover expectation: expected percentage change in firm turnover relative to the previous year (continuous).

The three primary outcomes are taken directly from single survey items and require only minimal coding; none is a composite index. (1) The qualitative expectation about the firm's own business situation is elicited on a three-point scale (better / unchanged / worse) and coded symmetrically as +1 (better), 0 (unchanged), and −1 (worse), so that the variable can be read as a directional balance measure of firm-level optimism. (2) The qualitative expectation about the overall economic situation in Germany is elicited and coded identically on the same +1 / 0 / −1 scale. (3) The quantitative turnover expectation is the respondent's reported expected percentage change in firm turnover relative to the previous year, used as a continuous variable without further transformation (declines entered as negative values). Each outcome is analyzed separately rather than aggregated into a single index, to avoid imposing untested assumptions about the relative weight of firm-specific versus macroeconomic and qualitative versus quantitative expectation dimensions.

We implement a three-arm, between-subjects randomized survey experiment embedded in the regular ifo Business Survey. Respondents are randomly assigned, within strata defined by firm size and sector, to one of three arms that differ only in the order in which three question blocks are presented: a block of expectation questions, a block of substantive firm-related attribute questions, and a neutral filler block. Question content, wording, and response scales are identical across arms; only the block order varies. In the control arm, the neutral filler block precedes the expectation block; in the first treatment arm, the attribute block precedes the expectation block (expectations at the end); in the second treatment arm, the expectation block comes first, before any other firm-related content. Within each block the order of topics is randomized, while the internal order of the expectation items is fixed across arms.
The primary comparison is between the two treatment arms, which answer the same questions in reverse order and thereby isolate the effect of the position of the expectation block. Comparisons of each treatment arm against the control arm serve diagnostic purposes, helping to separate content-specific priming from positioning or attention effects. Outcomes are the reported expectations, analyzed separately for each expectation dimension; treatment effects are estimated as differences in mean outcomes across arms.

Not available

We implement a three-arm, between-subjects randomized survey experiment embedded in the regular ifo Business Survey. Respondents are randomly assigned, within strata defined by firm size and sector, to one of three arms that differ only in the order in which three question blocks are presented: a block of expectation questions, a block of substantive firm-related attribute questions, and a neutral filler block. Question content, wording, and response scales are identical across arms; only the block order varies. In the control arm, the neutral filler block precedes the expectation block; in the first treatment arm, the attribute block precedes the expectation block (expectations at the end); in the second treatment arm, the expectation block comes first, before any other firm-related content. Within each block the order of topics is randomized, while the internal order of the expectation items is fixed across arms.
The primary comparison is between the two treatment arms, which answer the same questions in reverse order and thereby isolate the effect of the position of the expectation block. Comparisons of each treatment arm against the control arm serve diagnostic purposes, helping to separate content-specific priming from positioning or attention effects. Outcomes are the reported expectations, analyzed separately for each expectation dimension; treatment effects are estimated as differences in mean outcomes across arms.

Individual firm / responding establishment (one questionnaire per firm). No clustering of the randomization above the firm level.

No

Main wave (ifo Business Survey, August 2026): target the full active respondent pool of the wave across the three sectors (manufacturing, services, trade).

4000

1300

We conduct an ex ante power analysis for the primary estimand ΔBC at a two-sided significance level of α = 0.05 and power of 0.80. With a planned total of 4,000 responses and equal allocation across the three arms, the primary comparison (Group B vs. Group C) draws on approximately 1,333 firms per arm. For the quantitative turnover expectation, calibrating to the empirical standard deviation of reported turnover growth in recent waves (SD ≈ 7 percentage points), the minimum detectable difference in mean turnover expectations between arms B and C is approximately 0.76 percentage points (ranging from about 0.68 to 0.80 pp for SD between 6.3 and 7.4). For the qualitative expectation outcomes, coded symmetrically as +1 / 0 / −1, the minimum detectable difference in the mean balance statistic is approximately 0.076 scale points (assuming SD ≈ 0.70). Expressed as a two-sample difference in the share of firms reporting "better," the minimum detectable effect is roughly 4.5 to 5.4 percentage points, depending on the baseline share (calibrated to recent ifo Business Survey response distributions).