x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Predicting experimental results: The role of incentives, anchoring, and experience
Last registered on May 29, 2020

Pre-Trial

Trial Information
General Information
Title
Predicting experimental results: The role of incentives, anchoring, and experience
RCT ID
AEARCTR-0005944
Initial registration date
May 29, 2020
Last updated
May 29, 2020 3:34 PM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
ifo Institute - Leibniz Institute for Economic Research at the University of Munich
Other Primary Investigator(s)
PI Affiliation
ifo Institute
PI Affiliation
ifo Institute
PI Affiliation
ifo Institute
PI Affiliation
ifo Institute
Additional Trial Information
Status
In development
Start date
2020-06-03
End date
2021-12-31
Secondary IDs
Abstract
We let people predict the outcome of an online survey experiment and investigate the causal effects of (i) monetary incentives, (ii) anchoring, and (iii) participants’ experience in the experiment on the accuracy of predictions. Therefore, we implement an online-survey experiment among a representative sample of adults aged 18 to 69 years in Germany, and additionally survey a sample of experts (economics professors).
Respondents in the representative sample are randomized into six different experimental groups. The first three groups participate in an “information experiment” on preferences for increased school spending. Groups 4, 5, and 6 participate in a “prediction experiment” and predict the results from the “information experiment”. Group 4 receives no incentives for a correct prediction and no anchor when stating the prediction. Group 5 is offered a monetary incentive for a correct prediction, and group 6 is provided with an anchor. In addition, we study the causal effect of experience on prediction accuracy by letting groups 1, 2, and 3 predict the results in the “information experiment” after participating in that experiment. The expert sample completes the same prediction task as group 4.
Comparing prediction accuracy between experts and non-experts, we investigate whether providing incentives, anchors, and experience to non-experts improves prediction accuracy of the general population towards the levels of experts.
External Link(s)
Registration Citation
Citation
Grewenig, Elisabeth et al. 2020. "Predicting experimental results: The role of incentives, anchoring, and experience." AEA RCT Registry. May 29. https://doi.org/10.1257/rct.5944-1.0.
Experimental Details
Interventions
Intervention(s)
We let subjects predict the outcome of an online survey experiment and investigate the causal effects of (i) providing monetary incentives for correct predictions, (ii) providing an anchor of control-group behavior in the experiment, and (iii) participating in the experiment on the accuracy of subjects’ predictions. Therefore, we implement an online-survey experiment among a representative sample of adults aged 18 to 69 years in Germany, and additionally survey a sample of experts (economics professors) from a regularly conducted German expert sample.
Respondents in the representative sample are randomized into six different experimental groups. The first three groups participate in an “information experiment” on preferences for increased school spending. Group 1 receives no information when stating spending preferences; group 2 is informed about average current school spending per student, and respondents in group 3 have the option to acquire the spending-information provided to group 2. Groups 4, 5, and 6 participate in and “prediction experiment” and predict the results of the “information experiment”. Group 4 receives no anchor and no incentive when stating the prediction. Group 5 is offered a monetary incentive for a correct prediction, and group 6 is provided with an anchor. In addition, we study the causal effect of experience on prediction accuracy by letting groups 1, 2, and 3 predict the results in the “information experiment” after participating in that experiment. The experts complete the same prediction task as group 4. Comparing prediction accuracy between experts and non-experts, we investigate whether providing incentives, anchors, and experience to non-experts improves prediction accuracy of the general population towards the levels of experts.
Intervention Start Date
2020-06-03
Intervention End Date
2020-06-17
Primary Outcomes
Primary Outcomes (end points)
Our primary outcomes of interest are respondents’ predictions of average answers in the three groups of the “information experiment”.
Primary Outcomes (explanation)
The predictions of respondents in TREATMENT 4 (elicited in stage 2) will serve as the benchmark. Comparing these predictions to the predictions in TREATMENT 5 (stage 2), TREATMENT 6 (stage 2), and TREATMENTS 1 to 3 (stage 3) will allow us to assess the causal effects of providing monetary incentives, benchmarks, and experimental experience, respectively, on prediction accuracy. In addition, we will let experts (economics professors) predict experimental results using the questions of TREATMENT 4 (stage 2) as an additional benchmark.
Secondary Outcomes
Secondary Outcomes (end points)
Treatment-effect heterogeneities in the “prediction experiment” by prior beliefs (elicited in stage 1), and respondents’ highest educational degree.
Secondary Outcomes (explanation)
We will investigate treatment-effect heterogeneities in the “prediction experiment” by prior beliefs elicited in stage 1, which will allow us to assess whether respondents’ prediction quality is affected by own knowledge of education spending, i.e. the topic covered in “information experiment”. In addition, we will investigate heterogeneous treatment effects by respondents’ highest educational degree.
Experimental Design
Experimental Design
We conduct the experiment in a sample of 6,000 adults aged between 18 and 69 years. The survey is conducted in cooperation with respondi. The recruitment and polling is managed by respondi, who collect the data via an online platform. That is, our participants answer the survey questions autonomously on their own digital devices. Randomization is carried out by respondi at the individual level, using a computer. In addition, the expert sample from the “ifo Ökonomenpanel” will cover around 200 economics professors in Germany.

Our experiment in the representative sample is structured as follows:
Treatment 1:
Stage 1: prior beliefs on average school spending
Stage 2: preferences for school-spending increases (no information)
Stage 3: predicting experimental results
Stage 4: open-ended question on prediction formation
Stage 5: closed-ended question on prediction formation

Treatment 2:
Stage 1: prior beliefs on average school spending
Stage 2: preferences for school-spending increases (information provision)
Stage 3: predicting experimental results
Stage 4: open-ended question on prediction formation
Stage 5: closed-ended question on prediction formation

Treatment 3:
Stage 1: prior beliefs on average school spending
Stage 2: preferences for school-spending increases (voluntary information acquisition)
Stage 3: predicting experimental results
Stage 4: open-ended question on prediction formation
Stage 5: closed-ended question on prediction formation

Treatment 4:
Stage 1: prior beliefs on average school spending
Stage 2: predicting experimental results
Stage 3: open-ended question on prediction formation
Stage 4: closed-ended question on prediction formation

Treatment 5:
Stage 1: prior beliefs on average school spending
Stage 2: predicting experimental results (with incentives)
Stage 3: open-ended question on prediction formation
Stage 4: closed-ended question on prediction formation

Treatment 6:
Stage 1: prior beliefs on average school spending
Stage 2: predicting experimental results (with anchor)
Stage 3: open-ended question on prediction formation
Stage 4: closed-ended question on prediction formation

Our design in the expert sample is structured as follows:
Stage 1: prior beliefs on average school spending
Stage 2: predicting experimental results (from the representative sample)
Stage 3: open-ended question on prediction formation
Stage 4: closed-ended question on prediction formation
Experimental Design Details
Not available
Randomization Method
Randomization is carried out by the survey company respondi, using a computer.
Randomization Unit
at the individual level
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
6,000 adults aged 18 – 69 years + 200 experts (economics professors)
Sample size: planned number of observations
6,000 adults aged 18 – 69 years + 200 experts (economics professors)
Sample size (or number of clusters) by treatment arms
6,000 adults aged 18 – 69 years, approx. 1,000 will be assigned to each of the treatment groups. 200 experts (economics professors)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number