Elicitation of preferences and beliefs in light of new information: a laboratory experiment
Last registered on March 14, 2019

Pre-Trial

Trial Information
General Information
Title
Elicitation of preferences and beliefs in light of new information: a laboratory experiment
RCT ID
AEARCTR-0003815
Initial registration date
February 08, 2019
Last updated
March 14, 2019 5:13 AM EDT
Location(s)
Primary Investigator
Affiliation
University of Bristol
Other Primary Investigator(s)
PI Affiliation
Warwick Business School
PI Affiliation
University of Heidelberg
Additional Trial Information
Status
Completed
Start date
2019-02-11
End date
2019-02-15
Secondary IDs
Abstract
Bayesian Updating is the dominant theory of learning in economics and other disciplines. According to this theory, decision-makers have prior beliefs which they update according to Bayes rule after receiving new information. The theory is silent about how individuals react to receiving information that the subjects have not observed previously and, hence, which they may deem impossible. Recent theoretical literature has put forth a possible mechanism, called “reverse Bayesianism”, which decision-makers may use to react to unforeseen events. However, these and other possible mechanisms have not been experimentally tested. The project will fill this large gap in the literature by running a series of economic experiments.

We started to explore this matter in a previous experiment (AEARCTR-0003030) using Amazon MTurk to obtain some preliminary evidence in understanding decision-making behaviour in such situations. Given these findings, we fine-tuned our experimental design to implement in a lab setting. However, we found a large variation in the elicited Willingness to Accept with several unrealistically high and unrealistically low values. This could possibly be because MTurk subjects, notoriously less attentive than laboratory subjects, did not fully understand our methods of elicitation. A further important concern is that using a virtual urn instead of a real one may have induced the subjects to question the randomness of the draws they observed. This laboratory experiment is set to overcome both of the above concerns.

A summary description of our design follows. First, a randomly chosen subject will be asked to repeatedly draw balls from an urn in front of all other subjects in each experimental session. This will allow subjects to form beliefs about the proportion of different prizes in the urn. Subsequently, subjects’ beliefs about the composition of the urn and their willingness to accept for lotteries rewarding them according to draws out of urn are elicited. At this point, a new urn with several balls representing one new prize is presented and the subject responsible to make draws is asked to draw from the new urn. Subsequently, the contents of the new urn are poured into the old one (with the explicit note that the new urn does not contain any previously observed prizes) and subjects are told that they will be paid according to the draws from the resulting combined urn. Finally, we elicit subjects’ willingness to accept for a lottery rewarding them according to a draw out of the urn again (based on observations of drawings from the old urn and the subsequent draw from the combined urn) as well as their perceived likelihoods of each prize.

We have two "surprise" treatments and two control treatments. In the "good surprise" treatment the new prize will be higher than the one observed in the "bad surprise" treatment. In the control treatments, we reveal ex-ante the urn compositions.

This design will allow us to explore how adding an unexpected prize changes subjects’ beliefs about the composition of the urn. We will in particular address the following questions. Do individuals naturally expect events that were previously considered impossible? Do individuals learn to expect what is previously considered impossible? How individuals update their beliefs about the urn composition after an unexpected event takes place? Do individuals attach a positive or negative value to the consequences of such events?
External Link(s)
Registration Citation
Citation
Melkonyan, Tigran, Eugenio Proto and Andis Sofianos. 2019. "Elicitation of preferences and beliefs in light of new information: a laboratory experiment." AEA RCT Registry. March 14. https://www.socialscienceregistry.org/trials/3815/history/43339
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2019-02-11
Intervention End Date
2019-02-15
Primary Outcomes
Primary Outcomes (end points)
(i) The subjective probability of an unknown event (or prize) in the original urn for treated group
(ii) The subjective probability of an unknown event (or prize) in the updated urn for treated group
(iii) The ratio between subjective probabilities of the two prizes observed respectively in the original urn for the treated group
(iv) The ratio between subjective probabilities of the prizes observed respectively in the updated urn for the treated group
(v) The ratio between subjective probabilities of the two prizes observed respectively in the urn for the control group
(vi) The willingness to accept to sell the original lottery in the treated group
(vii) The willingness to accept to sell the updated lottery in the treated group
(viii) The willingness to accept to sell the lottery in the control group
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
(i) The subjective expected value of the extracted prizes of the original and updated urn
(ii) The subjective expected utility of the extracted prizes of the original and updated urn
iii) Coefficient of risk aversion
iv) Result of the Raven tests
Secondary Outcomes (explanation)
(i) The subjective expected value of the extracted prizes : is the expected values calculated using the subjective likelihood of each prize multiplied by the value of each prize
(ii) The subjective expected utility of the extracted prizes : is the expected values calculated using the subjective likelihood of each prize multiplied by the utility of each prize. The utility is calculated assuming a CRRA utility function where the coefficient of risk aversions are elicited using Eckel and Grossman method
iii) elicitation based on Eckel and Grossman method
Experimental Design
Experimental Design
We design an experiment where subjects are asked to report subjective probabilities of prizes and the certainty equivalent values of lotteries determined by drawing prizes from a real urn, from which they observe several realisations prior to making their decisions. These elicitations are incentivised by using methods similar to Becker-DeGroot-Marschak (BDM) to report the subjective probability of prizes and their willingness to accept to sell the lotteries (see the enclosed documents for more details).

There are two main treatments for the main part of the design:

A) In the control treatments:

1) Subjects in the laboratory observe 20 draws from a real urn containing 24 balls with prize B=80 TOKENS and 36 balls with prize C=190 TOKENS. This urn is called the “original urn” and they are told that this urn contains two and only two possible prizes . They need to report their subjective probabilities of the two prizes and their willingness to accept for selling the lottery determined by a random draw from the urn.
2) We bring the new urn, draw one prize and say on PC Screen: "This urn contains only this prize. Please click OK to confirm you understand this". In the bad control, the new prizes are 15 balls labelled with 15 tokens, while in good control there are 15 balls labelled with 375 tokens. Now we drop the contents of the original urn into the new urn which together form the updated urn. Subjects need to report their subjective probabilities of the three prizes and their willingness to accept for selling the lottery determined by a random draw from the updated urn.

B) In the surprise treatments:

1) In the surprise treatment: Subjects observe 20 draws from a urn -- that we call "original urn". The urn contains 24 balls with prize B=80 TOKENS and 36 balls with prize C=190 TOKENS. They are asked to report their subjective probabilities of the two prizes and their willingness to accept for selling the lottery determined by a random draw from the original urn.
2) At this point we bring the new urn, draw one prize and say on PC Screen: "This urn contains new prizes. One such prize is the one you see. The urn contains no prizes similar to what you have been observing as a result of random draws from the other urn. Please click OK to confirm you understand this." In bad surprise the new prizes are 15 balls labelled with 15 tokens, while in good surprise there are 15 balls labelled with 375 tokens. Now we drop the contents of the original urn into the new urn which together form the updated urn. Subjects are asked again to report their subjective probabilities of the three prizes and their willingness to accept for selling the lottery determined by a random draw from the modified urn.

After the main part is completed we elicit the level of risk aversion using Eckel and Grossman method.

Each session ends with a short (non-incentivized) Raven test and some general demographic questions.
Experimental Design Details
Randomization Method
Done by a computer and manual extraction from an urn
Randomization Unit
individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
350 laboratory participants
Sample size: planned number of observations
350 laboratory participant
Sample size (or number of clusters) by treatment arms
75 participants: bad surprise control
75 participants: good surprise control
100 participants: bad surprise treatment
100 participants: good surprise treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Humanities and Social Sciences Research Ethics Committee
IRB Approval Date
2019-01-11
IRB Approval Number
HSS 49/18-19
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Yes
Data Collection Completion Date
February 15, 2019, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
331 laboratory participants
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
331 laboratory participants
Final Sample Size (or Number of Clusters) by Treatment Arms
74 laboratory participants: bad surprise control 71 laboratory participants: good surprise control 88 laboratory participants: bad surprise treatment 98 laboratory participants: good surprise treatment
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers