The American Economic Association's registry for randomized controlled trials
Elicitation of preferences and beliefs in light of new information: a laboratory experiment
Last registered on February 10, 2019
View Trial History
Elicitation of preferences and beliefs in light of new information: a laboratory experiment
Initial registration date
February 08, 2019
February 10, 2019 7:47 PM EST
United Kingdom of Great Britain and Northern Ireland
University of Bristol
Contact Primary Investigator
Other Primary Investigator(s)
Warwick Business School
University of Heidelberg
Additional Trial Information
Bayesian Updating is the dominant theory of learning in economics and other disciplines. According to this theory, decision-makers have prior beliefs which they update according to Bayes rule after receiving new information. The theory is silent about how individuals react to receiving information that the subjects have not observed previously and, hence, which they may deem impossible. Recent theoretical literature has put forth a possible mechanism, called “reverse Bayesianism”, which decision-makers may use to react to unforeseen events. However, these and other possible mechanisms have not been experimentally tested. The project will fill this large gap in the literature by running a series of economic experiments.
We started to explore this matter in a previous experiment (AEARCTR-0003030) using Amazon MTurk to obtain some preliminary evidence in understanding decision-making behaviour in such situations. Given these findings, we fine-tuned our experimental design to implement in a lab setting. However, we found a large variation in the elicited Willingness to Accept with several unrealistically high and unrealistically low values. This could possibly be because MTurk subjects, notoriously less attentive than laboratory subjects, did not fully understand our methods of elicitation. A further important concern is that using a virtual urn instead of a real one may have induced the subjects to question the randomness of the draws they observed. This laboratory experiment is set to overcome both of the above concerns.
A summary description of our design follows. First, a randomly chosen subject will be asked to repeatedly draw balls from an urn in front of all other subjects in each experimental session. This will allow subjects to form beliefs about the proportion of different prizes in the urn. Subsequently, subjects’ beliefs about the composition of the urn and their willingness to accept for lotteries rewarding them according to draws out of urn are elicited. At this point, a new urn with several balls representing one new prize is presented and the subject responsible to make draws is asked to draw from the new urn. Subsequently, the contents of the new urn are poured into the old one (with the explicit note that the new urn does not contain any previously observed prizes) and subjects are told that they will be paid according to the draws from the resulting combined urn. Finally, we elicit subjects’ willingness to accept for a lottery rewarding them according to a draw out of the urn again (based on observations of drawings from the old urn and the subsequent draw from the combined urn) as well as their perceived likelihoods of each prize.
We have two "surprise" treatments and two control treatments. In the "good surprise" treatment the new prize will be higher than the one observed in the "bad surprise" treatment. In the control treatments, we reveal ex-ante the urn compositions.
This design will allow us to explore how adding an unexpected prize changes subjects’ beliefs about the composition of the urn. We will in particular address the following questions. Do individuals naturally expect events that were previously considered impossible? Do individuals learn to expect what is previously considered impossible? How individuals update their beliefs about the urn composition after an unexpected event takes place? Do individuals attach a positive or negative value to the consequences of such events?
Melkonyan, Tigran, Eugenio Proto and Andis Sofianos. 2019. "Elicitation of preferences and beliefs in light of new information: a laboratory experiment." AEA RCT Registry. February 10.
Sponsors & Partners
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
(i) The subjective probability of an unknown event (or prize) in the original urn for treated group
(ii) The subjective probability of an unknown event (or prize) in the updated urn for treated group
(iii) The ratio between subjective probabilities of the two prizes observed respectively in the original urn for the treated group
(iv) The ratio between subjective probabilities of the prizes observed respectively in the updated urn for the treated group
(v) The ratio between subjective probabilities of the two prizes observed respectively in the urn for the control group
(vi) The willingness to accept to sell the original lottery in the treated group
(vii) The willingness to accept to sell the updated lottery in the treated group
(viii) The willingness to accept to sell the lottery in the control group
Primary Outcomes (explanation)
Secondary Outcomes (end points)
(i) The subjective expected value of the extracted prizes of the original and updated urn
(ii) The subjective expected utility of the extracted prizes of the original and updated urn
iii) Coefficient of risk aversion
iv) Result of the Raven tests
Secondary Outcomes (explanation)
(i) The subjective expected value of the extracted prizes : is the expected values calculated using the subjective likelihood of each prize multiplied by the value of each prize
(ii) The subjective expected utility of the extracted prizes : is the expected values calculated using the subjective likelihood of each prize multiplied by the utility of each prize. The utility is calculated assuming a CRRA utility function where the coefficient of risk aversions are elicited using Eckel and Grossman method
iii) elicitation based on Eckel and Grossman method
We design an experiment where subjects are asked to report subjective probabilities of prizes and the certainty equivalent values of lotteries determined by drawing prizes from a real urn, from which they observe several realisations prior to making their decisions. These elicitations are incentivised by using methods similar to Becker-DeGroot-Marschak (BDM) to report the subjective probability of prizes and their willingness to accept to sell the lotteries (see the enclosed documents for more details).
There are two main treatments for the main part of the design:
A) In the control treatments:
1) Subjects in the laboratory observe 20 draws from a real urn containing 24 balls with prize B=80 TOKENS and 36 balls with prize C=190 TOKENS. This urn is called the “original urn” and they are told that this urn contains two and only two possible prizes . They need to report their subjective probabilities of the two prizes and their willingness to accept for selling the lottery determined by a random draw from the urn.
2) We bring the new urn, draw one prize and say on PC Screen: "This urn contains only this prize. Please click OK to confirm you understand this". In the bad control, the new prizes are 15 balls labelled with 15 tokens, while in good control there are 15 balls labelled with 375 tokens. Now we drop the contents of the original urn into the new urn which together form the updated urn. Subjects need to report their subjective probabilities of the three prizes and their willingness to accept for selling the lottery determined by a random draw from the updated urn.
B) In the surprise treatments:
1) In the surprise treatment: Subjects observe 20 draws from a urn -- that we call "original urn". The urn contains 24 balls with prize B=80 TOKENS and 36 balls with prize C=190 TOKENS. They are asked to report their subjective probabilities of the two prizes and their willingness to accept for selling the lottery determined by a random draw from the original urn.
2) At this point we bring the new urn, draw one prize and say on PC Screen: "This urn contains new prizes. One such prize is the one you see. The urn contains no prizes similar to what you have been observing as a result of random draws from the other urn. Please click OK to confirm you understand this." In bad surprise the new prizes are 15 balls labelled with 15 tokens, while in good surprise there are 15 balls labelled with 375 tokens. Now we drop the contents of the original urn into the new urn which together form the updated urn. Subjects are asked again to report their subjective probabilities of the three prizes and their willingness to accept for selling the lottery determined by a random draw from the modified urn.
After the main part is completed we elicit the level of risk aversion using Eckel and Grossman method.
Each session ends with a short (non-incentivized) Raven test and some general demographic questions.
Experimental Design Details
Done by a computer and manual extraction from an urn
Was the treatment clustered?
Sample size: planned number of clusters
350 laboratory participants
Sample size: planned number of observations
350 laboratory participant
Sample size (or number of clusters) by treatment arms
75 participants: bad surprise control
75 participants: good surprise control
100 participants: bad surprise treatment
100 participants: good surprise treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
INSTITUTIONAL REVIEW BOARDS (IRBs)
Humanities and Social Sciences Research Ethics Committee
IRB Approval Date
IRB Approval Number
Post Trial Information
Is the intervention completed?
Is data collection complete?
Is public data available?
Reports and Papers