Stochastic Choice and Preference (Im)Precision: An Online Replication Condition and A New Treatment Condition

2023-09-04

2024-01-09

The primary outcomes are stochastic choices in distant repetitions; stochastic choices in repetitions in a row; preference (im)precision measurements.

Our approach to measuring key primary outcomes will mirror that of Agranov & Ortoleva (2017), specifically in relation to stochastic choices during distant and consecutive repetitions. To gauge preference (im)precision, we will ask subjects the following question: “You chose Option [X] over Option [Y]. How sure are you that you prefer Option [X]?” Subjects can then select from the following responses: “I’m not sure at all, I’m somewhat unsure, I’m neither sure nor unsure, I’m somewhat sure, or I’m completely sure."

The experimental design consists of two treatments: a (i) replication treatment and (ii) preference (im)precision treatment.

The replication treatment will closely follow Parts I-IV of Agranov & Ortoleva's original experiment. In our study, Part IV will be followed by two additional sections, Part V and Part VI. These parts are randomised (in Qualtrics) and will either be a (i) 'Preference Imprecision Elicitation’ (PIE) or (ii) ‘Post Test’ (PT)

The PIE consists of ten gambles, namely FOSD1, FOSD2, FOSD3, EASY1, EASY2, EASY3, HARD1, HARD2, HARD3, and HARD4 (refer to Agranov & Ortoleva, 2017, for detailed descriptions of these gambles). After each decision within a gamble, subjects are posed the same preference imprecision question mentioned earlier, inquiring about the certainty of their choice.

The PT section is made up of seven gambles (i.e., FOSD1, EASY1, EASY2, HARD1, HARD2, HARD3, HARD4). Here, subjects must decide between any two pairs of lotteries in a given gamble as well as their convex combination. Thus, if lottery 1 is represented by A, lottery 2 by B, their convex combination is denoted AB. Subjects must then choose between A, B, and AB. Moreover, we systematically introduce a coin-flip for different options, resulting in a total of 28 choices for this section.

The preference (im)precision treatment will encompass Parts I-VI, as in the replication treatment, with the exception that in Part I, preference (im)precision measurements will be collected for the first time subjects encounter each of the ten gambles (without a coin-flip), and then when they will face the remaining 30 lottery decisions coin-flip option (again see Agranov & Ortoleva, 2017, for details) will always be available. This time, there is no cost imposed for the coin-flip.

Both treatments will conclude with a questionnaire, closely mirroring the one administered in Agranov & Ortoleva (2017).

Additionally, we will implement a Memory Test, asking subjects to recall if they have seen a lottery or not: 12 have been seen before, and another 12 are new. The ones previously seen are from FOSD1, FOSD3, EASY1, EASY2, HARD1, HARD3. The new ones are selected to have the same expected value as the 12 previously seen. This memory test is incentive-compatible, as subjects are asked to estimate the accuracy of their responses. If their estimate is within a 10% margin of their actual correct guesses, they receive an additional bonus fee.

References

Agranov, M., & Ortoleva, P. (2017). Stochastic choice and preferences for randomization. Journal of Political Economy, 125(1), 40-68.

The study will be conducted using Prolific Academic and Qualtrics for our data collection.

We target a sample of N = 120 in each treatment cell. There will be two treatments: a replication treatment and a preference (im)precision treatment. Overall, we plan to recruit 240 subjects in total.

The original study by Agranov & Ortoleva (2017) was conducted over four lab sessions with an overall sample of 80 subjects. However, our study will differ from the original study in a number of dimensions (e.g., population; lab vs. online experiment; US dollar vs. British pound payments) and thus we have conservatively increased the sample size to 120 subjects per treatment, representing an increase of 50% in relation to the original (treatment) sample.

Regarding payments, subjects in the original experiment earned an average of $19, equating to an hourly earnings of $25. In line with current exchange rates, we plan to pay our subjects £20 per hour, a sum considered ‘great’ pay on Prolific.

Randomisation into treatments:

Subjects are block-randomised into one of the two treatments using the “Randomiser” tool within Qualtrics. Within each treatment, there are two orders to which subjects are randomly assigned (for more details, see Agranov & Ortoleva, 2017). For Parts V and VI, subjects are also randomly assigned to two predetermined orders that have been determined by a random number generator in Stata 18 software.

Lottery randomisation:

In addition to a show-up fee, subjects have the opportunity to earn a substantial bonus based on their decisions in the several gambles at each stage. We use custom JavaScript code in Qualtrics to randomly determine the outcomes of chosen gambles.

This code randomly selects a question from:
(i) Parts I or III, and then plays out the lottery a subject has chosen in that question;
(ii) Parts V or VI, and plays out the lottery a subject has chosen in that question.

Moreover, for each of the questions in Parts II and IV, the code plays out the lotteries and sums up all earnings across these parts. All earnings are represented as Tokens, which are then converted to Pounds (£) at a predetermined exchange rate.

Individuals

No

240 Prolific subjects.

240 Prolific subjects.

120 Prolific subjects per treatment.