Recommender Systems and Consumer Choice - Experimental Evidence

Each subject completes a risk elicitation task. Following this task, each subject is asked to make choices from ordered lists of three outcome lotteries for 30 rounds. The lists of lotteries are created by two different algorithms. Subjects are aware of the fact that two different algorithms create their choice lists. After 20 rounds of the choice task, subjects are asked to disclose their willingness to pay to increase the probability that one or the other algorithm creates the lists of lotteries for the remaining 10 choice rounds.

2024-07-15

2024-08-05

- Expected value of chosen lotteries
- Expected utility of chosen lotteries
- List rank of chosen lotteries
- Willigness-to-pay for expected utility algorithm

- Expected utility will be calculated under the assumption of constant relative risk aversion (CRRA)

- Risk aversion
- Search costs

- Risk aversion will be calculated using the subjects choice in the first part of the experiment
- Search costs will be estimated with a logit model using the rank and expected utility of all chosen and non-chosen lotteries that the subject sees during the choice tasks

The experiment features three kinds of tasks. The first task is a risk preference elicitation task in the style of Johnson et al. (2021), where subjects have the prospect of gaining either the outcome of a lottery that pays 10$ with probability 50% and 0$ with probability 50% or a sure but unknown monetary amount X, that has been randomly drawn with equal probability between 0$ and 10$. Subjects are asked to declare a threshold
such that they would prefer receiving X if it is above the threshold, and they would prefer receiving the lottery if X is below the threshold.

The second task is a choice task, where subjects are asked to choose from lists of three-outcome lotteries that have been ordered and selected by one of two algorithms. These algorithms take into account the subjects risk aversion that has been calculated using the subjects threshold choice in the first task and assuming constant relative risk aversion. The outcomes of all lotteries are 10$, 5$, and 0$, and the respective probabilities are drawn to ensure that the probability of 10$ and 0$ does not exceed 50%.

Finally, I elicit subjects willingness to pay for one algorithm over the other. For this step, I give subjects an endowment of 1$. Using a slider, subjects can change the probability that they will face either algorithm in the next set of 10 choice rounds. Choosing equal probability between the two algorithms (50%-50%) is costless. Changing the probabilities by 1%, i.e. to 49% - 51% costs 0.02$. The cost increases linearly, so that the price of setting the probability to have either algorithm for the last 10 choice rounds is equal to the full endowment - 1$.

Timing:
• t=1
– General Iinstructions about the experiment
– Instructions on the risk elicitation task
– Instructions on the lottery choice task
– Instructions on theWTP-elicitation mechanism
• t=2
– Risk elicitation task
• t=3
– 10 rounds of choice task with platform or expected utility algorithm (order of algorithms is randomized)
• t=4
– 10 rounds of choice task with platform or expected utility algorithm (order of algorithms is randomized)
• t=5
– Willingness-to-pay elicitation
• t=6
– 10 rounds of choice task with either random or personalized algorithm, depending on random draw with probabilities determined in the willingness-to-pay elicitation round

Randomization done on the server by a computer

Individual

No

no clustered randomization

300 individuals

300 individuals (within subject treatment design)

Analysis Plan