The What, When and Why of AI Use in Online Preference Elicitation Experiments

We conduct canonical versions of risk, time, and social preferences elicitation tasks. We deploy these tasks in conjunction with a treatment, providing treated participants with the option to chat with an LLM within the experiment.

We employ a between-subjects design, where the control group is asked to perform a series of decision-making tasks without explicit access to an LLM within the experiment. In contrast, the treatment group has access to an LLM chat window simultaneously and in the exact virtual location where the decision-making tasks occur.
Subjects are randomized into one of three groups: risk preferences, time preferences, or social preferences. The tasks in these groups are:
Risk:
6 Becker-DeGroot-Marschak (BDM) lottery valuation tasks
6 Gneezy-Potters (GP) investment tasks
Time:
6 Bernheim-Sprenger Convex time-budget (CTB) tasks
6 money-now-or-later tasks
Social:
Trust game (TG)
Ultimatum game (UG)
Dictator game (DG)

Subjects perform the tasks within their assigned group in a randomized order. The experiment is incentivized with a random problem selection mechanism following Azrieli, Chambers, and Healy (2018). The random problem selected is incentivized with a task-specific mechanism explained to the subject at the time of conduct.

2025-10-14

2025-12-16

Risk elicitation: Risk preference parameters (assuming CRRA) and absolute deviation from expected value.
Time preference elicitation: time preference parameters under hyperbolic discounting.
Social preferences: in trust game -- amount sent, reciprocity amounts, in dictator game -- amount sent, in ultimatum game -- proposer's offer and minimum acceptable offer.
The comparison between treatment and control group for these key variables constitutes the intent-to-treat effect of AI availability in online preference elicitation experiments.
The proportion of subjects utilizing the LLM chat across experiments, in conjunction with measures of task-specific subjective AI value, as a measure of elicitation method-induced subject confusion. The take-up of AI option allows us to construct the treatment-on--the-treated measure of AI impact on preference elicitation in online experiments.

For the risk tasks, we estimate preferences separately for each elicitation method. In the BDM lottery valuation task, we fit a structural CRRA utility model under expected utility, assuming probabilistic choice. In the GP investment task, we similarly estimate a CRRA utility model based on observed investment shares. In addition, we report reduced-form measures: for the BDM, the certainty equivalents implied by stated valuations, and for the GP task, the fraction of the endowment invested in the risky asset.

For both MEL and CTB tasks, we estimate time preferences using a structural CRRA utility framework with quasi-hyperbolic beta–delta discounting, assuming probabilistic choice. In addition, for MEL tasks we report reduced-form measures based on implied discount rates from switching points.

For all decisions, we elicit the level of certainty in the choice, the time to reach the decision, the perceived helpfulness of AI, and the belief about how intuitive the task was. We also elicit the subject's willingness to pay for AI to make all decisions in the task and their willingness to pay for the AI to revise the decisions made by the subject, both before and after the task's completion. For the treatment group, we record the use of the LLM (take-up rate and chat history).

We run three experimental sessions on Prolific: separately for risk, time, and social preferences elicitation. Participants are randomized in control (no LLM) or treatment (LLM offered) group.

We run three experimental sessions on Prolific: separately for risk, time, and social preferences elicitation. Participants are randomized in control (no LLM) or treatment (LLM offered) group. In the risk preferences elicitation, they are asked to evaluate a series of lotteries using the BDM mechanism and perform a Gneezy-Potters (1997) task. In the time preferences group, subjects participate in a series of convex time-budget tasks and a series of money-now-or-later choices. In the social preferences task, subjects are asked to complete the trust game, the ultimatum game, and the dictator game in a random order.
We elicit their willingness to pay for having AI make all the decisions for them or revise their decisions before the task (but after getting familiar with the rules) and after the task is completed. We also elicit their beliefs about the helpfulness of AI, their use of AI outside the experiment for the control group, or their use of AI beyond the one provided by the experimenter in the task.

Done by computer

Subject

No

NA

128 subjects for risk elicitation, 100 subjects for social preferences

Risk: 64 subjects in treatment, 64 subjects in control. Social preferences: 50 people in treatment, 50 people in control

0.5 standard deviation