Abstract
This study evaluates whether access to a natural-language large language model assistant improves consumer decision-making and scam avoidance in simulated online second-hand marketplaces. Adult participants recruited through Prolific will complete nine marketplace choice tasks. In each task, participants will evaluate five listings for a standardized used consumer electronic product and must select the listing they perceive to be the best available purchase.
Participants will be randomly assigned at the individual level to one of two conditions: an AI-assistance treatment condition or a no-assistance control condition. Participants in the treatment condition will be permitted to consult an LLM-based assistant of their choosing while evaluating the listings. Participants in the control condition will evaluate listings generated through the same prespecified procedures without access to an assistant or other external resources.
Listings will differ in prespecified characteristics associated with transaction risk, including seller reputation, urgency language, implausible discounts, off-platform communication or transaction requests, poor product photographs, and unsecured payment methods. The study is designed to determine whether AI assistance causes consumers to respond to these signals more similarly to a benchmark sample of hired scam experts.
The observed primary outcome is whether each displayed listing is selected. A conditional logistic-regression model will estimate the effects of the prespecified listing flags for consumers assigned to AI assistance, consumers assigned to the no-AI control condition, and the expert benchmark sample while accounting for the one-of-five choice structure of each task. Average marginal effects on listing-selection probability will be the preferred common-scale representation. For each flag, the analysis will test whether the average marginal effect differs between AI-assisted consumers, control consumers, and experts. These comparisons will indicate whether AI assistance shifts consumer responses toward the pattern observed among experts. Conditional-logit coefficients will be retained as a secondary representation.
Secondary outcomes include expert-consistent choice, decision confidence, decision time, assistant use, and prespecified treatment heterogeneity by marketplace experience, product familiarity, prior AI use, and other participant characteristics.
The principal consumer sample will contain approximately 100 participants, randomized approximately equally between the two conditions. The study will be administered through Qualtrics, with consumers recruited through Prolific and experts recruited through Upwork. The expert sample is a nonrandomized benchmark group rather than a third treatment condition.