LLM Agents Predicting the Valuation of Goods

Last registered on December 24, 2024

Pre-Trial

Trial Information

General Information

Title
LLM Agents Predicting the Valuation of Goods
RCT ID
AEARCTR-0014817
Initial registration date
November 12, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 15, 2024, 1:50 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 24, 2024, 11:31 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Stanford

Other Primary Investigator(s)

PI Affiliation
Stanford
PI Affiliation
Stanford

Additional Trial Information

Status
On going
Start date
2024-11-01
End date
2025-05-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study evaluates the capability of Large Language Models (LLMs) to predict individuals' valuation of digital and analog goods, specifically their willingness to accept (WTA) compensation for foregoing services like Facebook. Using survey data from Coyle and Nguyen (2023), which includes demographic information and WTA valuations for various goods, we investigate whether LLMs can simulate individual responses when provided with personal characteristics and prior valuations.

To further assess the applicability of LLMs in predicting valuations, we conducted an extension by surveying participants on Prolific using a new subset of digital and analog goods. We then replicate this survey with LLMs employing the same methodology. The comparison between human and LLM responses provides additional insights into the strengths and limitations of LLMs in capturing individual valuations.
External Link(s)

Registration Citation

Citation
Enriquez, Jose Ramon, Sophia Kazinnik and David Nguyen. 2024. "LLM Agents Predicting the Valuation of Goods." AEA RCT Registry. December 24. https://doi.org/10.1257/rct.14817-1.2
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2024-11-01
Intervention End Date
2025-05-31

Primary Outcomes

Primary Outcomes (end points)
The key outcome is the accuracy with which our set of LLMs replicate the survey outcomes.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We replicate the survey structure using a number of large language models, and compare the accuracy of the synthetic outcomes with the ones produces in the actual survey(s).
Experimental Design Details
Not available
Randomization Method
Randomization is applied within the survey structure and is replicated with our synthetic set-up.
Randomization Unit
Individual; this is a survey + synthetic replication.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
This is a survey at an individual level, there are around 10,000 observation in total (not all will be included/used in the main analysis)
Sample size: planned number of observations
This is a survey at an individual level, there are around 10,000 observation in total (not all will be included/used in the main analysis)
Sample size (or number of clusters) by treatment arms
This is a survey at an individual level, there are around 10,000 observation in total (not all will be included/used in the main analysis)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Stanford IRB
IRB Approval Date
2024-09-19
IRB Approval Number
IRB 21 (Registration 349)