Measuring Economic Determinants of AI Use in Online Data Markets

Last registered on April 29, 2026

Pre-Trial

Trial Information

General Information

Title
Measuring Economic Determinants of AI Use in Online Data Markets
RCT ID
AEARCTR-0018491
Initial registration date
April 29, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 29, 2026, 4:34 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Columbia University

Other Primary Investigator(s)

PI Affiliation
Columbia University
PI Affiliation
UIUC

Additional Trial Information

Status
In development
Start date
2026-04-29
End date
2026-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project assesses the extent to which bots and LLMs are used in the online market for survey responses, and how survey payments and performance-based bonuses affect adverse selection (low-quality or automated respondents entering the survey sample) and moral hazard (human workers using LLMs instead of providing authentic responses). We further test whether technical prevention tools and moral persuasion & penalties reduce LLM-generated responses.
External Link(s)

Registration Citation

Citation
Kuruvila, Tara Mary, Suresh Naidu and Lena Song. 2026. "Measuring Economic Determinants of AI Use in Online Data Markets." AEA RCT Registry. April 29. https://doi.org/10.1257/rct.18491-1.0
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2026-04-29
Intervention End Date
2026-12-31

Primary Outcomes

Primary Outcomes (end points)
• AI use as detected by each of our methods (Lasso, PCA, pre-specified binary)
• Performance
Primary Outcomes (explanation)
See pre-analysis plan

Secondary Outcomes

Secondary Outcomes (end points)
See pre-analysis plan
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We recruit approximately 3,000 participants aged 18–65 in the United States from Prolific, with comparison samples from other online survey platforms. Each participant completes a survey consisting of a screener and consent form, demographic questions, a battery of incentivized tasks, and a post-task section on self-reported AI use and attitudes. The task battery spans probability reasoning, beliefs and attitudes questions, open-ended summarization, and data labeling, reflecting the range of activities common in behavioral research and machine-learning data work. We cross-randomize four dimensions: base payment (high/low), bonus payment (none/low/high), prevention tools designed to discourage AI use (present/absent), and sanctions for AI use (none/moral persuasion and monetary penalty). Base payment is randomized at the study-listing level; the remaining dimensions are randomized at the individual level. To detect AI use, we combine behavioral measures (keystrokes, mouse movement, copy-paste activity, tab-switching, time on page), text-based AI detection on open-ended responses, and self-reported AI use, aggregated into pre-specified indices.
Experimental Design Details
Not available
Randomization Method
In qualtrics-survey randomization. Exception is base pay which is done at the listing level.
Randomization Unit
Individual; Listing-level for base-pay.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
ca. 4000 online survey participants across multiple platforms
Sample size: planned number of observations
ca. 4000 online survey participants across multiple platforms
Sample size (or number of clusters) by treatment arms
We cross-randomize four dimensions in a 2 × 3 × 2 × 2 factorial design (24 conditions): base payment (high/low, randomized at the study-listing level) and bonus payment (none/low/high), prevention tools (present/absent), and sanctions (none/moral persuasion and monetary penalty). There are an equal number of participants in each treatment cell.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Columbia University IRB
IRB Approval Date
2026-02-20
IRB Approval Number
AAAV8844(M00Y01)
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information