LLMs, Compliance, and Decision-Making in Organizations: A Field Experiment

Last registered on June 29, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
LLMs, Compliance, and Decision-Making in Organizations: A Field Experiment
RCT ID
AEARCTR-0019059
Initial registration date
June 28, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 29, 2026, 9:43 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Yutong Chen
Affiliation
Frankfurt School of Finance and Management
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2026-06-28
End date
2026-11-01
Keywords
Agriculture, Behavior, Firms & Productivity, Labor
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project conducts a field experiment at a large-scale shrimp farm in China to examine how LLMs affect compliance in organizations. Workers receive AI-generated harvest recommendations, and in one treatment they can also interact with a LLM chatbot to explore the rationale behind the recommendations. Using daily pond-level data, we study how workers respond when AI recommendations conflict with managerial directives, focusing on whether LLMs improves compliance or instead provides a justification for deviation. We further test whether engagement with the AI system strengthens its behavioral impact and whether forecast errors weaken trust in AI. By linking behavioral, engagement, and economic outcomes, the study provides causal evidence on how LLMs may reshape coordination, authority, and control in organizations. The findings speak to the broader question of how information frictions within organizations undermine managerial control.
External Link(s)

Registration Citation

Citation
Chen, Yutong. 2026. "LLMs, Compliance, and Decision-Making in Organizations: A Field Experiment." AEA RCT Registry. June 29. https://doi.org/10.1257/rct.19059-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2026-07-01
Intervention End Date
2026-11-01

Primary Outcomes

Primary Outcomes (end points)
Primary outcome: worker compliance with the manager’s directive at the pond-day level, measured by the actual harvest decision and whether it matches or deviates from the manager’s instruction. The core endpoint is the binary daily harvest outcome for each pond.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will conduct a cluster-randomized field experiment at the pond level in a large shrimp-farm setting. The study has two treatment arms: in both arms, workers receive access to the same AI forecasting tool and numerical harvest recommendations as managers; in the second arm, workers also receive a chatbox interface that allows them to ask why and discuss the rationale behind the AI recommendation. The randomization unit is the pond, and all analyses will be clustered accordingly. Data are observed at the pond-day level over the production season, allowing us to measure whether workers comply with managerial directives when AI recommendations conflict with those directives. The intervention is implemented through a website using a fine-tuned forecasting model that integrates weather, hydrological, pond, and historical operational data.
Experimental Design Details
Not available
Randomization Method
Randomization will be conducted at the pond level, using a randomized assignment in the research office
Randomization Unit
Randomization unit: pond. The study randomizes treatment at the pond level, and standard errors are clustered at the pond level in analysis.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
12
Sample size: planned number of observations
67
Sample size (or number of clusters) by treatment arms
67
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Frankfurt School of Finance and Management
IRB Approval Date
2026-04-13
IRB Approval Number
N/A
Analysis Plan