Production with and by AI

Last registered on July 04, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
Production with and by AI
RCT ID
AEARCTR-0019010
Initial registration date
June 24, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 29, 2026, 9:20 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
July 04, 2026, 2:50 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
United States of America
Region

Primary Investigator

Name
John Conlon
Affiliation
Carnegie Mellon University
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2026-07-06
End date
2026-07-17
Keywords
Behavior
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Our experiment involves participants designing a product (a web game) with an AI coding agent to sell to other participants.
External Link(s)

Registration Citation

Citation
Conlon, John. 2026. "Production with and by AI." AEA RCT Registry. July 04. https://doi.org/10.1257/rct.19010-1.2
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Participants design a product (a web game) with an AI coding agent. We also vary whether and when participants see the AI design the game itself (i.e., without human input).
Intervention (Hidden)
Participants design three games, starting with a base game and using an AI coding agent to improve or redesign it. They do so over the course of three rounds for each game, where they send a message to the AI, and it re-codes the game in response to what they say. Throughout we collect participants WTP to keep designing the game rather than give up, as well as pre- and post-design predictions of their "customer's" WTP to play the game. We also elicit their WTP to keep the game (be sent a durable link to it) as well as to include it in a public gallery that other participants can see. Finally we elicit likert pride and enjoyment measures, as well as participants' beliefs about their ability relative to other participants.

In the customer phase (which happens after the design phase) participants express their WTP to play another participants' (or the AI's) game, relative to a basic game.

There are three treatment groups, assigned randomly. The first treatment group sees the AI design a game after they design their first game. The second treatment group sees the AI design a game only after they design their second game. The final treatment group does not see the AI design a game at all.

We also cross-randomize whether participants are told which games they are offered in the customer phase were designed by other participants or by the AI.
Intervention Start Date
2026-07-06
Intervention End Date
2026-07-17

Primary Outcomes

Primary Outcomes (end points)
WTPs (predicted and actual), pride and enjoyment measures, beliefs about ability.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
See above.
Experimental Design Details
See above.
Randomization Method
By a computer.
Randomization Unit
Person level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
300 participants.
Sample size: planned number of observations
300 participants, split across three treatment groups. The first treatment group sees the AI design a game after they design their first game. The second treatment group sees the AI design a game only after they design their second game. The final treatment group does not see the AI design a game at all. We exclude (by asking to return the study) participants who give more than three incorrect answers to comprehension questions.
Sample size (or number of clusters) by treatment arms
In expectation: 100 see the AI design after they design their first game, 100 only after they design their second game, and 100 not at all.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Carnegie Mellon University
IRB Approval Date
2026-05-07
IRB Approval Number
IRBSTUDY2016_000000482
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials