AI vs human writing

Last registered on January 24, 2025

Pre-Trial

Trial Information

General Information

Title
AI vs human writing
RCT ID
AEARCTR-0014949
Initial registration date
December 02, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 22, 2025, 8:07 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
January 24, 2025, 10:52 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Middlebury College

Other Primary Investigator(s)

PI Affiliation
Bowdoin College

Additional Trial Information

Status
On going
Start date
2024-12-02
End date
2025-01-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We investigate the effect of human vs AI writing.
External Link(s)

Registration Citation

Citation
Abel, Martin and Reed Johnson. 2025. "AI vs human writing." AEA RCT Registry. January 24. https://doi.org/10.1257/rct.14949-1.1
Experimental Details

Interventions

Intervention(s)
We test how people value AI vs human generated creative writing.
Intervention (Hidden)
As we describe in more detail in our analysis plan, we have 2 treatments: One group is made to believe that a story is written by AI and one group is made to believe the story was written by a human.
This follows a commong re-labeling technique used in the literature.
Intervention Start Date
2024-12-03
Intervention End Date
2025-01-30

Primary Outcomes

Primary Outcomes (end points)
willingness to pay; writing assessment (More details are included in the pre-analysis plan.)
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We randomize people to read and assess a short story either written by AI or by humans. More details are included in the pre-analysis plan.
Experimental Design Details
Everyone is introduced to a (human) author. The treatment group will then receive the additional information that the story was in fact written by AI. Halfway through the story, we collect information about people's willingness to pay and their assessment of the story. We also measure their reading time. The treatment is the individual - there is no clustering.
Randomization Method
in survey software
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
no clusers
Sample size: planned number of observations
700 individuals
Sample size (or number of clusters) by treatment arms
350 per arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Bowdoin IRB
IRB Approval Date
2024-11-26
IRB Approval Number
2024-61
Analysis Plan

Analysis Plan Documents

PAP 2024_12_02

MD5: 6f557c5a5f5b0450a99c167de248d949

SHA1: 47e139751d6da1ae95dd97c88759f212e2499201

Uploaded At: December 02, 2024

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials