Human AI Collaboration Preferences

Last registered on September 26, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
Human AI Collaboration Preferences
RCT ID
AEARCTR-0014311
Initial registration date
September 18, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 26, 2024, 12:25 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
Yuqing Ren
Affiliation
University of Minnesota
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Teng Ye
PI Affiliation
University of Minnesota
Contact Investigator
PI Name
Jundong Zhang
PI Affiliation
University of Minnesota
Contact Investigator

Additional Trial Information

Status
In development
Start date
2024-09-16
End date
2024-10-15
Keywords
Behavior
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In this trial, we hope to study human preferences with how they collaborate with Generative AI tools in performing writing tasks. The study will be posted as a survey on online platforms, such as Amazon Mechanical Turk and Prolific. We will be varying the nature of the task and level of difficulty and observe how they affect the collaboration mode that humans choose to work with AI.
External Link(s)

Registration Citation

Citation
Ren, Yuqing, Teng Ye and Jundong Zhang. 2024. "Human AI Collaboration Preferences." AEA RCT Registry. September 26. https://doi.org/10.1257/rct.14311-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
We have two main sets of interventions: task type and task difficulty.
Intervention (Hidden)
Intervention Start Date
2024-09-16
Intervention End Date
2024-10-15

Primary Outcomes

Primary Outcomes (end points)
Our main outcome is human preferences for AI collaboration modes, that is, whether to work with AI and how to work with AI.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will vary different types of writing tasks and the level of difficulty of the task and ask participants their preferences of whether and how to collaborate with a Generative AI agent, similar to Chat GPT. We will also collect data on the background and demographics of the participants to see if these factors also affect their preferences.
Experimental Design Details
We choose the context of a project manager in an AI company who is working with a fast food chain on an AI drive-through project. We vary two types of writing tasks: writing a report to the business client vs. writing an email to project team members. We also vary the sentiment and level of difficulty of the tasks by manipulating the outcome of the project (a success with increased revenue vs. a failure with decreased revenue) and the amount of instructions we provide (little, some template, template with detailed examples).

In addition, we will ask participants about their writing abilities, experiences with Generative AI, and demographics and consider these factors as covariates and controls in predicting human preferences.
Randomization Method
Randomization to experimental conditions will be done in Qualtrics.
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
300-500 Amazon Mechanical Turk workers
Sample size: planned number of observations
300-500 observations
Sample size (or number of clusters) by treatment arms
300-500 Amazon Mechanical Turk Workers as one cluster
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Minnesota Institutional Review Board
IRB Approval Date
2024-02-01
IRB Approval Number
STUDY00021153
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials