Coordination and Leadership: the impact of Artificial Intelligence

Last registered on September 29, 2024

Pre-Trial

Trial Information

General Information

Title
Coordination and Leadership: the impact of Artificial Intelligence
RCT ID
AEARCTR-0012434
Initial registration date
November 06, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 15, 2023, 4:03 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
September 29, 2024, 12:15 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Alma Mater Studiorum - Università di Bologna

Other Primary Investigator(s)

PI Affiliation
Alma Mater Studiorum - Università di Bologna

Additional Trial Information

Status
In development
Start date
2023-11-20
End date
2024-12-20
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We investigate the effect of Artificial Intelligence as a source of help for the leader of a group that is facing a coordination dilemma. In an on-line experiment, subjects play a one-shot minimum effort game with leadership. The group leader must send a short message to their teammates to enhance coordination. First, the leader writes the message, then they see ChatGPT's output for the same task and decide whether to send their own text or the one produced by the chatbot. Followers are informed whether the leader sent their own message or the one produced by ChatGPT, before making their decision. With this approach, we want to compare the differences in coordination levels and elicited beliefs between the groups with a human-written leader's message and those with an AI-generated one.
External Link(s)

Registration Citation

Citation
Bigoni, Maria and Damiano Paoli. 2024. "Coordination and Leadership: the impact of Artificial Intelligence." AEA RCT Registry. September 29. https://doi.org/10.1257/rct.12434-6.0
Experimental Details

Interventions

Intervention(s)
Participants will play a one-shot Minimum Effort Game with a leader. The leader has to send a message to the other group members, trying to entice them to choose the highest effort level. The leader can send them his/her own human-written message, or the message generated by ChatGPT4 for the exact same task.
Intervention Start Date
2023-11-20
Intervention End Date
2023-11-24

Primary Outcomes

Primary Outcomes (end points)
The first key outcome variable is the Individual Effort Level chosen by participants.
Given that participants will be divided in groups of 5, the second key outcome variable will be the Minimum Effort in the group, defined as the lowest Individual Effort Level chosen among group members.
This experiment aims to measure the Average Treatment Effect of receiving a human-written or AI-generated leader's message on the Individual Effort Level's choice.
We will also elicit the beliefs of Leaders and Followers about other group members' choices, in order to understand the potential mechanism behind the effect that we expect to detect.

Main hypotheses:
1. H0: Leadership with AI’s message does not worsen coordination with respect to leadership with human message.
H1: Leadership with AI’s message leads to a lower Pareto-ranked equilibrium.

2. H0: Followers do not differentiate between AI and Human message.
H1: Followers show algorithm aversion, choosing a lower effort level when the message is AI-generated, with respect to when it’s human-written.

3. H0: Subjects’ beliefs about others’ actions do not change based on whether the message is AI-generated or human-written.
H1: Leaders and/or followers update their beliefs when the message is AI-generated, expecting lower effort levels chosen by their teammates than when the message is human-written.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Given that the leader's choice between human and AI message is completely endogenous, we will observe leaders' preferences on it. We will check the quality of the text written by leaders and check if there is a significant difference between those who choose to keep their own text, and those who prefer ChatGPT's output.
The quality of the texts will be assessed by a different sample of subjects, who are not informed about the purpose of the experiment and who do not know whether the messages have been written by a human participant or by ChatGPT.
Moreover, we will ask leaders to what extent they think they have been convincing, how much they felt under pressure for the role they were given and how much they believe to be suited for a role of responsibility. Followers will also be asked about the leader's message persuasiveness.
Finally, we will study whether the leaders' and followers' behavior are affected by gender, familiarity and trust in AI, risk aversion, education level and standard demographics.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study is an asynchronous online survey experiment in which participants will answer questions without any real-time interaction with the others. The study is conducted entirely on Qualtrics.

Participants will play a one-shot Minimum Effort Game with leadership. The leader of the group has to send a message to the other group members; the message can be written by himself/herself or generated by Artificial Intelligence. Followers will receive the message before choosing their effort level.

We will elicit leaders' and followers' beliefs about the effort chosen by others.
Experimental Design Details
Not available
Randomization Method
Randomization will be done at the recruitment level through ORSEE.
We will identify a sample of eligible subjects and randomize them in the three different treatments.
Randomization Unit
Randomization at the individual level (participants in the experiment).
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will have 33 groups of 5 subjects that play the game in each treatment.
Sample size: planned number of observations
429 subjects. 64 subjects in the follow-up survey.
Sample size (or number of clusters) by treatment arms
165 subjects control, 132 AI message, 132 Human message, 64 in the follow-up survey.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
A one-tail Wilcoxon-Mann-Whitney test on the Individual Effort Level in the two treatments (AI message and Human message), including only followers observations (i.e. 132 per treatment), with alpha equal to 0.05 and power equal to 0.8, requires a minimum detectable Cohen's D equal to 0.314.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Comitato di Bioetica Alma Mater Studiorum - Università di Bologna
IRB Approval Date
2023-09-27
IRB Approval Number
0289950
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information