Generative AI & TeamWork: An experimental approach.

1. The introduction of AI, a Generative AI (GenAI) tool, to specific team compositions:
- T1: Human Employee + AI
- T2: Both Commercial and R&D functions work together with AI.

2. The assignment of participants to different team compositions:
- Control groups:
- Individual controls (not using AI)
- Commercial + R&D team without AI
- Treatment groups:
- T1: Human Employee + AI
- T2: Commercial + R&D + AI

Within each group, team members will be randomized. One team member will be designated as the leader, responsible for submitting the solution and leading the usage of AI by sharing their screen. The other team member will follow and contribute to the task. Teams will work remotely and transcribe their conversations during the hackathon.

After completing their initial task, participants in the control groups (both individual and team) will undergo GenAI training. They will then re-do the task, allowing for a within-participant comparison of performance before and after the training.

The interventions aim to study the effects of integrating AI on team dynamics, division of labor, and the potential substitution of certain functions within the context of a large multinational consumer product company. Participants will collaborate on real business problems using actual background data from various business units (Fabric Care, FemCare, Grooming, and Baby Care) during a one-day virtual hackathon.

2024-05-13

2024-07-24

Hackathon output.
Evaluations of the hackathon output.

Additionally, we are building outcomes from text:
GenAI prompts and responses: Collection of all the prompts given to the AI and the corresponding responses generated by the AI during the hackathon.
Transcripts of group interactions: Collection of the transcribed conversations among team members during the hackathon.

Hackathon output: Detailed business solutions to specific problems faced by the company's business units (Fabric Care, FemCare, Grooming, and Baby Care).
Evaluations of the output:

Three-level evaluations:
a. Workers trained by Business Unit (BU) leaders and rubrics on evaluating the solutions, possibly involving cross-BU evaluations.
b. Evaluation of the best solutions by BU leaders for final selection.
c. Complementary AI-generated evaluations of the solutions.

We will measure completion rates and timing for completing the task.

We are collecting survey question responses (see the Qualtrics flow).

Additionally, we are building outcomes from text:
GenAI prompts and responses: Collection of all the prompts given to the AI and the corresponding responses generated by the AI during the hackathon.
Transcripts of group interactions: Collection of the transcribed conversations among team members during the hackathon.

We will measure completion rates and timing for completing the task.

We are collecting survey question responses (see the Qualtrics flow).


Additionally, we are building outcomes from text:
GenAI prompts and responses: Collection of all the prompts given to the AI and the corresponding responses generated by the AI during the hackathon.
Transcripts of group interactions: Collection of the transcribed conversations among team members during the hackathon.

This study employs a randomized controlled trial design to investigate the effects of integrating Generative AI (GenAI) on team dynamics, division of labor, and performance in a large multinational consumer product company. Participants from Commercial and R&D functions will be randomly assigned to control and treatment groups to collaborate on real business problems during a one-day virtual hackathon.

Groups:
- Control groups:
- Individual controls (not using AI)
- Commercial + R&D team without AI
- Treatment groups:
- T1: Human Employee + AI
- T2: Commercial + R&D + AI

This study employs a randomized controlled trial design to investigate the effects of integrating Generative AI (GenAI) on team dynamics, division of labor, and performance in a large multinational consumer product company. Participants from Commercial and R&D functions will be randomly assigned to control and treatment groups to collaborate on real business problems during a one-day virtual hackathon.

Groups:
- Control groups:
- Individual controls (not using AI)
- Commercial + R&D team without AI
- Treatment groups:
- T1: Human Employee + AI
- T2: Commercial + R&D + AI

Participants will be randomly assigned to one of the following groups:
1. Individual controls (not using AI)
2. Commercial + R&D team without AI
3. Human Employee + AI (T1)
4. Commercial + R&D + AI (T2)

Within each group, team members will be randomized, with one designated as the leader responsible for submitting the solution and leading the usage of AI (if applicable). The treatment groups (T1 and T2) will have access to GenAI tools to assist them in collaborating and generating solutions to the business problems presented during the hackathon.

After completing the initial task, participants in the control groups will undergo GenAI training and re-do the task, allowing for a within-participant comparison of performance before and after the training.

Random assignment of participants to the intervention groups is achieved through a computerized random number generator, ensuring that the allocation is both random and concealed until the point of assignment.

The randomization happens within each unit/geography (Europe/Americas). We have 4 business units and 2 geographies, and thus 8 randomization clusters. The probability of assignment is equal across clusters.
1. Individual level randomization:
- Participants are randomly assigned to either the individual control group (not using AI) or one of the team-based groups (Commercial + R&D team without AI, T1: Human Employee + AI, or T2: Commercial + R&D + AI).
2. Within-team randomization:
- For the team-based groups (Commercial + R&D team without AI, T1: Human Employee + AI, and T2: Commercial + R&D + AI), team members are randomized within each group.
- One team member is designated as the leader, responsible for submitting the solution and leading the usage of AI (if applicable).

No

We are randomizing 852 participants. Additionally, we are including additional participants (participants in band 4+) who are assigned to treated individual group. Finally, participants coming in last minute will be accepted in the control individual group. We will use the additional participants for exploratory analyses.

We are randomizing 852 participants. Additionally, we are including additional participants (participants in band 4+) who are assigned to treated individual group. Finally, participants coming in last minute will be accepted in the control individual group. We will use the additional participants for exploratory analyses.

We are randomizing 852 participants. 364 are assigned to the paired control treatment (182 groups); 364 to the paired treatment (182 groups), 94 to the treated singleton; 30 to the control singleton.
Note that if one of the two paired members of a group does not show up, the other group member is automatically reassigned to the relevant singleton condition (either control or treatment based on the group assignment)