Anthropomorphism of Artificial Intelligence (AI)

Last registered on June 30, 2023

View Trial History

Pre-Trial

Trial Information

General Information

Title

Anthropomorphism of Artificial Intelligence (AI)

RCT ID

AEARCTR-0010921

Initial registration date

May 03, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

May 03, 2023, 4:39 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

June 30, 2023, 12:13 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name

Nanyin Yang

Affiliation

Texas A&M University

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Andreas Drichoutis

PI Affiliation

Agricultural University of Athens

Contact Investigator

PI Name

Marco Palma

PI Affiliation

Texas A&M University

Contact Investigator

Additional Trial Information

Status

In development

Start date

2023-05-07

End date

2024-05-31

Keywords

Behavior

Additional Keywords

authority-delegation game, online experiment

JEL code(s)

C90, D82

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

Experiment 1:
This experiment will study how the anthropomorphism of Artificial Intelligence (AI) (i.e., the gendered design) affects human users' trust and delegation of decision-making to an AI assistant.

This experiment implements an authority-delegation game following Fehr et al. (2013), in which subjects are matched with an assistant that can aid them to search for information for a card-picking decision.

In this game, there are 35 cards with four types: 1 Green Card, 1 Blue Card, 1 Red Card, and 32 Blank Cards. The followings are returns from each type of card:
(1) Blank Card: 0 token
(2) Green Card: 10 tokens
(3) Blue Card: 24 tokens
(4) Red Card: 40 tokens
Subjects will be presented with 35 cards as a five-by-seven matrix, with one Green Card, one Blue Card, one Red Card, and 32 Blank Cards. Initially, all cards are hidden and only one Green Card's position is always visible at position 18. Subjects need to conduct an information search. A successful search will uncover the positions of all colored cards and pick the most profitable card.

There are two ways of information search: (1) self-search; (2) delegating the search to an assistant.
(1) Self-search: subjects need to choose a search intensity to conduct a self-search. This search intensity is the probability for the search to be successful (i.e., all cards' positions are uncovered), associated with a cost of tokens following this function: cost = 25*(search intensity)^2. If a successful search occurs, subjects will be able to freely pick a card from all 35 cards with revealed colors.
(2) Delegating the search to an assistant: the assistant will conduct a search with a fixed search intensity (depending on the treatment, explained later). This search is cost-free, but once a successful search occurs, the assistant will automatically pick the Blue Card.

This game will repeat for 20 rounds divided into two blocks (of ten rounds each), occurring in random order. The search intensity of the assistant is 60% in one block and 80% in another. Subjects will be informed about this search intensity in a block only when they enter this block.

After the advice-following game, subjects will complete the following tasks: a lottery-choice decision designed by Eckel & Grossman (2000); a brief implicit association test on gender bias; a questionnaire surveying subjects' perception of assistant's gender, their daily use of virtual assistants, and their trust in others.

This experiment contains 6 treatments, in a between-subjects design.
Treatment 1: the assistant is a pre-programmed virtual assistant.
Treatment 2: the assistant is a pre-programmed virtual assistant, with the name "Mary".
Treatment 3: the assistant is a pre-programmed virtual assistant, with the name "James".
Treatment 4: the assistant is a human, with search data pre-collected from a real human subject in the Human Behavior Lab. The gender is not revealed.
Treatment 5: the assistant is a female, with search data pre-collected from a real female subject in the Human Behavior Lab. This subject will pick a fictitious name to present herself.
Treatment 6: the assistant is a male, with search data pre-collected from a real male subject in the Human Behavior Lab. This subject will pick a fictitious name to present himself.

In addition, we will conduct a lab session in the Human Behavior Lab to collect the data for the human assistant. Subjects will be invited to the lab to conduct 20 rounds of information search, divided into two blocks of ten rounds. In each block, they can choose a search intensity of either 60% or 80% and conduct the 10 trials of search with this chosen intensity. Their search results will be recorded. And they need to report their gender and pick a fictitious name from a given list of names to represent themselves. They will be informed that their data might be used for future research.

Experiment 2:
This study is a follow-up to Experiment 1. Subjects will still participate in an advice-following game, but the procedure is slightly different and it is designed to answer the question of whether interactions with gendered VA pose concerns for enforcing stereotypes and biases in subsequent interactions with humans.

The game will repeat for 20 rounds divided into two blocks of ten rounds. In Block 1, subjects will be interacting with a virtual assistant, whose characteristics vary by treatment conditions (explained later). In Block 2, subjects will choose among one of four human assistants (two male names and two female names) whose search intensities are always high-quality, 80%, and then play the games with their chosen human assistant.

This experiment contains 6 treatments, in a between-subject design:
Treatment 1: The virtual assistant in Block 1 does not have a name, with search intensity of 60%.
Treatment 2: The virtual assistant in Block 1 is named Jennifer, with search intensity of 60%.
Treatment 3: The virtual assistant in Block 1 is named Charles, with search intensity of 60%.
Treatment 4: The virtual assistant in Block 1 does not have a name, with search intensity of 80%.
Treatment 5: The virtual assistant in Block 1 is named Jennifer, with search intensity of 80%.
Treatment 6: The virtual assistant in Block 1 is named Charles, with search intensity of 80%.

After the advice-following game, subjects will complete the following tasks: a lottery-choice decision designed by Eckel&Grossman (2000); a brief implicit association test on gender bias; a questionnaire surveying subjects' perception of assistant's gender, their daily use of virtual assistants, and their trust in others.

In addition, we conduct a lab session in the Human Behavior Lab to collect data for the human assistant. The game procedure is similar to the lab session in Experiment 1, except that we excluded "Jennifer" and "Charles" from the fictitious name list for subjects.

External Link(s)

Registration Citation

Citation

Drichoutis, Andreas, Marco Palma and Nanyin Yang. 2023. "Anthropomorphism of Artificial Intelligence (AI) ." AEA RCT Registry. June 30. https://doi.org/10.1257/rct.10921-2.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Experiment 1:
This experiment contains an authority-delegation game, in which subjects will be paired with an assistant to search for information for a card-picking decision.

In this game, there are 35 cards with four types: Green Card, Blue Card, Red Card, and Blank Card. The followings are returns from each type of card:
(1) Blank Card: 0 token
(2) Green Card: 10 tokens
(3) Blue Card: 24 tokens
(4) Red Card: 40 tokens
Subjects will be presented with 35 cards as a five-by-seven matrix, with one Green Card, one Blue Card, one Red Card, and 32 Bland Cards. Initially, all cards are hidden and only one Green Card's position is always visible at position 18. Subjects need to conduct an information search. A successful search will uncover positions of all colored cards and pick the most profitable card.

There are two ways of information search: (1) self-search; (2) delegating the search to an assistant.
(1) Self-search: subjects need to choose a search intensity to conduct a self-search. This search intensity is the probability for the search to be successful (i.e. all cards' positions are uncovered), associated with a cost of tokens following this function: cost = 25*(search intensity)^2. If a successful search occurs, subjects will be able to freely pick a card from all 35 cards with revealed colors.
(2) Delegating the search to an assistant: the assistant will conduct a search with a fixed search intensity (depending on the treatment, explained later). This search is cost-free, but once a successful search occurs, the assistant will automatically pick the Blue Card.

This game will repeat for 20 rounds divided into two blocks of ten rounds, occurring in random order. The search intensity of the assistant is 60% in one block and 80% in another. Subjects will be informed about this search intensity in a block only when they enter this block.

After the advice-following game, subjects will complete the following tasks: a lottery-choice decision designed by Eckel&Grossman (2000); a brief implicit association test on gender bias; a questionnaire surveying subjects' perception of assistant's gender, their daily use of virtual assistants, and their trust in others.

This experiment contains 6 treatments, in a between-subjects design.
Treatment 1: the assistant is a pre-programmed virtual assistant.
Treatment 2: the assistant is a pre-programmed virtual assistant, with the name "Jennifer".
Treatment 3: the assistant is a pre-programmed virtual assistant, with the name "Charles".
Treatment 4: the assistant is a human, with search data pre-collected from a real human subject in the Human Behavior Lab. The gender is not revealed.
Treatment 5: the assistant is a female, with search data pre-collected from a real female subject in the Human Behavior Lab. This subject will pick a fictitious name to present herself.
Treatment 6: the assistant is a male, with search data pre-collected from a real male subject in the Human Behavior Lab. This subject will pick a fictitious name to present himself.

In addition, we conduct a lab session in the human behavior lab to collect the data for the human assistant. Subjects will be invited to the lab to conduct 20 rounds of information search, divided into two blocks of ten rounds. In each block, they can choose a search intensity of either 60% or 80% and conduct the 10 rounds of search with this chosen intensity. Their search results will be recorded. And they need to report their gender and pick a fictitious name from a given list of names to represent themselves. They will be informed that their data might be used for future research.

Experiment 2:
This study is a follow-up to Experiment 1. Subjects will still participate in an advice-following game, but the procedure is slightly different and it is designed to evaluate whether interactions with gendered VA pose concerns for enforcing stereotypes and biases in subsequent interactions with humans.

The game will repeat for 20 rounds divided into two blocks of ten rounds. In Block 1, subjects will be interacting with a virtual assistant, whose characteristics vary by treatment conditions (explained later). In Block 2, subjects will choose among one of four human assistants (two male names and two female names) whose search intensities are always 80%, and then play the games with their chosen human assistant.

This experiment contains 6 treatments, in a between-subject design:
Treatment 1: The virtual assistant in Block 1 does not have a name, with search intensity of 60%.
Treatment 2: The virtual assistant in Block 1 is named Jennifer, with search intensity of 60%.
Treatment 3: The virtual assistant in Block 1 is named Charles, with search intensity of 60%.
Treatment 4: The virtual assistant in Block 1 does not have a name, with search intensity of 80%.
Treatment 5: The virtual assistant in Block 1 is named Jennifer, with search intensity of 80%.
Treatment 6: The virtual assistant in Block 1 is named Charles, with search intensity of 80%.

After the advice-following game, subjects will complete the following tasks: a lottery-choice decision designed by Eckel&Grossman (2000); a brief implicit association test on gender bias; a questionnaire surveying subjects' perception of assistant's gender, their daily use of virtual assistants, and their trust in others.

In addition, we conduct a lab session in the Human Behavior Lab to collect data for the human assistant. The game procedure is similar to the lab session in Experiment 1, except that we excluded "Jennifer" and "Charles" from the fictitious name list for subjects.

Intervention Start Date

2023-05-07

Intervention End Date

2024-05-31

Primary Outcomes

Primary Outcomes (end points)

Experiment 1: A dummy variable indicating whether the subject chooses to delegate to the assistant or to self-search.
Experiment 2: (1) A dummy variable indicating whether the subject chooses a human assistant with a male name or a human assistant with a female name; (2) A dummy variable indicating whether the subject chooses to delegate to the assistant or to self-search.

Primary Outcomes (explanation)

Experiment 1: We are interested in understanding how humanizing the AI algorithm may change the propensity to use it as a search assistant.
Experiment 2: We are interested in how the experience with the virtual assistant spillovers to subjects' choices of human assistants. Also, we are interested in the impact of the assistant's characteristics on subjects' decisions of delegation.

Secondary Outcomes

Secondary Outcomes (end points)

Experiment 1: Conditional on self-search, the choice of search intensity.
Experiment 2: Conditional on self-search, the choice of search intensity.

Secondary Outcomes (explanation)

Experimental Design

Experiment 1:
This experiment contains an authority-delegation game, in which subjects will be paired with an assistant to search for information for a card-picking decision.

In this game, there are 35 cards with four types: Green Card, Blue Card, Red Card, and Blank Card. The followings are returns from each type of card:
(1) Blank Card: 0 token
(2) Green Card: 10 tokens
(3) Blue Card: 24 tokens
(4) Red Card: 40 tokens
Subjects will be presented with 35 cards as a five-by-seven matrix, with one Green Card, one Blue Card, one Red Card, and 32 Blank Cards. Initially, all cards are hidden and only one Green Card's position is always visible at position 18. Subjects need to conduct an information search. A successful search will uncover positions of all colored cards and pick the most profitable card.

There are two ways of information search: (1) self-search; (2) delegating the search to an assistant.
(1) Self-search: subjects need to choose a search intensity to conduct a self-search. This search intensity is the probability for the search to be successful (i.e. all cards' positions are uncovered), associated with a cost of tokens following this function: cost = 25*(search intensity)^2. If a successful search occurs, subjects will be able to freely pick a card from all 35 cards with revealed colors.
(2) Delegating the search to an assistant: the assistant will conduct a search with a fixed search intensity (depending on the treatment, explained later). This search is cost-free, but once a successful search occurs, the assistant will automatically pick the Blue Card.

This game will repeat for 20 rounds divided into two blocks of ten rounds, occurring in random order. The search intensity of the assistant is 60% in one block and 80% in another. Subjects will be informed about this search intensity in a block only when they enter this block.

After the advice-following game, subjects will complete the following tasks: a lottery-choice decision designed by Eckel&Grossman (2000); a brief implicit association test on gender bias; a questionnaire surveying subjects' perception of assistant's gender, their daily use of virtual assistants, and their trust in others.

This experiment contains 6 treatments, in a between-subjects design.
Treatment 1: the assistant is a pre-programmed virtual assistant.
Treatment 2: the assistant is a pre-programmed virtual assistant, with the name "Jennifer".
Treatment 3: the assistant is a pre-programmed virtual assistant, with the name "Charles".
Treatment 4: the assistant is a human, with search data pre-collected from a real human subject in the Human Behavior Lab. The gender is not revealed.
Treatment 5: the assistant is a female, with search data pre-collected from a real female subject in the Human Behavior Lab. This subject will pick a fictitious name to present herself.
Treatment 6: the assistant is a male, with search data pre-collected from a real male subject in the Human Behavior Lab. This subject will pick a fictitious name to present himself.

In addition, we conduct a lab session in the human behavior lab to collect the data for the human assistant. Subjects will be invited to the lab to conduct 20 rounds of information search, divided into two blocks. In each block, they can choose a search intensity of either 60% or 80% and conduct the 10 rounds of search with this chosen intensity. Their search results will be recorded. And they need to report their gender and pick a fictitious name from a given list of names to represent themselves. They will be informed that their data might be used for future research.

Experiment 2:
This study is a follow-up to Experiment 1. Subjects will still participate in an advice-following game, but the procedure is slightly different to evaluate whether interactions with gendered VA pose concerns for enforcing stereotypes and biases in subsequent interactions with humans.

The game will repeat for 20 rounds divided into two blocks of ten rounds. In Block 1, subjects will be interacting with a virtual assistant, whose characteristics vary by treatment conditions (explained later). In Block 2, subjects will choose among one of four human assistants (two male names and two female names) whose search intensities are always 80%, and then play the games with their chosen human assistant.

This experiment contains 6 treatments, in a between-subject design:
Treatment 1: The virtual assistant in Block 1 does not have a name, with search intensity of 60%.
Treatment 2: The virtual assistant in Block 1 is named Jennifer, with search intensity of 60%.
Treatment 3: The virtual assistant in Block 1 is named Charles, with search intensity of 60%.
Treatment 4: The virtual assistant in Block 1 does not have a name, with search intensity of 80%.
Treatment 5: The virtual assistant in Block 1 is named Jennifer, with search intensity of 80%.
Treatment 6: The virtual assistant in Block 1 is named Charles, with search intensity of 80%.

After the advice-following game, subjects will complete the following tasks: a lottery-choice decision designed by Eckel&Grossman (2000); a brief implicit association test on gender bias; a questionnaire surveying subjects' perception of assistant's gender, their daily use of virtual assistants, and their trust in others.

In addition, we conduct a lab session in the Human Behavior Lab to collect data for the human assistant. The game procedure is similar to the lab session in Experiment 1, except that we excluded "Jennifer" and "Charles" from the fictitious name list for subjects.

Experimental Design Details

Not available

Randomization Method

Experiment 1:
For the three treatments of AI assistants and the three treatments of human assistants, subjects will be randomized to one of the three treatments by the randomization program in oTree.
This study will be administered on Forthright Access and the random assignment of subjects to either treatments of AI assistants or treatments of human assistants will be handled with the Forthright Access platform.
Experiment2:
Subjects will be randomized into one of six treatments by the randomization program in oTree and by the Forthright Access platform.

Randomization Unit

Individual.

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

Experiment 1: Treatments will not be clustered.
Experiment 2: Treatments will not be clustered.

Sample size: planned number of observations

Experiment 1: We plan to collect at least 79 subjects per treatment, with a total of 79*6 = 474 subjects. Experiment 2: We plan to collect at least 100 subjects for the treatments with a female-named virtual assistant and the treatments with a gender-neutral virtual assistant. And we plan to collect at least 108 subjects for the treatments with a male-named virtual assistant. This is because in Experiment 1, we collected at least 100 subjects per treatment, and to keep both experiments consistent, we plan to target at least 100 subjects per treatment. In addition, because in Experiment 1, subjects delegated less to male-gendered virtual assistants, in order to assure equal number of delegations between male-gendered treatments and other treatments, we will over-recruit the sample size to at least 108 for the two male-gendered treatments in Experiment 2.

Sample size (or number of clusters) by treatment arms

Experiment 1: At least 79 subjects per treatment, and each subject with 20 trials of decisions (10 trials with an assistant search intensity of 60% and another 10 with assistant search intensity of 80%).

Experiment 2: We plan to collect at least 100 subjects for the treatments with a female-named virtual assistant and the treatments with a gender-neutral virtual assistant. And we plan to collect at least 108 subjects for the treatments with a male-named virtual assistant. This is because in Experiment 1, we collected at least 100 subjects per treatment, and to keep both experiments consistent, we plan to target at least 100 subjects per treatment. In addition, because in Experiment 1, subjects delegated less to male-gendered virtual assistants, in order to assure equal number of delegations between male-gendered treatments and other treatments, we will over-recruit the sample size to at least 108 for the two male-gendered treatments in Experiment 2.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Experiment 1: We will need at least 79 subjects per treatment, based on a power of 0.9 and a significance level of 0.05, to detect a statistically significant difference between the high-intensity and low-intensity conditions. Power calculations are based on anticipated proportions of control and treatment arms taken from Fehr, Herz, and Wilkening (2013, The Lure of Authority: Motivation and Incentive Effects of Power) i.e., 13.9% and 35.5% (as reported in Figure 2). Experiment 2: We plan to collect at least 100 subjects for the treatments with a female-named virtual assistant and the treatments with a gender-neutral virtual assistant. And we plan to collect at least 108 subjects for the treatments with a male-named virtual assistant. This is because in Experiment 1, we collected at least 100 subjects per treatment, and to keep both experiments consistent, we plan to target at least 100 subjects per treatment. In addition, because in Experiment 1, subjects delegated less to male-gendered virtual assistants, in order to assure equal number of delegations between male-gendered treatments and other treatments, we will over-recruit the sample size to at least 108 for the two male-gendered treatments in Experiment 2.

Supporting Documents and Materials

IRB

Institutional Review Boards (IRBs)

IRB Name

Texas A&M University

IRB Approval Date

2023-03-08

IRB Approval Number

IRB2023-0083M/153086

Analysis Plan