Humans versus Bots

Last registered on October 18, 2021
View Trial History

Pre-Trial

Trial Information

General Information

Title
Humans versus Bots
RCT ID
AEARCTR-0008269
Initial registration date
October 15, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 18, 2021, 9:40 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Chile
Region
Country

Primary Investigator

Name
Nicolas Ajzenman
Affiliation
Sao Paulo School of Economics - FGV
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Gregory Elacqua
PI Affiliation
Inter-American Development Bank
Contact Investigator
PI Name
Analia Jaimovich
PI Affiliation
Inter-American Development Bank
Contact Investigator
PI Name
Diana Hincapie
PI Affiliation
Inter-American Development Bank
Contact Investigator
PI Name
Graciela Perez
PI Affiliation
Inter-American Development Bank
Contact Investigator

Additional Trial Information

Status
In development
Start date
2021-10-31
End date
2022-03-31
Keywords
Behavior, Education
Additional Keywords
bots, motivation, education
JEL code(s)
D9, I2
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Qualified teachers are a fundamental input for any education system. This paper presents the results of a large-scale intervention to attract high-performing high-school students into the teaching profession in Chile. The intervention was a three-arm campaign that made salient three types of motivations: intrinsic/altruistic, extrinsic, and prestige-related. There are two treatment arms. In the first one, a team of promoters from a specialized NGO implemented an intensive campaign based on a structured script implemented through WhatsApp messages and phone calls. In the second one, the campaign was delivered using a pre-programmed Bot that operates through WhatsApp. The third arm is a passive control that does not receive any message. We use real-world behavioral outcomes to measure the effectiveness of each type of intervention. These include the probability that a high-school student applies to a teaching degree, the probability that she includes a teaching degree among the most preferred options, and the probability that a high-school student is finally offered (and accepts) a position in a university to pursue a teaching major.
External Link(s)

Registration Citation

Citation
Ajzenman, Nicolas et al. 2021. "Humans versus Bots." AEA RCT Registry. October 18. https://doi.org/10.1257/rct.8269-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This experiment will be carried out with approximately 24,000 high-school students in their final year who declared on a survey conducted by Elige Educar (an NGO) that they might be interested in pursuing a career in teaching or in social sciences. Students were randomly assigned to one of the two treatment arms (“humans”, “bots”) or to the control group, stratifying by gender, geographic region, and type of school they attended (public, private subsidized, and private non-subsidized).

The content of the messages delivered by each treatment arm was designed to be as similar as possible. The messages are intended to motivate students to pursue a degree in education by appealing to three types of potential motivations: intrinsic related to altruism (related to how teachers can make a difference in the world), extrinsic (teachers have good conditions, low unemployment rate, a good salary) and prestige (teachers are respected in the society). The difference between the treatment arms is how they were delivered and how intense the treatment is.

The "humans" treatment arm is an intensive campaign that relies on the work of human tutors, who send WhatsApp messages, offer students to talk by phone and answer questions. The tutors follow a script, although there is some flexibility based on the questions they receive.

The "bots" treatment arm is a campaign that relies on the work of bot "tutors", who send WhatsApp messages and answer questions using a pre-programmed algorithm. The bots intend to follow a script similar to the one used in the "human" arm, although there is less flexibility.

The control group does not receive any message.

The sample is composed of high-school students that reported to have some interest in Education as a career.


The timeline of the campaign is the following:

- "Humans": they start the campaign in mid-October until the first week of December. They stop for a week and restart for four weeks until mid-January. In total, this campaign lasts for 12 weeks.
- "Bots": they start the campaign in the first week of November until the first week of December. They stop and restart in the first week of January until the second week of January. In total, this campaign lasts for seven weeks.



Intervention Start Date
2021-10-31
Intervention End Date
2022-03-31

Primary Outcomes

Primary Outcomes (end points)
Ranked an education major as the first option: takes a one if the student ranked a major in education as the first preference of degree she/he wants to pursue.

The proportion of education majors included in the choice set: proportion of options that are majors in education out of the total of degree options selected by the student.

Applied to at least one education major: takes a one if the student applied to at least one major in education in her/his set.


All outcomes are real-world, high-stakes choices.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This experiment will be carried out with approximately 24,000 high-school students in their final year who declared on a survey conducted by Elige Educar (an NGO) that they might be interested in pursuing a career in teaching or in social sciences. Students were randomly assigned to one of the two treatment arms (“humans”, “bots”) or to the control group, stratifying by gender, geographic region, and type of school they attended (public, private subsidized, and private non-subsidized).

The content of the messages delivered by each treatment arm was designed to be as similar as possible. The messages are intended to motivate students to pursue a degree in education by appealing to three types of potential motivations: intrinsic related to altruism (related to how teachers can make a difference in the world), extrinsic (teachers have good conditions, low unemployment rate, a good salary) and prestige (teachers are respected in the society). The difference between the treatment arms is how they were delivered and how intense the treatment is.

The "humans" treatment arm is an intensive campaign that relies on the work of human tutors, who send WhatsApp messages, offer students to talk by phone and answer questions. The tutors follow a script, although there is some flexibility based on the questions they receive.

The "bots" treatment arm is a campaign that relies on the work of bot "tutors", who send WhatsApp messages and answer questions using a pre-programmed algorithm. The bots intend to follow a script similar to the one used in the "human" arm, although there is less flexibility.

The control group does not receive any message.

The sample is composed of high-school students that reported to have some interest in pursuing a career in education.


The timeline of the campaign is the following:

- "Humans": they start the campaign in mid-October until the first week of December. They stop for a week and restart for four weeks until mid-January. In total, this campaign lasts for 12 weeks.
- "Bots": they start the campaign in the first week of November until the first week of December. They stop and restart in the first week of January until the second week of January. In total, this campaign lasts for seven weeks.



Experimental Design Details
Randomization Method
Randomization done in an office by a computer
Randomization Unit
Students (no clusters)
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
No clusters.
Sample size: planned number of observations
Every arm will have approximately 8,000 individuals. No clusters.
Sample size (or number of clusters) by treatment arms
Every arm will have approximately 8,000 individuals. No clusters.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With a sample size of 16000 we would be able to identify a MDE of 2.2 pp (baseline 9pp) with a power of 80% and 95% of confidence.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials