The Impact of Algorithmic Decision-Making in Labor Market

Last registered on June 17, 2024

Pre-Trial

Trial Information

General Information

Title
The Impact of Algorithmic Decision-Making in Labor Market
RCT ID
AEARCTR-0013580
Initial registration date
June 08, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 17, 2024, 2:43 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Utrecht Universiteit

Other Primary Investigator(s)

PI Affiliation
Supervisor
PI Affiliation
Supervisor

Additional Trial Information

Status
On going
Start date
2024-04-27
End date
2024-08-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The adoption of AI in decision-making has seen rapid advances, especially in the context of the hiring process (Chalfin et al., 2016; Raghavan et al., 2020). Algorithms can accelerate the hiring process as well as improve the efficiency. However, ethical concerns have emerged due to potential bias and discrimination in hiring, posing more risks to vulnerable groups (Obermeyer et al., 2019; Giermindl et al., 2022). This project aims to comprehend how individuals adapt to and interact with algorithms in the labor market, focusing on how workers and HR managers interact with hiring algorithms. To achieve this goal, we will conduct several online, and lab experiments and collect data from the students population in the Netherlands as well as workers in online labor markets (i.e. Mturk, Prolifics) from the US and UK. In particular, this study serves as a pilot to initiate the previously mentioned project.
External Link(s)

Registration Citation

Citation
Cai, Cielo, Elena Fumagalli and Sarah Rezaei Khavas. 2024. "The Impact of Algorithmic Decision-Making in Labor Market ." AEA RCT Registry. June 17. https://doi.org/10.1257/rct.13580-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
We want to explore the impact of willful ignorance on HR managers' decisions when they receive recommendations from hiring algorithms. In the experiment, participants need to select high-potential workers for a particular task, and they will be provided with a series of information, includes the workers' basic performance information and the recommendation from an algorithm. They are informed that the algorithm might be biased, and they can acquire more information about each worker's performance and overrule or accept the algorithmic decision without further examination. HR managers in the control group have free access to the detailed information about each worker's performance, while in the treatment group, accessing this information is costly.
Intervention Start Date
2024-06-10
Intervention End Date
2024-06-24

Primary Outcomes

Primary Outcomes (end points)
1. The time each HR manager spent on the hiring task
2. The number of times each HR manager clicked to obtain additional information
3. The share of women of the workers selected by each HR manager
4. The comparison of the average real Raven performance for workers selected by humans and algorithms respectively
5. The role of social preference in the worker selection process
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This pilot includes two phases of experiments: the Workers' experiment and the HR Managers' experiment.

In the Workers' experiment, workers will perform a few real-effort tasks and we will generate workers' performance data based on their performance and their demographic data.

In the HR managers' experiment, HR managers need to select high-potential workers. They will receive the hiring table, includes the workers' demographic and task performance, the recommendation from the hiring algorithm, and information regarding the potential biases of the algorithm. The HR manager can acquire more information about each worker's performance and overrule or accept the algorithmic decision without further examination.
Experimental Design Details
Not available
Randomization Method
We will program on Qualtrics and randomization will therefore be done by Qualtrics
Randomization Unit
We will randomize in the individual level, each participant in the HR managers' experiment will be randomly assigned to either the control group or the treatment group.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will have two groups, the control group and the treatment group.
Sample size: planned number of observations
We plan to recruit 250 participants for this pilot, 100 for workers' experiment, 75 each for the control group and the treatment group for the HR managers' experiment.
Sample size (or number of clusters) by treatment arms
In Workers' experiment, we collect participants' performance to generate the performance data, therefore we will recruit 100 participants to act as workers.
In HR managers' experiment, we will have two groups, one control group and one treatment group, and each group will have 75 participants.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethische Toetsingscommissie Rebo
IRB Approval Date
2024-04-02
IRB Approval Number
24-006