Algorithmic Accuracy by Group and Recidivism Prediction

Last registered on February 08, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
Algorithmic Accuracy by Group and Recidivism Prediction
RCT ID
AEARCTR-0010807
Initial registration date
January 20, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 08, 2023, 12:09 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
Yong Suk Lee
Affiliation
University of Notre Dame
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2023-01-31
End date
2023-02-28
Keywords
Labor
Additional Keywords
Algorithmic Accuracy, Prediction, Recidivism
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study examines how AI performance metrics affects police prediction of re-arrests using an online randomized survey. Studies have found that base commercial risk assessment softwares are no more accurate or fairer than predictions made by people with little or no criminal justice expertise. However, algorithms do outperform human predictions of recidivism when enriched set of risk factors are included in the model. This study examines how police predication compares with algorithms and whether predictions differ when race/ethnicity is provided, and the accuracy of the algorithm by race/ethnicity.

External Link(s)

Registration Citation

Citation
Lee, Yong Suk. 2023. "Algorithmic Accuracy by Group and Recidivism Prediction." AEA RCT Registry. February 08. https://doi.org/10.1257/rct.10807-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
This study examines how AI performance metrics affects police prediction of re-arrests using an online randomized survey. A representative sample of the US population born between 1980 and 1983 with arrest histories are randomly presented to respondents. Respondents are asked to predict the likelihood that this person will be arrested again in the next three years. Respondents are then shown an AI algorithm's recidivism prediction score for each profile and allowed to adjust their prediction. Different treatment groups will be presented with a combination of information related to race and ethnicity, as well as the AI algorithm's prediction accuracy.
Intervention Start Date
2023-01-31
Intervention End Date
2023-02-28

Primary Outcomes

Primary Outcomes (end points)
prediction probability of re-arrests
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Each respondent will see 20 profiles and make predictions on each.
Base group: Will have no race/ethnicity information
Treatment Group 1 : Will be shown race/ethnicity information
Treatment Group 2 : Will be shown the AI prediction accuracy by race/ethnicity
Treatment Group 3: Will be shown race/ethnicity information plus the AI prediction accuracy by race/ethnicity
Experimental Design Details
Randomization Method
online randomization using survey company
Randomization Unit
individuals
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
no clusters
Sample size: planned number of observations
150
Sample size (or number of clusters) by treatment arms
50 no race, 50 race, 50 mixed
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Notre Dame
IRB Approval Date
2023-01-04
IRB Approval Number
ID: 22-12-7547
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials