Blind Justice: Algorithmically Masking Race in Prosecutorial Charging Decisions

To prevent race from influencing charging decisions, we created an algorithm that automatically redacts race-related information from free-text police incident reports. Our tool finds and removes not only explicit mentions of race, but also other information that a prosecutor could use to guess an individual’s race, including physical descriptions, names, and locations. Our intervention is showing prosecutors a race-blind version of each case when they make charging decisions.

2025-01-01

2025-06-30

The primary outcome of interest for our RCT is whether race-blind reviews reduce differences in charging rates between Black or Hispanic arrestees and all other arrestees.

Our secondary outcome of interest is whether race-blind reviews change overall charging rates (e.g., if the lack of personal information like names or neighborhoods causes prosecutors to lose empathy, increasing charging rates overall; or if the race-blind stage causes prosecutors to make decisions that lower the risk of a decision reversal in either direction).

We also plan to measure the amount of time it takes to conduct a race-blind review, and compare it against a similar estimate for the status-quo review procedure.

We will randomly assign each incoming case to either a control or treatment condition. In the control condition, cases will be processed according to the status quo procedure, where all evidence is available in an unmodified form at the time of the charging decision.

In the treatment condition, prosecutors will follow a new two-stage charging procedure that allows them to make race-blind charging decisions, while balancing their need to review all available evidence (including photos or videos, which would be difficult to redact). Prosecutors will begin a race-blind review by reading the redacted incident report and recording a preliminary charging decision. Within a couple days, prosecutors will return to the case to review all available materials, including an unredacted version of the incident report, and will make a final decision about whether or not to charge the individual. If their final decision differs from their preliminary, race-blind decision, they will be required to explain the reason for the change, discouraging unconscious or conscious changes made when race information becomes available.

Randomization done in software by a computer.

Randomization by case. Different research sites may decide to randomize at different rates (e.g., 10% of cases to control instead of 50% of cases).

No

5 research sites (see note below about site withdrawal).
4 research sites will randomize 50% of cases to each arm; with the remaining site randomizing only 10% of cases to control, and the remainder to treatment.

Site 1: 10,000 cases, with 5,000 in control Site 2: 2,000 cases, with 1,000 in control Site 3: 7,500 cases, with 3,750 in control Site 4: 24,700 cases, with 2,470 in control Site 5: 4,800 cases, with 2,400 in control This is 49,000 cases in total. We expect the final number may vary considerably depending on site willingness to continue participation in our experiment. We will run the experiment until we obtain the target number of cases, until two years have elapsed since the start of the experiment in that site, or until a site no longer wishes to participate in our experiment, whichever comes first.

34,380 cases in treatment, under race-blind review, and 14,620 cases in control.

Note these may change closer to the launch date as sites finalize what proportion of cases they plan to send to treatment and control.

We ran 1,000 power simulations to determine statistical power for our experiment. In each simulation, we incorporated estimates of each site's charging rate and proportion of Black/Hispanic and all other arrestees based on historical data from each site. We also estimated each site's number of cases they will send to our experiment each year, as well as the proportion of cases they will send to control, based on recent conversations with each site. We assumed that pre-existing charging rates for Black and Hispanic arrestees were 2.75pp higher than the charging rate for white arrestees, and that the treatment arm reduced 90% of this gap. We generated a synthetic experiment population for each experiment simulation using these parameters. We then fit a logistic regression model to the synthetic population of the form logit(Pr(Y = 1)) = β_0 + β_1 * race + β_2 * treatment + β_3 * race * treatment, where "race" was 1 if the arrestee were Black or Hispanic, and 0 otherwise; and where "treatment" was 1 if the case was randomly assigned to the treatment arm. We then calculated whether the 95% confidence interval for β_3 crossed zero. Under this setup, we expect to detect the primary effect—a reduction in bias in charging decisions—in 81.3% of experiments of this size and design, indicating adequate power to detect small reductions in charging rate differences for Black and Hispanic arrestees. For our secondary outcome, we ran a similar set of simulations with no pre-existing disparity, instead assuming that charging rates for all arrestees were 2.25pp higher or lower in the treatment arm. We then fit a logistic regression model to the synthetic population of the form logit(Pr(Y = 1)) = β_0 + β_1 * race + β_2 * treatment. This is nearly identical to the above model, though we dropped the interaction to simplify the power analysis. We then examined the coefficient β_2 and calculated whether the 95% confidence interval crossed zero. Given this setup, we expect to detect this effect in 80.5% of experiments of this size and design, again indicating adequate power to detect small changes in the overall charging rate. We expect to gain additional statistical power by including a random effect for the prosecutor assigned to make the charging decision and by adjusting for case covariates, including arrestee sex, and age; the day, month, and year of the arrest; the presence of flags on the incident report indicating e.g., domestic violence, elderly victims, gang involvement, weapons, or the use of a body-worn camera; the Census-derived racial composition of the area in which the incident occurred, if the address is available; the precinct or police department where the arrest occurred; two-year retrospective arrest and felony arrest counts for the suspect; the alleged charges; and the number of alleged charges in total.