Algorithms and Legal Decisions: From Private Information to Common Knowledge

This study employs a randomized controlled trial (RCT) conducted in collaboration with public prosecutors in the Czech Republic.All active public prosecutors, as well as those currently in training, will be invited to participate through an official email distributed by the Deputy Supreme Prosecutor. In the survey, participants will be presented with two hypothetical theft-related criminal cases and asked to make decisions on each. Following their decisions, prosecutors will respond to a series of questions about their decision-making process and their expectations regarding the outcomes of the cases.

In the treatment groups, prosecutors will receive an algorithmic risk assessment score indicating the defendant’s likelihood of reoffending before their sentencing recommendations and answering the questions. This risk assessment score is based on similar tools used internationally and within the Czech prison system. It provides detailed information on recidivism risk, scores across various predictive factors, and insights into the tool itself. By integrating this tool into the experiment, we aim to evaluate how algorithmic risk assessments influence prosecutorial decision-making and their subsequent justifications.

2025-01-29

2025-02-20

1. Decision to recommend incarceration.
2. Sentencing length.
3. Assessment of the defendant's risk.
4. Reasoning of the decision.

1. Decision to recommend incarceration. This is a binary variable.
2. Sentencing length: this will be computed through a two-part (or multinomial) model. We will also compute a severity score that considers both the penalty type and length.
3. Assessment of the defendant's risk: After the completion of the vignette, we ask the prosecutor about their own beliefs of recidivism risk for the defendants in each vignette. We will use this to check if prosecutors updated their belief as a result of the treatment.
4. Reasoning of the decision: Whether the prosecutor would include in their justification for the decision the expectation that the defendant will live an orderly life (or not).

1. Expectation on the judge's decision to incarcerate & incarceration length
2. Expectation of defendant's likelihood to appeal
3. Consistency or Variance
4. Their expectations on how other prosecutors would decide relative to them (i.e., more or less severe).
5. (Only for treatments): whether they agree with the risk assessment score provided.

In addition to full specification, we will also consider the comparison only between the two treatment arms. In addition, depending on the final number of responses we receive (and the associated power), we may also consider grouping treatments 1 and 2 as a joint treatment group.

1. Expectation on the judge's decision to incarcerate & incarceration length: Following the same methods as in the primary outcomes, but regarding the prosecutors' expectation of the judge's decision. In addition, we will measure the difference between the prosecutors' decision and their expectation of the judge's decision (to measure to what extent the prosecutor thinks that the judge will agree with their decision).
2. Expectation of defendant's likelihood to appeal: We will assess their perception of the likelihood of the defendant to appeal the judge’s decision if the judge were to agree with the prosecutor.
3. Consistency or Variance: We will measure the consistency across prosecutors within a treatment arm when deciding on the same case. To do this, we will consider: a) type of sentence (i.e., incarceration or others); and b) length of sentence. Method: Bartlett's test or Levene's test/Brown-Forsythe test. Alternative methods could be negative binomial random effects model or gini coefficients
4. Their expectations on how other prosecutors would decide relative to them (i.e., more or less severe).
5. (Only for treatment arms): whether they agree with the risk assessment score provided.

In addition to the full specification, we will also compare the two treatment arms directly. Furthermore, depending on the final number of responses received (and the associated statistical power), we may consider grouping treatments 1 and 2 into a joint treatment group.

This study consists of an RCT that will be implemented with public prosecutors of the Czech Republic. All active public prosecutors and prosecutors in training will be invited to participate in the study via email. In this survey, prosecutors will be asked to decide on two cases related to theft offenses. After deciding on such cases, the prosecutors will be asked a series of questions regarding their beliefs on the case.

In the treatment groups, the prosecutors receive a risk assessment score prior to their decision, which indicates the defendant’s likelihood of reoffending. There are two treatment arms: in the public information treatment, prosecutors are informed that the risk assessment score is part of the official court docket—accessible to judges and other criminal justice actors—whereas in the private information treatment, they are informed that the score is only available to them as prosecutors.

Not available

Randomization done by the survey platform (Qualtrics). It will be a randomized experiment, where treatment arms will be stratified by gender and whether the prosecutors are district prosecutors or not. These two strata are significantly associated with case decisions according to previous research in the Czech context.

We will implement the randomization only among prosecutors who open the survey. We do this by coding it into the survey platform, so that every time a respondent opens the survey they are randomly assigned into a treatment group (considering their stata).

Individual prosecutors.

No

The invitation to participate will be shared with 1,250 active prosecutors, with additional 80 prosecutors in training. However, based on prior research with this same population, we expect approximately 200/250 prosecutors to complete the survey.

If there are less than 282 prosecutors who complete the survey after the first email invitation, a reminder will be sent by the prosecutors' office to remind prosecutors to participate. Adding responses from law students might be considered only as a last resort if the number of responses remains too low.

As above.

One third per treatment arm. So, if 300 prosecutors completed the survey, we would have approximately 100 prosecutors in each treatment group.