Algorithms and Legal Decisions: From Private Information to Common Knowledge

This study employs a randomized controlled trial (RCT) conducted in collaboration with public prosecutors in the Czech Republic.All active public prosecutors, as well as those currently in training, will be invited to participate through an official email distributed by the Deputy Supreme Prosecutor. In the survey, participants will be presented with two hypothetical theft-related criminal cases and asked to make decisions on each. Following their decisions, prosecutors will respond to a series of questions about their decision-making process and their expectations regarding the outcomes of the cases.

In the treatment groups, prosecutors will receive an algorithmic risk assessment score indicating the defendant’s likelihood of reoffending before their sentencing recommendations and answering the questions. This risk assessment score is based on similar tools used internationally and within the Czech prison system. It provides detailed information on recidivism risk, scores across various predictive factors, and insights into the tool itself. By integrating this tool into the experiment, we aim to evaluate how algorithmic risk assessments influence prosecutorial decision-making and their subsequent justifications.

This study employs a randomized controlled trial (RCT) conducted in collaboration with public prosecutors in the Czech Republic.All active public prosecutors, as well as those currently in training, will be invited to participate through an official email distributed by the Deputy Supreme Prosecutor. In the survey, participants will be presented with two hypothetical theft-related criminal cases and asked to make decisions on each. Following their decisions, prosecutors will respond to a series of questions about their decision-making process and their expectations regarding the outcomes of the cases.

In the treatment groups, prosecutors will receive an algorithmic risk assessment score indicating the defendant’s likelihood of reoffending before their sentencing recommendations and answering the questions. This risk assessment score is based on similar tools used internationally and within the Czech prison system. It provides detailed information on recidivism risk, scores across various predictive factors, and insights into the tool itself. By integrating this tool into the experiment, we aim to evaluate how algorithmic risk assessments influence prosecutorial decision-making and their subsequent justifications.

The treatment arms differ in how prosecutors are told this information is shared:
- Public information arm: Prosecutors are informed that the risk assessment score is part of the court docket, and is thus available to the judge and other actors in the criminal process.
- Private information arm: Prosecutors are informed that the risk assessment score is only available to them, and thus is not shared with other actors in the judicial process.

This design allows us to evaluate both the impact of algorithmic risk assessment on prosecutors’ sentencing recommendations and how those recommendations differ when prosecutors believe the information is private to them or public.

2025-01-29

2025-02-20

1. Decision to recommend incarceration.
2. Sentencing length.
3. Assessment of the defendant's risk.
4. Reasoning of the decision.

1. Decision to recommend incarceration. This is a binary variable.
2. Sentencing length: this will be computed through a two-part (or multinomial) model. We will also compute a severity score that considers both the penalty type and length.
3. Assessment of the defendant's risk: After the completion of the vignette, we ask the prosecutor about their own beliefs of recidivism risk for the defendants in each vignette. We will use this to check if prosecutors updated their belief as a result of the treatment.
4. Reasoning of the decision: Whether the prosecutor would include in their justification for the decision the expectation that the defendant will live an orderly life (or not).

1. Expectation on the judge's decision to incarcerate & incarceration length
2. Expectation of defendant's likelihood to appeal
3. Consistency or Variance
4. Their expectations on how other prosecutors would decide relative to them (i.e., more or less severe).
5. (Only for treatments): whether they agree with the risk assessment score provided.

In addition to full specification, we will also consider the comparison only between the two treatment arms. In addition, depending on the final number of responses we receive (and the associated power), we may also consider grouping treatments 1 and 2 as a joint treatment group.

1. Expectation on the judge's decision to incarcerate & incarceration length: Following the same methods as in the primary outcomes, but regarding the prosecutors' expectation of the judge's decision. In addition, we will measure the difference between the prosecutors' decision and their expectation of the judge's decision (to measure to what extent the prosecutor thinks that the judge will agree with their decision).
2. Expectation of defendant's likelihood to appeal: We will assess their perception of the likelihood of the defendant to appeal the judge’s decision if the judge were to agree with the prosecutor.
3. Consistency or Variance: We will measure the consistency across prosecutors within a treatment arm when deciding on the same case. To do this, we will consider: a) type of sentence (i.e., incarceration or others); and b) length of sentence. Method: Bartlett's test or Levene's test/Brown-Forsythe test. Alternative methods could be negative binomial random effects model or gini coefficients
4. Their expectations on how other prosecutors would decide relative to them (i.e., more or less severe).
5. (Only for treatment arms): whether they agree with the risk assessment score provided.

In addition to the full specification, we will also compare the two treatment arms directly. Furthermore, depending on the final number of responses received (and the associated statistical power), we may consider grouping treatments 1 and 2 into a joint treatment group.

This study consists of an RCT that will be implemented with public prosecutors of the Czech Republic. All active public prosecutors and prosecutors in training will be invited to participate in the study via email. In this survey, prosecutors will be asked to decide on two cases related to theft offenses. After deciding on such cases, the prosecutors will be asked a series of questions regarding their beliefs on the case.

In the treatment groups, the prosecutors receive a risk assessment score prior to their decision, which indicates the defendant’s likelihood of reoffending. There are two treatment arms: in the public information treatment, prosecutors are informed that the risk assessment score is part of the official court docket—accessible to judges and other criminal justice actors—whereas in the private information treatment, they are informed that the score is only available to them as prosecutors.

This study consists of an RCT that will be implemented with public prosecutors of the Czech Republic. All active public prosecutors and prosecutors in training will be invited to participate in the study via email. In this study, prosecutors will be asked to decide on two cases related to theft offenses. After deciding on such cases, the prosecutors will be asked a series of questions regarding their beliefs on the case.

Every prosecutor will read a summary of each case, including the background on the offense and the characteristics of the offender. After reviewing the case, prosecutors will be asked to decide on what penalty to impose (including whether to impose incarceration or not, and length of the penalty), as well as a series of questions about their decision. Before making these decisions, prosecutors will be divided into the following treatment arms:

- T1 Algorithmic risk score (private to the prosecutor): Prosecutors will receive an algorithmic risk assessment for the defendant, which indicates the risk of recidivism of the defendant. The prosecutor will be informed that the risk assessment is only available to them.
- T2 Algorithmic risk score (public): As above, prosecutors will receive an algorithmic risk assessment for the defendant, which indicates the risk of recidivism of the defendant. However, in this case, the prosecutor will be informed that the risk assessment is part of the court’s dossier, and thus is available to everyone involved in the case, including the judge who will review their sentencing recommendation.
- Control group: Prosecutors will not receive the algorithmic risk assessment.

After each vignette, prosecutors will recommend a sentencing decision and respond to a series of questions about their decision-making process. These questions will assess their beliefs of the defendant's risk of reoffending, to what extent they would like to share this information with the judge, and what they think the judge would do, among other topics. At the end, prosecutors in the treatment groups will also be asked about their agreement with the risk assessment score in each vignette.

Randomization done by the survey platform (Qualtrics). It will be a randomized experiment, where treatment arms will be stratified by gender and whether the prosecutors are district prosecutors or not. These two strata are significantly associated with case decisions according to previous research in the Czech context.

We will implement the randomization only among prosecutors who open the survey. We do this by coding it into the survey platform, so that every time a respondent opens the survey they are randomly assigned into a treatment group (considering their stata).

Individual prosecutors.

No

The invitation to participate will be shared with 1,250 active prosecutors, with additional 80 prosecutors in training. However, based on prior research with this same population, we expect approximately 200/250 prosecutors to complete the survey.

If there are less than 282 prosecutors who complete the survey after the first email invitation, a reminder will be sent by the prosecutors' office to remind prosecutors to participate. Adding responses from law students might be considered only as a last resort if the number of responses remains too low.

As above.

One third per treatment arm. So, if 300 prosecutors completed the survey, we would have approximately 100 prosecutors in each treatment group.