Making Information Actionable: Experimental Evidence from Kenyan Courts

This study will test two interventions:

1. Actionable information: Providing judicial officers a one-page, simplified feedback form every month, that shows them 1) the case clearance rate (CCR; number of cases resolved divided by number of cases filed), 2) the number of adjournments and their top three reasons, 3) the predicted impact of reducing adjournments on the CCR

2. Actionable information with deliberation: As above, but also sharing the information with Court User Committees (CUCs) that include representatives from public security agencies and local communities, to focus discussion around performance, create bottom-up accountability, and collectively arrive at ways to remove the bottlenecks.

The feedback forms were sent once in January 2019. The goal will be to send additional feedback forms in 2020 and 2021 every month for a duration of one year.

2019-01-01

2022-12-01

The primary outcome is the proportion of hearings ending in an adjournment.

We will use a difference-in-differences analysis, with court and month fixed effects.

The research plan includes subgroup analyses by:

• Baseline time to disposition

• Baseline number of adjournments, disaggregated by their main reasons

• Strength of feedback (i.e., 1) spread between CCR and recommended target, 2) number of adjournments reported, and 3) size of predicted impact)

• Nature of top reasons of adjournments

• Type of court (high court, magistrate court, emloyment and labour relations court, environment and land court)

• Type of case (civil (e.g., succession, commercial), criminal (e.g., property, violent))

The randomization was stratified on 1) regions (8), and 2) time to disposition. These stratification dummies will be added to the model.

• Adjournments disaggregated by reason for adjournment. In particular, adjournments can be “internal”, i.e., under the control of the judge (e.g., court not sitting) or “external”, i.e., caused by other parties (e.g., lawyer not ready).

• Case clearance rate (cases disposed of divided by cases filed) as a standard measure of productivity of courts, as well as cases disposed, cases filed

• Time to disposition: Considering the feedback forms were only sent once in January 2019, it may seem unreasonable to expect an effect on the time to disposition of cases. If additional feedback forms can be sent in 2020 and 2021 every month for a duration of one year, there may be an effect on time to disposition. Since the study period is only a year and given that it takes an average of seven months to resolve a criminal case, it is reasonable to expect an effect on the time to resolve criminal cases. In contrast, it takes on average four years to resolve a civil case, so it may be harder to detect an effect on the time to disposition of civil cases--however, some categories of civil cases are resolved faster than others, so we will narrow down on these categories by considering all case types that are resolved (on average) within the course of a year.

• Backlog (percentage of cases that have been pending for more than 1 year)

• Age of case resolved (to see whether judges decide to resolve new versus older cases)

• Time taken for case hearing

• Time taken from filing to first hearing of cases

• Legal representation

• Quality: appeal rates, conviction, dismissal

• Access: total caseload

From digitization of cases extracted from Kenya Law database at the High Court level:

• assignment of judge to litigant [both a check-of and distortion of random assignment]

• decision direction

• whether the decision is appealed

• citation count of the decision, length of decision, number of laws/acts cited in the judgement, how many times the judgement in question has been cited

• time to disposition [if scrapeable from the text of the data]

• number of published cases

• gender slant, gender access to courts

• If possible to identify economic disputes (contracts, security of property rights, credit), number of such cases; time to disposition and direction of decision in favor of certain parties (creditors versus lenders, or big firms versus individuals)

For the above data at the High Court level, the research plan includes further subgroup analyses by:

• Proportion of lower courts treated (since cases for lower-level courts will percolate the High Courts)

Additionally, a daily event study when the lower court decisions percolate into the higher court will be implemented. The dates of the lower court filing or resolution might be used as separate event study dates along side the treatment dates of the lower courts, to check whether the quality of decisions at the lower-level courts impacts the process of cases at the High Court level.

Other analysis if data is available:

• If court user satisfaction surveys are available: satisfaction with court services

• If employee surveys are available: use of case management, motivation

• If CUC meeting minutes are available: procedures put in place to deal with adjournments

• If PMMU evaluations are available: evaluation score, court use of funds (compliance of courts with budget utilization of allocated funds), attendance of judges to the yearly judicial training

If there is an effect on adjournment and time to disposition (especially if the additional feedback forms are sent in 2020 and 2021 every month for a duration of one year), then we may expect downstream economic effects:

• If Kenya Integrated Household Budget Survey: investment, business creation, access to credit, consumption

• If World Bank Enterprise Surveys is available: contracting behavior, trust in courts, courts as obstacles to business

Multiple hypothesis testing will be done with the Sharpened False Discovery Rate (FDR) adjusted q-values (Anderson, 2008).

The unit of randomization is the court station level. A court station is a physical compound that may contain several courts. One station has one Court User Committee (CUC). There are 124 court stations in Kenya. There are three groups of equal size:

• Court stations receiving the feedback report every month

• Court stations receiving the feedback report every month, shared with local court user committees (CUC) and discussed at quarterly CUC meetings

• A control group of stations

To randomize, we stratified the sample on geographical variables (the Kenyan judiciary established a list of 8 regions that closely match the eight existing regions of Kenya) and baseline time to disposition in the station.

The randomization was done in office by a computer.

The unit of randomization is the court station level.

Yes

The planned number of clusters is the 123 court stations in Kenya.

The planned number of observations is the universe of cases going through the courts in Kenya.

41 court stations receiving the feedback report, 41 receiving the feedback report shared with local court user committees (CUC), 41 court stations control.