x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Making Information Actionable: Experimental Evidence from Kenyan Courts
Last registered on July 24, 2020

Pre-Trial

Trial Information
General Information
Title
Making Information Actionable: Experimental Evidence from Kenyan Courts
RCT ID
AEARCTR-0006228
Initial registration date
July 24, 2020
Last updated
July 24, 2020 12:05 PM EDT
Location(s)
Region
Primary Investigator
Affiliation
McGill University
Other Primary Investigator(s)
Additional Trial Information
Status
In development
Start date
2018-12-01
End date
2022-01-01
Secondary IDs
Abstract
Courts in developing countries face numerous constraints that prevent them from providing efficient and fair justice to citizens, or a strong institutional environment conducive to investment and growth. We ask: can low-cost, information-based interventions, using data regularly captured by administrative systems, help judicial officers overcome common incentive and behavioral constraints, and in so doing improve court performance? This note describes the pre-analysis plan of an intervention that will be implemented together with the World Bank and the Kenyan judiciary.
External Link(s)
Registration Citation
Citation
Chemin, Matthieu. 2020. "Making Information Actionable: Experimental Evidence from Kenyan Courts." AEA RCT Registry. July 24. https://doi.org/10.1257/rct.6228-1.0.
Experimental Details
Interventions
Intervention(s)
This study will test two interventions:

1. Actionable information: Providing judicial officers a one-page, simplified feedback form every month, that shows them 1) the case clearance rate (CCR; number of cases resolved divided by number of cases filed), 2) the number of adjournments and their top three reasons, 3) the predicted impact of reducing adjournments on the CCR

2. Actionable information with deliberation: As above, but also sharing the information with Court User Committees (CUCs) that include representatives from public security agencies and local communities, to focus discussion around performance, create bottom-up accountability, and collectively arrive at ways to remove the bottlenecks.

The feedback forms were sent once in January 2019. The goal will be to send additional feedback forms in 2020 and 2021 every month for a duration of one year.
Intervention Start Date
2019-01-01
Intervention End Date
2021-12-01
Primary Outcomes
Primary Outcomes (end points)
The primary outcome is the proportion of hearings ending in an adjournment.

We will use a difference-in-differences analysis, with court and month fixed effects.

The research plan includes subgroup analyses by:

• Baseline time to disposition

• Baseline number of adjournments, disaggregated by their main reasons

• Strength of feedback (i.e., 1) spread between CCR and recommended target, 2) number of adjournments reported, and 3) size of predicted impact)

• Nature of top reasons of adjournments

• Type of court (high court, magistrate court, emloyment and labour relations court, environment and land court)

• Type of case (civil (e.g., succession, commercial), criminal (e.g., property, violent))

The randomization was stratified on 1) regions (8), and 2) time to disposition. These stratification dummies will be added to the model.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
• Adjournments disaggregated by reason for adjournment. In particular, adjournments can be “internal”, i.e., under the control of the judge (e.g., court not sitting) or “external”, i.e., caused by other parties (e.g., lawyer not ready).

• Case clearance rate (cases disposed of divided by cases filed) as a standard measure of productivity of courts, as well as cases disposed, cases filed

• Time to disposition: Considering the feedback forms were only sent once in January 2019, it may seem unreasonable to expect an effect on the time to disposition of cases. If additional feedback forms can be sent in 2020 and 2021 every month for a duration of one year, there may be an effect on time to disposition. Since the study period is only a year and given that it takes an average of seven months to resolve a criminal case, it is reasonable to expect an effect on the time to resolve criminal cases. In contrast, it takes on average four years to resolve a civil case, so it may be harder to detect an effect on the time to disposition of civil cases--however, some categories of civil cases are resolved faster than others, so we will narrow down on these categories by considering all case types that are resolved (on average) within the course of a year.

• Backlog (percentage of cases that have been pending for more than 1 year)

• Age of case resolved (to see whether judges decide to resolve new versus older cases)

• Time taken for case hearing

• Time taken from filing to first hearing of cases

• Legal representation

• Quality: appeal rates, conviction, dismissal

• Access: total caseload

From digitization of cases extracted from Kenya Law database at the High Court level:

• assignment of judge to litigant [both a check-of and distortion of random assignment]

• decision direction

• whether the decision is appealed

• citation count of the decision, length of decision, number of laws/acts cited in the judgement, how many times the judgement in question has been cited

• time to disposition [if scrapeable from the text of the data]

• number of published cases

• gender slant, gender access to courts

• If possible to identify economic disputes (contracts, security of property rights, credit), number of such cases; time to disposition and direction of decision in favor of certain parties (creditors versus lenders, or big firms versus individuals)

For the above data at the High Court level, the research plan includes further subgroup analyses by:

• Proportion of lower courts treated (since cases for lower-level courts will percolate the High Courts)

Additionally, a daily event study when the lower court decisions percolate into the higher court will be implemented. The dates of the lower court filing or resolution might be used as separate event study dates along side the treatment dates of the lower courts, to check whether the quality of decisions at the lower-level courts impacts the process of cases at the High Court level.

Other analysis if data is available:

• If court user satisfaction surveys are available: satisfaction with court services

• If employee surveys are available: use of case management, motivation

• If CUC meeting minutes are available: procedures put in place to deal with adjournments

• If PMMU evaluations are available: evaluation score, court use of funds (compliance of courts with budget utilization of allocated funds), attendance of judges to the yearly judicial training

If there is an effect on adjournment and time to disposition (especially if the additional feedback forms are sent in 2020 and 2021 every month for a duration of one year), then we may expect downstream economic effects:

• If Kenya Integrated Household Budget Survey: investment, business creation, access to credit, consumption

• If World Bank Enterprise Surveys is available: contracting behavior, trust in courts, courts as obstacles to business

Multiple hypothesis testing will be done with the Sharpened False Discovery Rate (FDR) adjusted q-values (Anderson, 2008).
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The unit of randomization is the court station level. A court station is a physical compound that may contain several courts. One station has one Court User Committee (CUC). There are 124 court stations in Kenya. There are three groups of equal size:

• Court stations receiving the feedback report every month

• Court stations receiving the feedback report every month, shared with local court user committees (CUC) and discussed at quarterly CUC meetings

• A control group of stations

To randomize, we stratified the sample on geographical variables (the Kenyan judiciary established a list of 8 regions that closely match the eight existing regions of Kenya) and baseline time to disposition in the station.
Experimental Design Details
Not available
Randomization Method
The randomization was done in office by a computer.
Randomization Unit
The unit of randomization is the court station level.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
The planned number of clusters is the 123 court stations in Kenya.
Sample size: planned number of observations
The planned number of observations is the universe of cases going through the courts in Kenya.
Sample size (or number of clusters) by treatment arms
41 court stations receiving the feedback report, 41 receiving the feedback report shared with local court user committees (CUC), 41 court stations control.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
McGill University Research Ethics Board
IRB Approval Date
2020-07-08
IRB Approval Number
20-06-027