Intervention (Hidden)
The Department of Institutional Development (DDI) of the Chilean judicial system developed an electronic platform called Quantum in 2018. Quantum displays comprehensive information on court indicators, such as the number of cases filed, the case clearance rate, the average duration of cases finished in a month, and the percentage of realized hearings. It also allows users to compare their courts to other courts in the same jurisdiction. Quantum was launched in 2018, but take-up has been limited: 20% of court managers have never logged in, and overall there is an average of 20 logins per court manager in 1 year and 2 months. The platform is technologically well developed and rich in information, yet it is unclear whether it has any impact on the management of the court. Our project consists of evaluating an intervention through a randomized controlled trial (RCT) with two main branches. First, we will (randomly) promote the Quantum platform in multiple ways, such as sending court managers a survey that implicitly markets the platform, making phone calls, and sending them emails. Second, we will (also randomly) provide court managers a new dashboard that summarizes the main statistics displayed in Quantum and compares them to themselves to a reference group of courts. There would be a total of six distinct intervention or treatments:
Treatment 0: Control. No change to their Quantum dashboards nor provided with any Quantum promotion.
Treatment 1: Quantum Promotion
The tribunals randomized into Treatment 1 will have their court staff receive both emails with a Quantum link to increase accessibility and salience of Quantum, and a small baseline and post-intervention survey that includes Quantum promotion. The baseline survey given to the court managers at the beginning of the RCT will ask them about their beliefs about some productivity metrics, how much these metrics affect their decision at work, inform them that these metrics can be seen in Quantum, and describe the effect of Quantum usage on people’s productivity through the results of an event study using historical data. A sample survey is provided in the appendix.
Treatment 2: No Quantum Promotion + New Dashboard
The tribunals randomized into treatment 2 will not receive any promotion but will have their home page in Quantum, what we call the dashboard, present various statistics at the tribunal level.
Treatment 3: Quantum Promotion + New Dashboard
The tribunals randomized into treatment 3 will receive the same promotion as that in treatment 1 and the new dashboard as in Treatment 2.
Treatment 4: No Quantum Promotion + New Dashboard + Comparative to others
The tribunals in treatment 4 will receive the new dashboard plus another tab or pop-up window that focuses on the tribunal’s best performing and worst-performing dimension from the previous month in comparison to the performance of peer tribunals (same competence) in the same month. This comparison leans into social comparison motivation.
Treatment 5: Quantum Promotion + New Dashboard + Comparative to others
The tribunals in treatment 4 will receive the new dashboard plus another tab or pop-up window that focuses on the tribunal’s best performing and worst-performing dimension from the previous month in comparison to the performance of peer tribunals (same competence) in the same month. This comparison leans into social comparison motivation.
The court managers' job satisfaction level will be measured with pre and post-surveys to court managers that measure perceptions of their tribunals and their satisfaction with their positions. By informing the court managers about their court's standing in the new dashboard and comparative to others, this could change how empowered or satisfied a court manager is with his or her position and power role.
Overall, the objectives of the survey for court staff are threefold. First, it will measure their knowledge of Quantum statistics. How close or far is their perception of their court’s performance from the truth. We can later use this information (prior beliefs) to understand if greater access to Quantum updates the beliefs closer to the truth when we measure their posteriors (survey at the end of intervention). Notice that the measurement of beliefs and opinions is something unique to the survey that the rest of the interventions cannot. Second, the survey will allow us to understand if the court staff find the statistics important and in what order of importance. This is useful for Quantum to know which variables are important for users and make them more salient in the dashboard or in the rest of the Quantum pages. That is, the results from the survey can help tailor the intervention to make it more effective. Third, the survey will promote Quantum as a source of accurate and useful information through the event study results. This may help influence those that are skeptical of Quantum to give it a chance.