Improving Legal Training: the Impact of Social-Emotional Learning and Class Monitoring on Judicial Performance

Last registered on February 12, 2021

Pre-Trial

Trial Information

General Information

Title
Improving Legal Training: the Impact of Social-Emotional Learning and Class Monitoring on Judicial Performance
RCT ID
AEARCTR-0007113
Initial registration date
February 11, 2021
Last updated
February 12, 2021, 11:04 AM EST

Locations

Region

Primary Investigator

Affiliation
The World Bank

Other Primary Investigator(s)

PI Affiliation
UCLA
PI Affiliation
The World Bank

Additional Trial Information

Status
On going
Start date
2020-05-13
End date
2021-09-01
Secondary IDs
Abstract
Despite the importance of high-stake judicial decisions on litigants' wellbeing and economic development, there is little evidence on how to improve judicial performance. In this paper, we use a randomized controlled trial to evaluate the impact of teacher monitoring and social-emotional exercises on the performance of judges and prosecutors in Peru's Judicial Academy. We test the impact of our interventions on both soft and hard skills that aim to improve not only educational attainment and social-emotional skills, but also contribute to fairer and more efficient judicial decision-making.
External Link(s)

Registration Citation

Citation
Chen, Daniel, Manuel Ramos Maqueda and Bernardo Silveira. 2021. "Improving Legal Training: the Impact of Social-Emotional Learning and Class Monitoring on Judicial Performance." AEA RCT Registry. February 12. https://doi.org/10.1257/rct.7113-1.0
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
First, we are implementing an online monitoring program in which classes are observed by a member of the agency’s pedagogical team and teachers receive personalized feedback after each monitored session. Each class in the monitoring group of this intervention is monitored by a member of Judicial Academy's (AMAG's) pedagogical team. The monitor joins the day-long classes for a total of one hour divided into four slots of fifteen minutes spread across the duration of the class. After each class, the teacher meets with the monitor to receive and discuss her feedback during a 30-minute follow-up meeting. Classes in the control group are not monitored and teachers do not receive any feedback.

Second, we are applying online self-reflection exercises to the AMAG's students. Once a month, students will be asked to reflect and write about topics related to the work of judicial professionals. The treatment group will be asked to reflect about their personal experiences whereas the control treatment group will be asked about the same topic but reflecting on others' experience as a judge or prosecutor. These questions will change every month from a list of 9 topics, each of which has a treatment and a control question. To ensure that the writing task is effective, a minimum of words will be required and answers will be peer-reviewed. At the end of the month, students will be asked to rate their peers’ (anonymized) responses to the exercise in terms of how useful they are as advice for future judges and prosecutors. Each student will rate two pairs of responses, selecting their preferred response for each pair.
Intervention Start Date
2020-05-23
Intervention End Date
2020-12-31

Primary Outcomes

Primary Outcomes (end points)
There are 3 main sets of outcomes of interest:
1. Educational attainment: this includes student’s grades, attendance, participation in class, and satisfaction with the course.
2. Empathy and bias: at the end of the program, an exit survey will evaluate a set of outcomes of judges and prosecutors. These outcomes will include a dictator game, a redistribution game, an honesty test, an Implicit Association Test and the decision to choose a book on a topic related to morality, law and economics, gender in law or judicial independence.
3. Professional performance: these outcomes refer to judges’ and prosecutors’ performance on the job. We aim to measure their productivity via the following indicators: case clearance rate, case length, fraction of decisions appealed or reversed, length and direction of judicial decisions, textual measures of implicit bias, and future promotion.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Immediate short-term outcomes will also include program participation and the grade of the peer-review ratings on their colleague’s responses to the social-emotional learning interventions. Secondary outcomes might also include the income and employment of those litigants and firms who might be affected by the decisions of these judges and prosecutors.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The research design consists in a randomized controlled trial (RCT) of two main interventions: online monitoring of teachers during classes, and social-emotional learning (SEL) exercises written by the students.

Online monitoring will take place every class (twice a month), and will be randomized at the class level. Those students whose teachers are monitored in the first round will remain in the monitoring group in future courses. The teacher will likely change every month (course), yet the new teacher will also be monitored if the previous teacher was monitored too.

The SEL exercises will take place once a month, when students will be asked to reflect and write about topics related to the work of judicial professionals. At the end of the month, students will review two pairs of anonymized responses from their peers and select their preferred ones from each pair. The SEL topics will not only be randomized across students in each course; the order in which students complete such topics will also be randomized. This will allow us to study the comparative effect of each of these topics, and thus understand which interventions are most helpful to improve educational attainment and professional performance of judges and prosecutors.
Experimental Design Details
Randomization Method
Randomization done by a computer.
Randomization Unit
We need to distinguish the two interventions. For the monitoring intervention, the unit of randomization will be the class. For the SEL intervention, the randomization unit is the student.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
For the monitoring intervention, there are 22 classes (clusters) over the period of 9 courses. The classes remain on the same group (treatment or control) during the entire program. We cluster at the class level, program level, course type, and location. Given that some locations only have 1 class per program, level and course, we aggregate all of the single locations in one common group.

For the SEL intervention, there are 600 students who receive 10 topics to write about in random order, one topic for each course from course two onwards. For each topic, whether the student is assigned the treatment question or the control question is also randomly assigned. The treatment/control randomization for each question is stratified based on the following variables: level of the program, location of student, class, and profession (judge or prosecutor).
Sample size: planned number of observations
SEL: 600 students for the SEL intervention over the period of 8 months. Monitoring: There are 22 classes for the monitoring of teachers over the period of 9 months, affecting 600 students in total.
Sample size (or number of clusters) by treatment arms
Monitoring: 11 classes in treatment, 11 classes in control; over the period of 9 months.
SEL: 60 students per treatment arm stratified by level of the program, location, class, and profession (judges or prosecutors) over the period of 8 months.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials