Reducing Violent Crime through Improved Police Performance: An Experimental Evaluation of the Policing Leadership Academy

This study is an evaluation of The University of Chicago Policing Leadership Academy (PLA). The PLA is a five-month, comprehensive management and leadership education program for police managers, developed and delivered by a team of experienced police leaders from two of the largest US metropolitan police departments. The PLA is designed for police managers with responsibility and oversight over patrol operations in a defined geographic area, such as the commander of a district in a large police agency or a chief overseeing all patrol operations in a midsize agency.

The PLA curriculum is designed to provide a mix of (1) general management skills of the kind taught in a top-tier MBA program; (2) domain-specific understanding of best-practice violence prevention strategies in policing; (3) domain-specific understanding of topics in policing on which there is growing consensus, such as procedural justice, community relations, and the role of trauma among both residents and officers alike; and (4) the skills to integrate and apply this knowledge through real-world exercises. The instructional team includes professors from the University of Chicago, University of Pennsylvania, and other academic institutions, as well as management experts from law enforcement, the military, government, the private sector, and community organizations working to reduce violence.

The curriculum is delivered over five consecutive months, with one week per month of in-person instruction totaling approximately 120 instructional hours. Instruction is tailored for adult learning, drawing on transformational, self-directed, flip, and experiential learning methods. Between in-person sessions, students complete asynchronous coursework and attend a monthly virtual course session. The students also deliver “micro-trainings” to officers under their command developed by PLA curriculum leads.

Throughout this course of study, each student completes a capstone project, addressing a relevant issue or need in their home agency by applying tools and concepts from the curriculum. The student works with their chief to propose and begin to implement a project. During the final week, students present their capstone projects to a panel of three reviewers representing law enforcement, the community, and city government.

2023-05-22

2027-12-31

In addition to receiving training aimed at improving general management and leadership skills, police managers attending the PLA receive instruction on developing strategies to reduce violence and other crime while minimizing harm to the affected communities. We will measure the PLA’s effect on a set of outcomes covering the breadth of focus areas addressed in the PLA program, including measures designed to explore shifts in police managers’ perceptions, reported behaviors, and adoption of practices that are likely to improve police-community relations. These measures will be derived from primary data collected through interviews with randomized commanders and others at their departments.

Using police administrative data, we will also track outcomes related to how frontline police carry out their duties in areas overseen by randomized commanders, as well as key public safety outcomes such as violent crime and other types of crime that influence citizens’ perception of safety in their communities. Specific primary and secondary outcomes will be described in further detail in our pre-analysis plan.

We estimate the impact of the PLA on violent crime and police practices using a large-scale, multi-city RCT. The study design incorporates two distinct approaches to recruitment and randomization among police agencies depending on their size. For large metropolitan police agencies, we randomize between high-violence police districts within agencies. The current and subsequent managers of the treatment group districts are invited to attend the PLA and outcomes are tracked at the district level. For midsize police agencies—those with populations of at least 40,000, but that are too small to support within-agency randomization—we randomize entire departments into the treatment and control groups. Current and subsequent managers at treatment departments are invited to attend the PLA, and we track outcomes over the entire jurisdiction of the randomized department.

The sample selection process begins by identifying agencies to invite to participate in the study. Guided by both a desire to have impact at scale and practical considerations related to the research design, agencies are identified based on three factors:

1. Part 1 violent crime incidents (homicides, rapes, aggravated assaults, and robberies) occurring within their jurisdiction according to the FBI’s Uniform Crime Reporting (UCR) Program and National Incident-Based Reporting System (NIBRS), or public crime data published by individual cities;
2. Number of geographic patrol districts or precincts; and
3. Availability of administrative data for measuring key outcomes.

Invited agencies large enough to support within-agency randomization are asked to identify districts meeting a minimum violence threshold, which are then paired based on crime and demographic characteristics. A random lottery assigns one district in each pair to treatment and control groups, assignments that will remain in place for the duration of the study.

Midsize agencies with too few violent districts to support within-agency randomization must meet a minimum citywide violence threshold to be considered for department-level randomization. Agencies that meet this threshold are paired with other eligible cities based on crime and demographic characteristics, and a random lottery assigns one agency from each pair to treatment and control groups.

After randomization occurs, a manager of the treatment unit (district or department) will be invited to attend an upcoming PLA cohort. To the extent possible, we will invite additional managers from the treatment unit to attend the PLA in later cohorts, particularly in cases where the original manager leaves the treatment unit due to promotion or reassignment.

We will measure the PLA’s effect on public safety using police administrative data. We are in the process of negotiating data use agreements with large police agencies to collect administrative data directly from departments that enable district-level tracking of outcomes. For midsize police agencies, we will rely primarily on publicly available administrative data reported to the NIBRS.

We will also collect novel survey and interview data to assess the impact of the PLA on leadership approaches and on the quality of management practices, directly measuring whether the intervention achieves its goal of improving police management. To measure these outcomes, we draw on primary data collected via interviews administered to treatment and control group members at baseline and again one-year post-PLA.

To estimate the effects of inviting the police manager(s) of a district (or department, in the case of midsize agencies) to attend the PLA, we will use ordinary least squares (OLS) regression to estimate a two-way fixed effects (TWFE) difference-in-differences specification with fixed effects for treatment units and time periods relative to randomization. The coefficient of interest is an indicator for whether a district is currently assigned to treatment. Due to random assignment of treatment, this coefficient provides an unbiased estimate of the intent-to-treat (ITT) effect of the PLA. This ITT estimate represents the treatment-control difference in the trend of the outcome in the post-treatment period relative to the pre-treatment period. The advantage of this approach is that the inclusion of district fixed effects explains a large amount of the variation in the outcome, allowing for much more precise estimation than would otherwise be possible with a given number of experimental districts. Because randomization will occur in pairs, the treatment assignment indicator is mechanically negatively correlated across districts within the same randomization block. As a result, we will estimate heteroskedasticity-robust standard errors clustering at the randomization block level.

We are also interested in estimating treatment-on-the-treated (TOT) effects of a district actually being overseen by a PLA-educated manager, rather than simply having its manager be invited to attend the PLA. To do this, we will use two-stage least squares (2SLS) to estimate an analogous specification, using the treatment assignment status of a district as an instrumental variable (IV) in the first stage to estimate the TOT effect in the second stage. The recent literature on TWFE estimators shows that they can produce biased estimates of average treatment effects in settings where treatment effects are heterogeneous and treatment adoption is staggered. Fortunately, in our context, even though treatment effects may be (and likely are) heterogeneous, treatment adoption, within pairs, is not staggered: each pair of districts is randomly assigned at the same time either to treatment or control, and comparisons are only made within this randomization block. There will therefore never be a situation in which an earlier-treated district acts as a control for a later-treated district.

Not available

Using a computer script, a lottery randomly assigns one district (or department, in the case of midsize agencies) within each pair to treatment and the other to control. These assignments will remain in place for the duration of the study. The randomization script and accompanying documentation will be made public upon publication of the study.

For large agencies, the randomization unit is a police district or equivalent departmental subdivision (e.g., police precinct, police division). For midsize agencies, the randomization unit is the entire department. Each unit is paired with another using crime and demographic characteristics prior to paired randomization.

No

We expect to enroll approximately 120 districts from large agencies and 80 midsize departments for randomization through nine planned PLA cohorts, for a planned total of approximately 200 randomization units.

The design is not clustered; therefore, our planned number of observations is also 200 randomization units (120 districts + 80 midsize departments).

One unit per randomization pair is assigned to treatment (manager of unit receives PLA training) and one is assigned to control (manager does not receive PLA training). We therefore expect 100 units in the treatment group and 100 units in the control group through nine planned PLA cohorts.