Selection Neglect in Policing Decisions

Block 1 is designed to estimate individual-level cognitive biases—particularly selection neglect. Participants engage in a belief updating task where they are shown a series of hypothetical neighborhoods within a city. Each neighborhood has a true (but unobserved) level of crime. Participants are told that crime levels across neighborhoods follow a normal distribution, and they are provided with the variance of that distribution.

Over the course of 15 rounds, participants receive information about one randomly drawn neighborhood per round. For each neighborhood, they observe (i) the number of reported crimes and (ii) the reporting rate—that is, the proportion of crimes that are reported to the police. Importantly, participants must recognize that the true number of crimes can be inferred by dividing the reported crimes by the reporting rate.

Using the variation in the information shown across rounds, we estimate two structural parameters for each participant:

1. Their Bayesian updating weight.
2. Their propensity to exhibit selection neglect—that is, the extent to which they fail to account for reporting rates when inferring crime levels.

In Block 2, participants are asked to compare two neighborhoods belonging to two different districts. Crime levels within each neighborhood are normally distributed, but the two districts differ in their means. Without seeing any specific neighborhood information in advance, participants must infer which neighborhood is likely to have the higher expected crime level and decide where to send a patrol. Their decision must therefore rely on information accumulated across previous rounds.

After making their patrol decision, participants are shown the number of reported crimes and the reporting rate for both neighborhoods. A key feature of this design is that the neighborhood receiving the patrol always has a highly precise reporting rate of 95%, while the other neighborhood has a less precise rate strictly below 95%. Every five rounds, participants are also asked to state their guess of the mean crime level for the neighborhood.

Block 1 is designed to estimate individual-level cognitive biases, particularly selection neglect. Participants are shown a series of hypothetical neighborhoods within a city, each with a true but unobserved crime level. They are informed that crime levels across neighborhoods follow a normal distribution, and they are given the variance of that distribution.

Over 15 rounds, participants receive information about one randomly drawn neighborhood per round. For each neighborhood, they observe (i) the number of reported crimes and (ii) the reporting rate, which is the proportion of crimes reported to the police. To infer the true crime level, participants must recognize that they need to divide the reported crimes by the reporting rate. Participants are asked to infer the mean crime levels in the neighborhoods of the city.

The variation in information across rounds allows us to estimate two structural parameters for each participant:

- Their Bayesian updating weight.
- Their propensity to exhibit selection neglect, meaning the degree to which they fail to account for variation in the reporting rate when inferring crime levels.

Block 2 extends the task by introducing an experimental intervention. Participants are presented with two neighborhoods drawn from different districts. While crime levels in each neighborhood are normally distributed, the two districts differ in their means. Without observing specific neighborhood-level data in advance, participants must infer which neighborhood has the higher expected crime level and decide where to send a patrol, relying on information accumulated in earlier rounds.

After making their patrol decision, participants are shown the number of reported crimes and the reporting rate for both neighborhoods. A key feature is that the patrolled neighborhood always has a highly precise reporting rate of 95 percent, while the non-patrolled neighborhood has a less precise rate strictly below 95 percent. Every five rounds, participants are also asked to state their guess of the mean crime level of the neighborhoods of each district.

To test the influence of external decision cues, participants in Block 2 are randomly assigned to one of three conditions:

- Control Group: Participants see only the raw data and make their decision unaided.
- Correct Recommendation Treatment: An "intelligence agency" recommends to patrol
- Incorrect Recommendation Treatment: The agency recommends the neighborhood with the lower crime level

This intervention mimics real-world decision support systems in policing and allows us to test whether external cues help improve decision-making or instead exacerbate selection biases.

2025-09-12

2025-09-13

In Block 1, we estimate two outcome variables at the individual level. The first is a parameter that captures the degree of selection neglect—that is, the extent to which a participant fails to account for the fact that observed crime data is shaped by civilian reporting behavior. A higher value of this parameter indicates a greater tendency to overlook the selective nature of reported crime data. The second is the individual's Bayesian updating weight (i.e., kallman gain), a standard parameter in the Bayesian learning literature. See attached pdf to learn more about the model.

Importantly, we can recover both parameters using only participants’ crime-level guesses across rounds. Identification relies on the randomization of the information shown to participants in each round, which enables us to analyze how variation in observed data influences their predictions.

In Block 2, we measure participants’ prediction accuracy, defined as the number of correct responses across rounds. We calculate accuracy separately for each treatment group and the control group to evaluate the impact of our interventions by comparing the number of correct answers in each treatment group.

For the model description, please see the attached file. Basically each parameter will be estimated using maximimum likelihood. The randomization in the crime reports and the reporting rates allows us to estimate each parameter separatedly at the individual level.

The outcomes in part 2 are easy to construct. There will always be one neighborhood with the correct higher predicted crime level based on previous data if participants do the back up of the data (from reports to crime levels using the reporting rate) correctly. We count how many of the decisions of the participants are consistent with this way of thinking.

There will be variation also in the average reprting rate that each participant observes. In particular, we will be dividing participants into groups where the average reporting rate is higher than 100% and lower than 100%. Because of that, we expect that the participants in the group with a lower reporting rate are going to have lower guesses than the other group. This would be a manifestation of selection neglect because they are not using the reporting rate information to back up their beliefs on crime. We will compare the deviations from the true main crime rate across groups, as well as how close they get to the real mean crime rate of the city across rounds.
For block 1 we will also compare the parameters estimated for different chunks of the data. This will be helpful because it can be the case that some participants learn about the selection process throughout the game and therefore register a much lower selection neglect for later rounds of the game.

In block 2 we will measure the frequency of switches between neighborhoods. This will tell us about the value of exploration that agents put in gathering more precise information from the patrolls.

For both experiments we register their response times, the number of times they use the calculator and click on the information of data of past rounds and the operations that they do on the computer.

We conduct a framed field experiment with Colombian police officers to measure selection neglect and its effect on belief updating. In Block 1, participants observe noisy information on reported crimes and reporting rates across neighborhoods from a city and then to then make a guess on the average number of crimes that take place in the neighborhoods of a city. We use a Bayesian framework to estimate (1) the extent of selection neglect and (2) the relative weight placed on prior beliefs.

In Block 2, participants compare crime levels across two neighborhoods based on reported crime and reporting rates. We introduce two interventions: one where an intelligence agency provides a correct recommendation and another where it gives an incorrect one. We compare these to a control group to assess how external recommendations influence decision accuracy.

In the first block of the experiment, participants are shown the distribution of crime levels across neighborhoods within a given city. Then, over the course of 20 rounds, they observe both the number of reported crimes in a specific neighborhood and the reporting rate for that neighborhood. The number of reports is constructed as a noisy signal of both the true crime level and the reporting rate. Using this information, participants are asked to estimate the underlying crime level of the neighborhood in each round.

We model this belief-updating process within a Bayesian framework, which allows us to estimate two key parameters: (1) the degree of selection neglect, i.e., the extent to which participants fail to properly account for the reporting rate when inferring crime levels; and (2) the relative weight participants place on prior versus posterior beliefs.

In the second block, participants are presented with information from neighborhoods in two different cities. For each neighborhood, they receive only two pieces of information: the number of reported crimes and the reporting rate. Based on this, they make an incentivized guess as to which neighborhood has the higher true crime level.

We introduce two interventions in Block 2 to examine whether prediction accuracy improves or deteriorates. In the real-world context of the Colombian police, an intelligence agency analyzes crime reports to infer differences in crime levels across neighborhoods and uses this to guide patrol allocation. We leverage this institutional feature to conduct an environmentally valid intervention. Specifically, we inform participants of the intelligence agency’s recommendation regarding which neighborhood has the highest level of criminality.

In one treatment, the agency provides an incorrect recommendation (i.e., identifying the lower-crime neighborhood as the higher one), while in the other, it provides the correct recommendation. We then compare prediction accuracy across the control group and each treatment group to assess the influence of external recommendations on officers' decision-making.

Randomization done in office by a computer

Individual

No

Between 200 and 300 Police officers were invited to participate. Attritionis uncertain

We estimate that between 100 and 200 officers

Each arm will be (in expectation) of equal size and it will depend on the final number of participants.