State-building on the Margin: An Urban Experiment in Medellín

NOTE: We participated in the JDE pre-results review process after the original registration but before the collection of results. That review contains the most recent analysis plan. See attached analysis plan a copy of this pre-results review.

The city government aims to improve governance and increase service delivery to certain sectors (neighborhoods). The aim is to increase the strength and legitimacy of the state at the expense of organized criminal governance. The intervention will begin in Medellin and within the first year we expect to expand the program and evaluation to neighboring municipalities.

The office of the Secretary of Security in Medellin currently has a staff of city liaisons who work to maintain strong relationships with between the government and the community.

The street-level staff members in these units are community organizers and city liaisons ("community organizer" for short). Normally, the Secretariat of Security has 1-2 community organizers per comuna -- about 1 per 6,000 to 12,000 people. For the intervention, the Secretariat of Security will aim to have their usual one community organizer for the comuna, plus an additional 40 community organizers dedicated to 40 small sectors assigned to treatment. A sector is an informal neighborhood within a comuna and typically has an area between 1.5 to 4.5 Hectares.

Thus, the city will aim to provide treated sectors with roughly one city community organizer per 1300-2000 people. The city has very poor population estimates, however (a sign of state weakness), and so these population figures are at best guesses.

The main roles of the community organizers will be to:

1. Encourage the organization and functioning of community governance organizations and social groups, including locally elected bodies.
2. Connect residents to appropriate dispute resolution bodies in the city government, including the police, courts, or dispute resolution officials and train them in effective communication and dispute resolution skills.
3. Coordinating delivery of existing city services where needed (such as education, health, welfare, legal, and maintenance services).

Community organizers are not just simple liaisons. The idea is for them to interact with the community, get to know people individually, identify problems, capabilities and social capital, as well as the nature of criminal governance in the specific neighborhoods, to build the solutions from the bottom up. This implies there is not a predetermined strategy from the top, but rather that the day to day activities by these community organizers should be adaptive.

Community organizers are not the only face of the state but an interface that eases the way in which governance is provided. Once community organizers have identify main problems of the community they can use the networking of the liason of the Secretary of Security to address the problem to the right municipal agency. They should also identify whether community governance organizations are able to substitute for public officials as a first resource, and request the strengthening of these organizations with dispute resolution techniques that would be addressed with training received by the community organizers.

2018-02-01

2019-12-31

We are principally interested in changes in (a) specific governance roles and services provided by the three relevant sets of actors the combo, the state, and community leaders and institutions; and (b) the relative perceived legitimacy of three actors.

See the attached analysis plan for specifics (part of the JDE pre-results review process completed after the original registration but before the collection of results).

We expect the intervention to take at least two years for impacts, but will check for intermediate impacts after the first year.

We have yet to determine how we will specifically measure governance and legitimacy of these different actors. Similar interventions and measures are rare. These outcomes are difficult to conceptualize as well as measure. We will pilot various measurement tools and strategies in the first months of the intervention, and update the registration before collecting systematic endline data. Currently, the main candidate approach is a population-based survey of each sector where we measure perceived legitimacy of each actor plus citizen demand for and use of various services (e.g. dispute resolution, permits) and to which authority they go to for these services.

Outcomes will be finalized and pre-registered prior to the endline survey, after qualitative observation of the intervention and protesting of measures.

Other outcomes of interest include:
1. Resident survey-based measures of perceived security
2. Crime reports
3. Application for subsidies and public services

To the extent they can be measured, we will also seek to measure “first-stage” outcomes to measure levels of service delivery and personnel per neighborhood sector.

See the attached analysis plan for specifics (part of the JDE pre-results review process completed after the original registration but before the collection of results).

We do not have strong priors about an increase or decrease in crime, and hence these are secondary outcomes.

For our experimental sample the city initially identified 97 sectors with major presence of criminal organizations. The research team eliminated the 13 sectors that were too close to one another (to maximize spread of experimental sectors) and dropped the 4 sectors with the lowest combo presence or governance (according to the baseline data). This led to a final experimental sample of 80. 40 were assigned to treatment used blocked randomization.

Randomization was done in an office using a computer. We blocked the sectors into pairs based on a measure of multivariate “distance” between one another. Given the relatively small number of sectors, we matched sectors based four variables: an index of combo presence and visibility, an index of combo governance, a measure of perceived insecurity, and an index of administrative crime. Within each blocked pair of sectors, we selected one to be treated randomly, such that all sectors have an equal probability of selection into treatment. Specifically, this process was done using the Blocktools package in R.

The unit of analysis is an informal neighborhood called a sector. These are discussed above.

Initially the study will be conducted in Medellin, but we expect to be able to expand the experiment to neighboring municipalities in the Medellin metro area, to enhance statistical power.

Medellín is divided administratively into 16 comunas (urban) and 5 corregimientos (semi-rural) ("comuna" for short). The comunas are divided into neighborhoods or veredas (in the case of corregimientos). Each neighborhood or vereda is divided into "sectors." The sector is not a formal administrative unit, but rather an informal territory that contains an identifiable community.

No

80 independent sectors. Any measures taken from multiple sources within a sector will be aggregated or clustered.

80 independent sectors. There may be multiple observations within a sector (eg from citizen surveys).

80 independent sectors. 40 sectors will be treated.

Using a generic calculation, a study with an ideal power of .8, an alpha of .05, and a sample size of 80, where half the units are treated, we have a minimal detectable effect size of .63 standard deviations. Conditional on an effect being statistically significant and the 5% level, we would need to see a treatment effect of .63 times the standard deviation of the outcome variable under the null in order to be 80% confident that the true effect size is at least that large. If we construct an index which measures combo governance among the sectors, and that the errors from our regressions are normally distributed, a minimum detectable effect would be equivalent to sectors in the 62nd percentile of the distribution moving to the 37th percentile of the distribution.

JDE pre-results review (accepted)