A Cluster Randomized Controlled Trial of Blight Remediation and Community Outreach to Improve Residents’ Perceptions of Safety and Wellbeing in Buffalo, NY

The intervention targets approximately 200 properties within a 2-3 block area, and occur each Wednesday between May and October for 4.5 hours (8:30am to 1pm). Approximately 32 interventions occur per year and each involves 50-70 community partners and city employees. Activities typically include: boarding up abandoned structures, removing graffiti and debris, code enforcement (mostly related to exteriors of homes), community policing (police peacemakers making introductions with residents), trimming trees, repairing signs, and going door-to-door offering governmental, employment and health & human services information [1]. Issues that cannot be addressed at the time of the intervention (less than 10% of issues) are typically addressed within 24 hours.

Implementation of the intervention is rolled out with the same or similar city and community partners each week. However, the type of work that is done depends on the needs in a given area (e.g. graffiti removal teams only carry out work in blocks with graffiti). If there are known problem properties in a block, code enforcement officers may plan to enter the home and conduct an inspection. However, most of code enforcement work is related to the exterior of homes, unless inspectors are invited in.

2021-05-19

2021-11-26

1) Change in residents’ perception of wellbeing (a composite score measuring stress and perception of visible disorder, neighborhood cohesion, and perception city cares). Changes in specific metrics on the survey will also be examined such as perception of safety, stress related to block conditions, perception of physical disorder, and perception that city cares about residents.

Wellbeing will be aggregated into a composite wellbeing score to account for multiple hypothesis testing. The score will be comprised 21 questions to elicit perception of safety, stress related to block conditions, perception of physical disorder, and perception of engagement from the city. These questions will be measured on a 4-point scale. The index will be generated by calculating an average for each of the following four categories below and taking the average across all four:
a. Neighborhood satisfaction and cohesion: nbh_satis_coh_1 — nbh_satis_coh_8
b. Safety and fear: safe_fear_1 — safe_fear_5
c. Social disorder: disord_soc_1 — disord_soc_4
d. Physical disorder: disord_phy_1 — disord_phy_4
Researchers may conduct additional analysis on the above four subgroups separately to supplement the analysis. The Wellbeing survey also collects self-reported data on 311 calls and 911 calls, which may be used to supplement the analysis. Measured at property-level 1 week before and 5 weeks after the intervention in treatment areas, and over 6-week interval in control 1 neighborhoods through the Wellbeing survey.

1) Crime - Change in all categories of crime will be measured through administrative data collected from the Buffalo Police Department.
2) Reduction in Visible Blight- measured by observation checklist 1 week prior to intervention and 1 month after intervention (5 weeks apart in control clusters) and 6 months after the last intervention.
3) Social perception of neighborhood activities/characteristics - an indication of changes in social perception of neighborhood activities/characteristics

1) Crime: Changes in specific crime types will also be examined including theft/larceny, assault, burglary/robbery, drug-related, nuisance, and gun-related crimes. Crime data that is not attributable to a specific property will be assigned to the nearest property. Crime will be measured at residential and non-residential properties and will include crime data from 6 months before the intervention and 6 months after the intervention in control groups 1 and 2 through administrative data.

2) Reduction in Visible Blight: Changes in observed physical disorder and physical/social order will be collapsed into a single index to account for multiple hypothesis testing. Visible Blight will be measured at the property-level 1 week before and 5 weeks after Clean Sweep in treatment areas, and over 6-week interval in control group 1 through the observation checklist. Researchers plan to conduct additional data collection 6 months after the intervention to observe longer-term changes in social and physical disorder. The index will be comprised of the number of check boxes on an observation checklist weighted by the volume estimate over study period:
a. Physical disorder: Trash on street (volume estimate); Piles of various objects, such as tires, furniture, electronics, plastic (volume estimate); Graffiti (count); Vacant properties boarded up (count); Vacant properties not boarded up (count); Overgrown yards (height estimate); Trees in need of trimming
b. Physical/Social Order: New construction (count); Bulletins/ posters about neighborhood-based community events (count)

3) Social perception of neighborhood activities/characteristics - Social perception of neighborhood conditions will be aggregated into a composite score to include administrative data on 311 calls. City-reported administrative data on 311 calls will be comprised of the count of 311 calls over the study period. Researchers may weight the count of 311 calls by the type of incident reported. Social perception of neighborhood conditions will be measured at residential and non-residential properties and will include data from 6 months before the intervention and 6 months after the intervention in control groups 1 and 2 through administrative data.

A data-driven selection process will be used to generate the sampling frame of eligible clusters. All census block groups (CBGs) in the city will be assigned a “priority score”. The scores will be based on a combination of factors including: building permits, police incidents, property sales, current use (commercial, vacant, residential, recreation, natural areas, community services, industrial, other), low-income population (percent at the census block level), unemployment rate (percent at the census block level), and housing cost burden (percent at the census block level). Additionally, each property in the city will be assigned a priority score based on a combination of factors including: 311 calls, police incidents, building permits, code violation, condition of property, fix and flip, property sales, and property tax delinquency status. These scores will be used to determine where clusters should be located within priority CBGs. CBGs with a score of over 55 and properties with an eligibility score of over 40 will be eligible to receive the intervention.

Final generation of the sampling frame will take place within BuildingBlocks, wherein the researchers will eliminate any non-qualifying CBGs, which include a large proportion of properties that are not eligible (i.e. cemeteries, parks, large commercial districts, etc.). Within the priority areas, primarily contained within one CBG, clusters of 200 properties will be selected as eligible. Most clusters will be contained to one council district and one CBG, however based on neighborhood layout, there will be a few exceptions. Where this is the case, in the analysis we will use a weighted average of CBG characteristics for each cluster, proportional to the number of properties from each CBG. The City of Buffalo will visit eligible locations to assess suitability to provide final certification for inclusion in the sampling frame.

Lists of eligible blocks for the intervention will be generated in early March 2021. A data-driven selection process will be used to generate the sampling frame of eligible clusters. All census block groups (CBGs) in the city will be assigned a “priority score” for a Clean Sweep. The scores will be based on a combination of factors including: building permits, police incidents, property sales, current use (commercial, vacant, residential, recreation, natural areas, community services, industrial, other), low-income population (percent at the census block level), unemployment rate (percent at the census block level), and housing cost burden (percent at the census block level). Additionally, each property in the city will be assigned a priority score based on a combination of factors including: 311 calls, police incidents, building permits, code violation, condition of property, fix and flip, property sales, and property tax delinquency status. These scores will be used to determine where Clean Sweep clusters should be located within priority CBGs. CBGs with a score of over 60 and properties with an eligibility score of over 40 will be eligible to receive a clean sweep.

Final generation of the sampling frame will take place within BuildingBlocks, wherein the researchers will eliminate any non-qualifying CBGs, which include a large proportion of properties that are not eligible to receive a clean sweep (i.e. cemeteries, parks, large commercial districts, etc.). Within the priority areas, primarily contained within one CBG, clusters of 200 properties will be selected as eligible for a Clean Sweep in 2021. Most clusters will be contained to one council district and one CBG, however based on neighborhood layout, there will be a few exceptions. Where this is the case, in the analysis we will use a weighted average of CBG characteristics for each cluster, proportional to the number of properties from each CBG. The City of Buffalo will visit eligible locations to assess community assets and fit for a Clean Sweep to provide final certification for inclusion in the sampling frame.

Eligible clusters will be randomized into treatment and control groups in April 2021. Given the small sample size of clusters, we will couple stratification on a geographic area with a rerandomization procedure on key baseline observables. There will be 8 strata, one for each council district included in the sampling frame: Ellicott, Fillmore, Lovejoy, Masten, Niagara, North, University and South. No clusters were eligible to receive the intervention in the Delaware council district. Given the large number of strata and low sample size, misfits will be randomly allocated independently across all strata. This method weights treatments as specified by the unequal treatment size fractions, to prioritize balance across the sample. [2] [3]

Researchers will conduct a rerandomization procedure with the above strata to achieve balance on the average police incidents and average housing cost burden (the percent of household income spent on housing costs) in the cluster. The sample will be rerandomized 1000 times, then, for each draw, police incidents and housing cost burden will be regressed against treatment. The draw with the minimum maximum t-statistic will be used. [4]

The control group will then be randomly assigned into two groups: one whose residents will be enrolled in the Wellbeing survey and the other whose residents will not be enrolled in the Wellbeing survey. This assignment follows the same rerandomization procedure with stratification as specified above.

[2] A. Carril, “Dealing with Misfits in Random Treatment Assignment,” Stata J., vol. 17, no. 3, pp. 652–667, Sep. 2017, doi: 10.1177/1536867X1701700307.
[3] “Tools of the trade: Doing Stratified Randomization with Uneven Numbers in some Strata.” https://blogs.worldbank.org/impactevaluations/tools-of-the-trade-doing-stratified-randomization-with-uneven-numbers-in-some-strata (accessed Apr. 21, 2021).
[4] M. Bruhn and D. McKenzie, “IN PURSUIT OF BALANCE: RANDOMIZATION IN PRACTICE IN DEVELOPMENT FIELD EXPERIMENTS#,” p. 41.

Neighborhoods of approximately 200 properties

Yes

96 neighborhoods

19,200 properties in 96 clusters. Given the anticipated survey response rate, there is an expected 2,400 responses to the Wellbeing survey.

The 96 clusters will be randomly assigned such that 32 clusters will be randomly assigned to treatment and 64 will be randomly assigned to control group. Among the 64 control clusters, 32 will be randomly assigned for enrollment in the Wellbeing Survey.