Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We use existing apartment listing data from the same online platform in a pre-trial in five cities: Houston, TX, San Jose, CA, Atlanta, GA, Philadelphia, PA, and Cincinnati, OH, to identify the sample size requirements for statistical power.
The pre-trial data contains 6,015 listings. The pre-trial yielded a 43.36% response rate to white names and 39.75% to minority, non-white, names (38.99% for African American and 40.52% for LatinX/Hispanic names). Moreover, approximately 10\% of the listings are in downtown areas. We estimate odds ratios of 0.66 (0.4883,0.9053) , 0.68 (0.6185, 0.7681) for downtown and suburban listings respectively. 95% confidence intervals are reported in parenthesis.
We then simulate the effect of increasing the sample size in a conditional logit model with matched inquiries sampling an equal number from downtown and suburban areas. Simulation results indicate that effect sizes of 0.75 for downtown and 0.58 for suburbs can be detected with a sample size of 2,400 properties. Figures 2 and 3 in our supporting materials plot simulation results ( p-values and odds ratio) at different sample sizes.
To detect significant effects across cities our simulations suggest that between 65 and 95 listings per city are needed, which would imply between 3,240 and 5,000 total listings in our sample. Figure 4 in the supporting materials plots the number of significant results at 10% and 5% significance levels by the number of listings in each city. The figure shows that at 65 listings per city we get can detect discrimination in 35 cities at 5% and 40 at 10% cities. Between 90 and 95 listings we are able to detect discrimination in 45 cities. Giving the significance level, some cities are not significant just by chance. The plot suggests that given our sampling this may be at least 5 cities.
References
- Baum-Snow, N. and Hartley, D. (2017). Accounting for Central Neighborhood Change. Federal Reserve Bank of Chicago.
- Couture, V. and Handbury, J. (2017). Urban Revival in America, 2000 to 2010. National Bureau of Economic Research, Working Paper No. w24084.
- Edlund, L., Machado, C., and Sviatschi, M. M. (2015). Bright Minds, Big Rent: Gentrification and the Rising Returns to Skill. National Bureau of Economic Research,Working Paper No. w21729.
- Holian, M. J. and M. E. Kahn (2012). The Impact of Center City Economic and Cultural Vibrancy on Greenhouse Gas Emissions from Transportation.
- Manning, A. and Petrongolo, B. (2017). How Local are Labor Markets? Evidence from a spatial job search model. American Economic Review, 107(10):2877-2907.
- Marinescu, I. and Rathelot, R. (2018). Mismatch Unemployment and the Geography of Job Search.American Economic Journal: Macroeconomics, 10(3):42-70.
- van Vuuren, A. (2018). City Structure and the Location of Young College Graduates. Journal of Urban Economics, 104:1-15.