|
Field
Trial End Date
|
Before
August 02, 2019
|
After
August 31, 2019
|
|
Field
Last Published
|
Before
May 16, 2019 11:18 AM
|
After
August 02, 2019 10:05 AM
|
|
Field
Experimental Design (Public)
|
Before
Our experiment uses a paired (tripled) experimental design. Responses from property managers will be captured in email (gmail address associated with each name), phone messages (individual phone numbers associated with each name), and text messages. Responses that indicate housing availability will be coded, as well as the time stamp, message length, and sentiment of responses. The full set of rental listings will be compiled for the neighborhood (zip codes within 1 mile) of a toxic plant that reports toxic emissions to the Environmental Protection Agency's Toxic Release Inventory (TRI).
|
After
Our experiment uses a paired (tripled) experimental design. Responses from property managers will be captured in email (gmail address associated with each name), phone messages (individual phone numbers associated with each name), and text messages. Responses that indicate housing availability will be coded, as well as the time stamp, message length, and sentiment of responses. The full set of rental listings will be compiled for the neighborhood (zip codes within 1 mile) of a toxic plant that reports toxic emissions to the Environmental Protection Agency's Toxic Release Inventory (TRI).
|
|
Field
Randomization Method
|
Before
The paired experimental design used in this study requires that inquiries for each listing are made from each of the three racial groups that we study. Immediately following compilation of the relevant listings in a given market, an inquiry will be sent for each using a randomly assigned name drawn from each of the 3 racial groups. Each rental apartment will therefore receive 3 separate inquiries in the course of an experimental trial. Listings are divided into 3 blocks to ensure that inquiries for the same listing will never be sent from two race groups on the same day. Inquiries will be sent at an interval of 2-10 minutes over the course of the 3 day period. The names and within each racial group are randomly assigned to a listing with equal probability and the sequence of inquiries is also randomly assigned.
|
After
The paired experimental design used in this study requires that inquiries for each listing are made from each of the three racial groups that we study. Immediately following compilation of the relevant listings in a given market, an inquiry will be sent for each using a randomly assigned name drawn from each of the 3 racial groups. Each rental apartment will, therefore, receive 3 separate inquiries in the course of an experimental trial. Listings are divided into 3 blocks to ensure that inquiries for the same listing will never be sent from two race groups on the same day. Inquiries will be sent at an interval of 2-10 minutes over the course of the 3 day period. The names and within each racial group are randomly assigned to a listing with equal probability and the sequence of inquiries is also randomly assigned.
|
|
Field
Planned Number of Clusters
|
Before
110 zip codes within one mile of a toxic plant
|
After
16-18 zip codes out of 110 zip codes within one mile of a toxic plant and in the 80% percentile of total toxic releases
|
|
Field
Planned Number of Observations
|
Before
3500-4000 property listings. Expected sample: 3700 listings (property managers)
|
After
2400-2700 property listings. Expected sample: 2600 listings (property managers)
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
3500-4000 property listings. Expected sample: 3700 listings (property managers). Sample of listings within 1 mile varies by zip code, but at least 30% of listings fall within 1 mile of a facility. Based on matched design, inquiries will be sent out in equal numbers from racial groups (e.g. 3700 African American, 3700 LatinX/Hispanic, 3700 White)
|
After
2400-2700 property listings. Expected sample: 2600 listings (property managers).
Sample of listings within 1 mile varies by zip code, but at least 30% of listings fall within 1 mile of a facility. Based on a matched design, inquiries will be sent out in equal numbers from racial groups (e.g. 2600 African American, 2600 LatinX/Hispanic, 2600 White)
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
We use existing apartment listing data from the same online platform in a pre-trial in Houston, TX to identify the sample size requirements for statistical power. The Houston pre-trial data contains 1563 listings. The pre-trial yielded a 17.9% response rate to white names and 16.7% to names associated with African American or LatinX/Hispanic names (non white names). It also yielded a relatively balanced sample with respect to proximity to TRI facilities: 45% of the rental properties where in the neighborhood of a toxic plant (within 1 mile).
To compute the sample sizes and the minimum detectable effects of the interaction of race and proximity to toxic plant we assume 90% test power and .05 significance level. In the small of data from the Houston pre-trial, we estimate an odds ratio of 1.27 (0.42) for the interaction. Standard errors are clustered at the Houston zip code level. We then simulate the effect of increasing sample size in a conditional logit model with paired inquiries. Simulation results suggest an effect size 1.54 that can be detected with 3017 properties. Figures 3 and 4 in our supporting materials shows simulation results for different sample sizes, for odds ration and p-values. Alternatively, if we use Demidenko (2007, 2008) approach to calculate the number of listings it yields that we need about 2,433 properties to obtain for that detectable odds ratio.
Phillips (2016) provides evidence of within-trial impacts when multiple inquiries sent in matched correspondence designs in competitive labor markets. In a sample restricted to responses to the first inquiry and based on a simple logit model, our simulations show that we should be able to detect an effect with an odds ratio of 1.43 at 3676 properties. Figures 5 and 6 shows the results of these simulations.
These power calculations are limited by available data from the Houston pre-trial and the incidence of discriminatory behavior that may be particular to the Houston housing market.
References
Demidenko E. (2007). "Sample size determination for logistic regression revisited." Statistics in Medicine 26:3385-3397
Demidenko E. (2008) "Sample size and optimal design for logistic regression with binary interaction." Statistics in Medicine, 27:36-46
Phillips, David C. "Do comparisons of fictional applicants measure discrimination when search externalities are present? evidence from existing experiments." The Economic Journal (2016).
|
After
We use existing apartment listing data from the same online platform in a pre-trial in Houston, TX to identify the sample size requirements for statistical power. The Houston pre-trial data contains 1563 listings. The pre-trial yielded a 17.9% response rate to white names and 16.7% to names associated with African American or LatinX/Hispanic names (non white names). It also yielded a relatively balanced sample with respect to within Zipcode quartiles of toxic concentrations: 25% for properties in the first quartile of toxic concentration, 21% in the second, 21% in the third and 33\% in the quartile with the highest toxic concentration. With respect to proximity to TRI facilities, 45% of the rental properties are located within 1 mile of a toxic plant.
To compute the sample sizes and the minimum detectable effects of the interaction of race and proximity to a toxic plant, we assume a test power of 90% and a .05 significance level. In the sample of data from the Houston pre-trial, we estimate odds ratios of 0.65 (0.48), 0.76 (0.30), 0.70 (0.32) and 0.79 (0.37) for each of the quartiles and 1.27 (0.42) for the interaction with plant proximity. Standard errors are clustered at the Houston zip code level.
We then simulate the effect of increasing the sample size in a conditional logit model with matched inquiries. Simulation results indicate that effect sizes of 0.41, 0.35, 0.65 and 1.12 can be detected with a sample size of 2,400 properties. Simulations for plant proximity suggest an effect size of 1.54 that can be detected with 3017 properties. Figures 5-7 in our supporting materials plot simulation results (odds ratios and p-values) at different sample sizes. Using an alternate approach from Demidenko (2007, 2008), our simulations indicate that we need approximately 2,680 properties to obtain for detectable odds ratios.
Phillips (2016) provides evidence of within-trial impacts when multiple inquiries sent in matched correspondence designs in competitive labor markets. In a sample restricted to responses to the first inquiry and based on a simple logit model, our simulations show that we should be able to detect an effect with odds ratios of 0.52, 1.41, 1.70, 0.98 at 2,337 properties and an odds ratio for the interaction with proximity to toxic plants of 1.43 at 3676 properties. Figures 8-10 plot the results of these simulations. These power calculations are limited by available data from the Houston pre-trial and the incidence of discriminatory behavior that may be particular to the Houston housing market.
References
Demidenko E. (2007). "Sample size determination for logistic regression revisited." Statistics in Medicine 26:3385-3397
Demidenko E. (2008) "Sample size and optimal design for logistic regression with binary interaction." Statistics in Medicine, 27:36-46
Phillips, David C. "Do comparisons of fictional applicants measure discrimination when search externalities are present? evidence from existing experiments." The Economic Journal (2016).
|