|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
Assuming conventional 80% power and 95% confidence intervals, preliminary power calculations suggest that the minimum detectable effect size in the first year of the intervention is a 5 percentage-point change (46%) in the likelihood of moving to an opportunity neighborhood when comparing treatment and control groups. Using statistics provided by the Seattle and King County Housing Authorities, we based our calculations on assumptions on the annual number new families with at least one child under age 15 issued a voucher each year (N = 1284), equally sized treatment and control group sizes, and the usual fraction of families issued vouchers who move to an opportunity neighborhood absent any intervention (11%, standard deviation = 0.31). Although we are interested in outcomes conditional on voucher take up, randomization will occur at the level of issued voucher, and we assume that historical 68% lease-up rates (weighted average across SHA and KCHA) will prevail for both treatment and control. We further assume that gamma = 5% of families assigned to the treatment group will opt-out of services.
To calculate the minimum detectable effect size with an expected sample size of 1284 households, we followed equation (4.3) in Orr (1999)
Minimum Detectable Effect = 2k*sigma/sqrt((1-gamma)*N) = 0.0502
where k = 2.8 is a constant determined by desired statistical size (5%) and power (80%); sigma = 0.31 is the standard deviation of the unconditional outcome of interest (the fraction of households issued a voucher that lease up in an opportunity neighborhood); gamma = 0.05 is the fraction of sample that decline program participation; and N = 1284 is the total number of vouchers to eligible families expected to be issued during the study across the two housing authorities.
Although this is a large effect in relative terms, it is small in absolute terms (less than an extra 100 families a year moving to opportunity neighborhoods), and it is smaller than effects seen in the original Moving to Opportunity experiment evaluation and those estimated non-experimentally for the Baltimore Housing Mobility Program. In year two, we anticipate two or three treatment arms with more focused interventions based on qualitative lessons learned from year one, each consisting of smaller sample sizes and resulting in larger minimum detectable effects.
|
After
Assuming conventional 80% power and 95% confidence intervals, preliminary power calculations suggest that the minimum detectable effect size in the first year of the intervention is a 5 percentage-point change (46%) in the likelihood of moving to an opportunity neighborhood when comparing treatment and control groups. Using statistics provided by the Seattle and King County Housing Authorities, we based our calculations on assumptions on the projected number of new families with at least one child under age 15 issued a voucher each year (N = 1284), equally sized treatment and control group sizes, and the usual fraction of families issued vouchers who move to an opportunity neighborhood absent any intervention (11%, standard deviation = 0.31). Although we are interested in outcomes conditional on voucher take up, randomization will occur at the level of issued voucher, and we assume that historical 68% lease-up rates (weighted average across SHA and KCHA) will prevail for both treatment and control. We further assume that gamma = 5% of families assigned to the treatment group will opt-out of services.
To calculate the minimum detectable effect size with an expected sample size of 1284 households, we followed equation (4.3) in Orr (1999)
Minimum Detectable Effect = 2k*sigma/sqrt((1-gamma)*N) = 0.0502
where k = 2.8 is a constant determined by desired statistical size (5%) and power (80%); sigma = 0.31 is the standard deviation of the unconditional outcome of interest (the fraction of households issued a voucher that lease up in an opportunity neighborhood); gamma = 0.05 is the fraction of sample that decline program participation; and N = 1284 is the total number of vouchers to eligible families expected to be issued during the study across the two housing authorities.
Although this is a large effect in relative terms, it is small in absolute terms (less than an extra 100 families a year moving to opportunity neighborhoods), and it is smaller than effects seen in the original Moving to Opportunity experiment evaluation and those estimated non-experimentally for the Baltimore Housing Mobility Program. In phase two, we anticipate two or three treatment arms with more focused interventions based on qualitative lessons learned from phase one, each consisting of smaller sample sizes and resulting in larger minimum detectable effects. For example, to separately examine phase one and phase two with N/2 households in each phase, our minimum detectable effect increases by a factor of sqrt(2) to 0.0710 meaning that we anticipate having sufficient power to detect a 7.1 percentage point treatment effect in phase one or across all treatment arms of phase two.
|