Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The Pre Analysis Plan provides a more comprehensive account of the power calculations we have performed. We list here the Minimum Detectable Effects for T1 vs T2. Consider the following to interpret the MDE calculations:
For all calculations, our alpha was set at the standard 0.05 level, power was set at 0.8. and we considered critical values for a one-sided test.
For each outcome, we show (where possible -and relevant) the baseline mean of the control group and the Minimum Detectable Effect of our intervention.
Some outcomes (Business is open, Business applied for help, Business got help) do not have a measure in the world without COVID, so we calculate MDEs with respect to 3 possible scenarios for the control mean.
For the outcome “jobs lost from January to follow up”, we also make an assumption about the jobs that will be lost in the control group. We know the average number of jobs lost in the control group from January to the moment of the baseline. The approximation we use is 1.5 times this number.
In general, we expect the take up of the treatment to be 60% in the treatment group (0.6 compliance) and 0% in the control group (0% spillover). We adjust our MDEs taking this into account (multiplying by 1/0.6 = 1.66)
MDE calculations T1 vs T2 in 5 countries, primary outcomes:
(1) Business is open, MDE relative to 3 possible values for control mean
if control mean = 0.98, MDE = 0.0111
if control mean = 0.95, MDE = 0.0179
if control mean = 0.90, MDE = 0.0251
(2) Number of jobs lost from baseline to follow up
control mean (assumed) = 3.5, MDE = -0.0864
(3) Proportion of owners aware of programs
control mean = 0.21, MDE = 0.0357
(4) Proportion that applied for help, MDE relative to 2 possible values for control mean
if control mean = 0.1, MDE = 0.0267
if control mean = 0.15, MDE = 0.0315
(5) Proportion that got help, MDE relative to 2 possible values for control mean
if control mean = 0.03, MDE = 0.0156
if control mean = 0.06, MDE = 0.0213