Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
List experiments: List experiments are less efficient than direct survey questions since the indirect question technique introduces noise. Blair and Imani (2012) suggest power is not an issue in samples with around 1500 respondents, although that will also depend on the number and variance of the control statements. This may mean that the Malawian RDS sample with a target size of 1000 is slightly underpowered. Chuang et al. (2021) find that choosing less innocuous control statements that are at least somewhat sensitive improves the precision of the estimates in their study. We follow a similar approach, with control statements that are also expected to minimize floor or ceiling effects.
Blair, Graeme and Kosuke Imai. 2012. Statistical Analysis of List Experiments, Political Analysis, 20: 47-77.
Chuang, Erica, Pascaline Dupas, Elise Huillery and Juliette Seban. 2021. Sex, Lies, and Measurement: Consistency Tests for Indirect Response Survey Methods, Journal of Development Economics, 148, 102582.
Conjoint analysis: Power calculations for the conjoint analysis are based on two recent working papers: Schuessler and Freitag (2020) and Stefanelli and Lukac (2020), which both yield similar results.
Using the Schuessler and Freitag approach, we expect the following given a sample of 1,000 respondents in Malawi and 2,000 respondents in Zambia, with 4 rounds of 2 hypothetical jobs and up to 4 values per attribute (5 values in Zambia for location):
- In Malawi sample: 89% power to detect an effect of 0.05 at alpha = 0.05 (and 81% power for an effect of 0.04 at alpha = 0.1)
- In Zambia sample: 99% power to detect an effect of 0.05 at alpha = 0.05 , 95% power for an effect of 0.04 at alpha = 0.05 (and 85% power for an effect of 0.03 at alpha = 0.1)
References:
Schuessler, Julian and Markus Freitag. 2020. Power Analysis for Conjoint Experiments. Working paper.
Stefanelli, Alberto and Martin Lukac. 2020. Subjects, Trials, and Levels: Statistical Power in Conjoint Experiments. Working paper.