Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
The power analysis, based on Schuessler and Freitag (2020) and using their R package (Freitag & Schuessler, 2020), indicates that to detect Average Marginal Component Effects (AMCEs) of 0.05 in a conjoint experiment with two community profiles at a 5 percent significance level, an effective sample size of 3150 observations is required to achieve 80% power. This ensures that the probability of Type S errors (incorrect sign) is 0% and the exaggeration ratio (Type M error) is approximately 1.13.
Note that the median of published AMCEs is reported as 0.05 by Schuessler and Freitag (2020).
The effective sample size refers to the total number of observations in the dataset. To calculate the actual number of respondents, this effective sample size is divided by the number of profiles each respondent will evaluate. For a conjoint with an effective sample size of 3150, where each respondent chooses between 2 profiles across 5 rounds, the number of respondents needed is 3150 / (2*5) = 315 respondents.
Therefore, with an effective sample size of 3150, I need 315 respondents, as each will view five pairs of community options. To achieve a balanced sample of refugees and host respondents, I would thus have to recruit approximately 630 participants in total.
However, I aim for a total sample of 700 respondents to account for potential non-response, incomplete surveys, and attrition.
Reference:
Schuessler, J., & Freitag, M. (2020). Power analysis for conjoint experiments. https://osf.io/preprints/socarxiv/9yuhp/