Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
To determine the appropriate sample size for our study, we conducted an a priori power analysis using G*Power 3.1 (Faul et al., 2009). Given our study design, we aimed to detect a medium effect size with a power of 0.80 and an alpha level of 0.05. We plan to compare mean differences in worry (0-10 scale) across four treatment conditions, while controlling for covariates (both continuous and categorical, e.g., age, income, gender, knowledge). Given this, we conducted a power analysis for an Analysis of Covariance (ANCOVA) and a Linear Multiple Regression.
Power Analysis for ANCOVA
We used the “ANCOVA: Fixed effects, main effects and interactions” option in G*Power to estimate the required sample size. The parameters were set as follows:
• Effect size (f) = 0.25 (medium, based on Cohen, 1988, 1992) / 0.10 (small effect)
• Number of groups = 4 (treatment conditions)
• Number of covariates = 6 (e.g., age, income, gender)
• Alpha level (α) = 0.05
• Power (1-β) = 0.80
The analysis estimated a required sample size of 179 participants (if medium effect) (or 1,095 if small effect) to achieve sufficient power.
Power Analysis for Linear Multiple Regression
Given that we also planned to conduct an Ordinary Least Squares (OLS) regression with worry (0-10) as the dependent variable, treatment dummies as independent variables, and the same covariates, we conducted an additional “Linear multiple regression: Fixed model, R² deviation from zero” analysis in G*Power:
• Effect size (f²) = 0.15 (medium) / 0.02 (small effect)
• Number of predictors = 9 (3 treatment dummies + 6 covariates)
• Alpha level (α) = 0.05
• Power (1-β) = 0.80
This analysis suggested a lower sample size of 114 participants (if medium effect) (or 791 if small effect).
Final Sample Size Decision
This power analysis ensures adequate sensitivity to detect meaningful treatment effects while controlling for key covariates. Given that ANCOVA generally requires a larger sample than linear multiple regression, we adopted the more conservative estimate. The targeted sample size of 2,400 respondents per jurisdiction (United States and Canada) ensures sufficient statistical power for this experimental study.
References
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, New Jersey: Lawrence Erlbaum Associates.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160.