Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We reviewed relevant literature to determine the expected effect size for our study. A meta-analysis by McCarthy and Morina (2020) reviewed the impact of social comparisons on mental health, specifically in relation to symptoms of depression and anxiety. This study found that the average effect size to be 0.44. However, effect sizes varied depending on the specific type of comparison and the targeted mental health outcome.
Another systematic review on social interventions focused on the effectiveness of social incentives by Nguyen-Van et al (2021), including social comparison, for promoting pro-environmental behaviors. The average effect size for internal social influences that motivate pro-environmental behaviors was 0.8454 while that of external influence was 0.468.
Given the evidence provided by these two systematic reviews, we chose to use a conservative lower bound effect size of 0.44 in our power calculations. We calculate the estimated sample sizes below to detect a significant difference using the following values:
1. effect_size = 0.44
2. alpha = 0.05 # significance level
3. power = 0.80 # typical power value
Based on our conservatively assumed values, we estimate a minimum sample size of 82 observations is required to detect an effect size of 0.44 in our randomized controlled trial. These power calculations suggest that we have sufficient observations to observe an effect if our intervention is successful.
Our design allows for significant attrition over time without compromising the integrity of our study. From our experience, we might expect an attrition rate of between 10 - 30%. However, given this intervention is under a directive from the SC executive, we may observe less attrition over time. In any case, we have overpowered our design to mitigate such risks.