Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We reviewed relevant literature to determine the expected effect size for our study. A meta-analysis by McCarthy and Morina (2020) reviewed the impact of social comparisons on mental health, specifically in relation to symptoms of depression and anxiety. This study found that the average effect size to be 0.44. However, effect sizes varied depending on the specific type of comparison and the targeted mental health outcome.
Another systematic review on social interventions focused on the effectiveness of social incentives by Nguyen-Van et al (2021), including social comparison, for promoting pro-environmental behaviors. The average effect size for internal social influences that motivate pro-environmental behaviors was 0.8454 while that of external influence was 0.468.
Given the evidence provided by these two systematic reviews, we chose to use a conservative lower bound effect size of 0.44 in our power calculations. We calculate the estimated sample sizes below to detect a significant difference using the following values:
1. effect_size = 0.44
2. alpha = 0.05 # significance level
3. power = 0.80 # typical power value
Based on our conservatively assumed values, we estimate a minimum sample size of 82 observations is required to detect an effect size of 0.44 in our randomized controlled trial. These power calculations suggest that we have sufficient observations to observe an effect if our intervention is successful.
Our design allows for significant attrition over time without compromising the integrity of our study. From our experience, we might expect an attrition rate of between 10 - 30%. However, given this intervention is under a directive from the SC executive, we may observe less attrition over time. In any case, we have overpowered our design to mitigate such risks.
Exclusion Criteria
To properly randomize the sample, a set of exclusion criteria was developed to ensure that the randomization frame. They are divided into layers/filters and are as stated as follows:
For the first layer, the court must exist in all three data sources. A court needs to have the complete set of administrative datasets across different court management indicators, meaning it appears in the caseflow file, the case aging file, and the accused profile file. a court entirely absent from any one source is excluded before the month counting even starts.
As for the second filter, each of the months in 2025 is checked against all eight identified balance variables such as pending cases, total caseload, filed, decided, clearance rate, disposition rate, total case age, and accused in detention. With this, a month is considered complete only if all eight have values. If even one is missing, the whole month is stamped incomplete. Importantly, the randomization strategy counts a month as incomplete in two instances that may look different in the raw files but are operationally the same: (1) the row exists but a value is blank, or (2) the court-month row is absent entirely (the court never submitted that month's report). Therefore, the skeleton construction by building every court by 12 months first, then merging the data on is what guarantees absent rows get counted.
Lastly, the next filter determines a court is eligible if it has at most 2 incomplete months out of 12. Essentially courts with at least 10 non-missing months or more are included in the randomization frame. This means that three or more incomplete data across 12 months are excluded and slated for remedial record-keeping support rather than the study. Hence, the excluded set of courts are the ones that failed to provide all eight variables in at least ten of the 12 months of 2025.