Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Since we have already run one wave we elected to use the average effect size derived from Wave 1, but to adjust it as follows. First, we take the average of the effect sizes of different treatments that we calculated in the first wave of the study. For the matrix task, the average effect size was 0.72. Assuming that the variance of the population in the Wave 2 is the same as the Wave 1 of the study, we expect to observe an effect size of 0.72 in the modified version of the matrix puzzle. Technically this means we need only 32 subjects per treatment. However, last time data cleaning necessitated removing 20% of the data. Second, we wish to maintain comparability with wave 1 which featured two games, one of which featured a much lower average effects size (0.11). Therefore we plan to have 100 subjects per treatment which easily satisfies our initial average effect size calculation, then, after gaining a better idea of the variance of the underlying Wave 2 population (which might differ from Wave 1), we will potentially follow up with a larger sample size (rising to 200) if needed.