Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
Power assumes two-sided tests, α = 0.05, power = 0.80, individual-level randomization (no clustering). Main treatment effects pool two arms vs. two arms, giving approximately 1,280 vs. 1,280 at the planned analytic sample (640/arm assuming a retation rate of 80%).
Survival (primary 1). Immediately after the video, the survival treatment moved the raw gap by about 2 percentage points. Comparing arm means alone, we could detect persistence only if roughly 87% of that effect survived (MDE approximately 1.7 pp) — i.e., only a barely-attenuated effect. Adjusting for the pre-treatment population belief, a covariate that strongly predicts the outcome, removes enough noise to bring the threshold down to about 1.4 pp (planning scenario), so we remain powered even if the effect has faded to roughly two-thirds to three-quarters of its original size.
A more optimistic covariate assumption or the 800/arm ceiling lowers the MDE slightly further; the control-arm SD as a conservative input raises it to about 1.5 pp — none of which changes the conclusion. The same logic applies to the population-survival gap, where the pre-treatment baseline is of the same construct and if anything stronger. The MDE refers to the pooled main effect; a single-arm contrast (e.g., survival vs. control alone) is powered only for the full 2 pp effect.
Financial-literacy knowledge (primary 2). The outcome is the share of correct answers on the four comprehension vignettes. Because these items are new to the follow-up, there is no same-construct Survey 1 baseline to adjust for, so — unlike the survival outcome — no covariate reduction is applied (a deliberate, stated asymmetry between the two primary outcomes). Using a provisional standard deviation of 0.30, the design detects a main effect of about 3.3 percentage points in the share correct. This value is provisional; the items were written to be harder than the post-treatment attention checks (which approximately 93% of treated respondents passed) to avoid a ceiling effect that would erode power, and the standard deviation will be updated from the pilot.
Most-biased subgroup (best-powered persistence test). The immediate effect is highly concentrated: among the third of respondents with the most downward-biased pre-treatment population belief (bottom tertile of that variable), the Survey 1 effect is 7.56 pp — nearly four times the full-sample average — with a within-arm SD of approximately 17. Restricting to this tertile cuts sample size by roughly three (213 per arm at 80% retention, 426 vs. 426 in the main-effect contrast), which raises the MDE to about 3.3 pp. Because the effect grows far more than the MDE, this is the study’s most detectable persistence test: it remains significant even if only about 44% of the immediate effect survives, versus about 70% for the full-sample primary. This makes it the analysis most tolerant to the two-month attenuation we are most concerned about, and we pre-specify it accordingly. Two points are noted: the subgroup is defined on a pre-treatment variable, so randomization holds within it and the contrast is an unbiased causal effect; and re-contact must be tracked within the tertile, since the most mis-calibrated respondents may also attrit differentially.
Interaction effects in the 2 × 2 design require far larger samples and are exploratory.