Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We used data from our baseline to explore possible statistical power for analysis of impact with a potential endline survey. We chose two initial indicators: 1) the International Development and Early Learning Assessment (IDELA), our primary measure of child development, and 2) total number of meals in the past three days, a question from our food security module included in the baseline.
We calculated results for two different samples: first, our “research sample,” which includes the 894 households that participated in the baseline data collection exercise. Anticipating that statistical power might be a concern, we also included analysis for the “full sample,” which includes an additional 323 households that were randomly selected to receive the graduation program in order to reach implementation targets for the project but were not included in the baseline survey.
In Table 1 we include the inputs we set and calculated as well as the results of the power calculations, the output being the Minimum Detectible Effect (MDE). As expected, the MDE is slightly smaller for the research sample than the full sample. In addition, other research using IDELA scores suggest that we might expect a correlation of 0.3 or 0.4 between baseline and endline scores, which would increase our statistical power.
To help interpret our MDE, we provide some references to related interventions. For child development, we start by considering other research on outcomes for children in pre-primary education. Meta-analysis of pre-primary education in LMICs finds effect for children’s academic skills (0.15 sd) and social emotional skills and behavior (0.12 sd) (Holla et al., 2021). Improving the quality of early childhood centers can also improve children’s development, a study in India found that children in ECCE centers that received an additional part-time worker scored 0.29, 0.46, and 0.18 standard deviations higher on independent tests in math, language and executive function (Ganimian, Muralidharan & Waters, 2023). Our study does not address the provision or quality of the pre-primary education provided, and instead investigates children’s improved ability to benefit from pre-primary education. A relevant concern might be attendance, and if the graduation program improves children’s attendance in Play Labs. A study on pre-primary education in Chile found that for children’s vocabulary scores, every 1 unit increase in predicted likelihood of absenteeism index (where a 1 unit increase indicates one day out of ten, or a 10% increase) the end-of-intervention vocabulary score decreased by 0.35 SD (Arbour et al., 2016). Our MDE of 0.036 SD we accordingly believe to be very promising, especially considering that including controls for baseline IDELA scores and other factors would mean that our MDE is likely even lower.
Outcome IDELA IDELA Food security Food security
Unit Avg. Correct answers # of meals in last 3 days
Sample Full Research Full Research
Sample size 1218 894 1218 894
Power 0.8 0.8 0.8 0.8
Alpha 0.05 0.05 0.05 0.05
T/C cluster ratio 1 1 1 1
Clusters per arm 20 13.5 20 13.5
Cluster size 30.45 33.1481 30.45 33.1481
ICC 0.0271 0.0271 0.0427 0.0427
Base mean 0.3283 0.3283 1.6982 1.6982
SD 0.1686116 0.1686116 0.892793 0.892793
MDE 0.0363 0.0432 0.2153 0.2577