Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
Component 1: Comparison of school feeding models
We implemented power calculations for two main outcomes (among others), students' dietary diversity (using data from the Jordan National Nutrition Survey 2019) and for Math test scores (using Jordan PISA data for 2018). Power was set to 0.8 and the significance level to 0.05. The number of students included per cluster is equal to 10 and was chosen as the maximum number that would fit the budget envelope. Note that our power calculations are conservative since they do not consider efficiency improvements obtained by using control variables for which we currently do not have data for.
With 318 treatment and 138 control school clusters, and 10 students per cluster interviewed, the minimum detectable effect (MDE) for child dietary diversity is 0.31 additional food groups, from a mean of 4.79 food groups out of 10 measured. The community-based kitchen model provides three food groups (dairy, fruit, and vegetable) that the date bar/high protein biscuit model does not provide. Since only 65 % consume dairy, 32 % consume fruit, and 56 % consume vegetables among the children in the lowest wealth quintile of the settled Jordanian population, increasing consumption of these three food groups to 90 % on average (i.e., accounting for absences, or not all students receiving meals which could preclude 100 %) would yield an expected effect of 1.17 food groups. The expected effect is thus larger than the MDE.
On the other hand, we may not be fully powered to detect impacts on the math score as measured by the PISA test, with a MDE of 0.16 SD, at least without efficiency improvements. Most other evaluations of school-feeding programs that compare the introduction of school feeding versus no school feeding – rather than different modalities – have found smaller impacts on math scores (e.g., 0.09 SD over 5 years in Chakraborty and Jayaraman (2019), 0.147 SD over 2 years in Aurino et al. (2018)).
Component 2: Employment in community-based kitchens
Randomisation for the comparison of workers will be done at the worker level based on the list of eligible workers provided by the implementing partners. The randomisation will be stratified by kitchen and the experience of workers in commercial kitchens. Each hired and non-hired worker will be surveyed for baseline and endline surveys, and for high-frequency surveys in between.
With this given sample size for the randomised trial, we implemented power calculations for two main outcomes (among others), household income (using mVAM data from the WFP Jordan Country Office for poverty pockets) and for food security measured by the reduced coping strategy index (rCSI, using WFP Jordan Country Office community assessment data). The power was set to 0.8 and the significance level to 0.05. Note that these power calculations are conservative since they do not consider efficiency improvements obtained by using control variables for which we currently do not have data (e.g., stratification variables, baseline outcomes).
With 315 individuals in the treatment and 215 individuals in the control group, we are powered to detect effects of 0.25 SD for a simple comparison of means at endline. For household income, this translates into an MDE of 270 USD and of (minus) 4.23 for the rCSI. The MDE of 270 USD compares to the expected monthly income from employment in the community-based kitchens of about 250 USD. The expected effect may therefore be smaller than this MDE.
Therefore, in addition to baseline and endline surveys, we plan four shorter high-frequency surveys to be administered every two months, and whose repeated measurements of the same main outcomes will improve power. We do not have data on autocorrelation of these outcomes in Jordan, and therefore assume a medium autocorrelation of 0.3 for the six measurements. All outcomes will be included in the baseline and endline, and (to keep high-frequency surveys short) all outcomes will be included in only two of the four high-frequency rounds, for a total of four measurements per outcome. With these repeated measurements, the MDE reduces to about 0.17 SD, which translates into an increase of 180 USD for household income, and 2.82 for the rCSI.