Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We conduct power calculations using parameters estimated from the 2011/12 Uganda National Panel Survey (UNPS) data. The ACDP study sample will be a more homogeneous population than the entire Ugandan population, from which the UNPS sample is drawn, as it is restricted by: (1) region and crop, including only maize farmers in eastern Uganda, bean farmers in western Uganda, and rice farmers in northern Uganda; (2) farmer characteristics, determined by the specific eligibility rules established by ACDP. To approximate the more homogeneous ACDP target population using the UNPS data, we estimate parameters using data from a restricted sample of households farming between 3 and 5 acres, a criterion for ACDP eligibility. Even the restricted UNPS sample is more heterogeneous than what the ACDP sample will be, and we are therefore confident that our power calculations are conservative.
Our power calculations take maize yields as the outcome variable. While outcome variables other than yields are of interest, yields typically exhibit higher coefficients of variation than do outcome measures of input use and other farming practices to be studied in the evaluation. We are thus confident that designing a sample with adequate power for maize yields will give us sufficient power to look at other key indicators and crops as well. We report calculations for two cases in which we assume mean yields of 345 kg/ac, the mean maize yields for farmers in eastern Uganda cultivating 3-5 acres of land in the second season of the 2011/12. The remaining parameters for the two cases are:
(1) Expected parameter case - The standard deviation of yields is assumed to be 300 kg/ac, which is based on the coefficient of variation for bean yields of farmers in western Uganda, and the parish-level intra-cluster correlation is assumed to be 12%, which is just below the village-level intra-cluster correlation for bean yields of farmers in western Uganda (14%);
(2) Conservative parameter case - The standard deviation of yields is assumed to be 388 kg/ac, which is based on the coefficient of variation for maize yields of farmers in eastern Uganda, and the parish-level intra-cluster correlation is assumed to be 18%, which was the village-level intra-cluster correlation for maize yields of farmers in eastern Uganda.
Based on our power analysis (summarized below), we propose to select our study sample from 18 sub-counties across the three study districts. In each sub-county, we will randomly select 6 parishes (2 with high initial subsidy, 2 with low initial subsidy, and 2 with no subsidy), yielding 108 clusters for our sample. We will then randomly select 20 households per parish cluster from among ACDP-eligible households, yielding a total sample size of 2160 households for our main sample of directly treated farmers. These figures are feasible as there are a total of 35-45 sub-counties across the three districts, 5-10 parishes per sub-county, 5-15 villages per parish, and 30-130 households per village.
Our minimum detectable effect calculations assume a net uptake rate of 75%. At the expected voucher uptake rate of 75%, the proposed design is powered to detect effects of less than half that magnitude (22%); even under our case of conservative assumptions, the study design could detect a minimum effect of 32%. As long as net uptake of the e-voucher is no lower than 50%, the proposed study design will have adequate power to detect program impacts if ACDP achieves its 50% yield growth goal. While this yield growth goal is high, it appears achievable in Uganda based on government estimates as well as estimates from experimental trials and on-farm trials of farm inputs in Uganda. We are thus confident that we have adequate power to pick up expected yield changes, as well as changes in the other, less variable, outcome indicators of interest. Moreover, we expect the actual power of the study to be higher than what is estimated here, because we will collect multiple (seasonal) rounds of follow-up data and will ensure high levels of data quality through carefully programmed electronic survey instruments.