Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We conduct power calculations using parameters estimated from the 2011/12 Uganda National Panel Survey (UNPS) data. The ACDP study sample will be a more homogeneous population than the entire Ugandan population, from which the UNPS sample is drawn, as it is restricted by: (1) region and crop, including only maize farmers in eastern Uganda, bean farmers in western Uganda, rice farmers in northern Uganda, and coffee farmers in central Uganda; (2) farmer characteristics, determined by the specific eligibility rules established by ACDP. To approximate the more homogeneous ACDP target population using the UNPS data, we estimate parameters using data from a restricted sample of households farming between 3 and 5 acres, a criterion for ACDP eligibility. Even the restricted UNPS sample is more heterogeneous than what the ACDP sample will be, and we are therefore confident that our power calculations are conservative.
Our power calculations take maize yields as the outcome variable. While outcome variables other than yields are of interest, yields typically exhibit higher coefficients of variation than do outcome measures of input use and other farming practices to be studied in the evaluation. We are thus confident that designing a sample with adequate power for maize yields will give us sufficient power to look at other key indicators and crops as well. We report calculations for two cases in which we assume mean yields of 345 kg/ac, the mean maize yields for farmers in eastern Uganda cultivating 3-5 acres of land in the second season of the 2011/12. The remaining parameters for the two cases are:
(1) Expected parameter case - The standard deviation of yields is assumed to be 300 kg/ac, which is based on the coefficient of variation for bean yields of farmers in western Uganda, and the parish-level intra-cluster correlation is assumed to be 12%, which is just below the village-level intra-cluster correlation for bean yields of farmers in western Uganda (14%);
(2) Conservative parameter case - The standard deviation of yields is assumed to be 388 kg/ac, which is based on the coefficient of variation for maize yields of farmers in eastern Uganda, and the parish-level intra-cluster correlation is assumed to be 18%, which was the village-level intra-cluster correlation for maize yields of farmers in eastern Uganda.
Based on our power analysis (summarized below), we propose to select our study sample from 5 sub-counties across the four study districts. In 2 sub-counties, we will randomly select 24 farmer groups (8 with high initial subsidy, 8 with low initial subsidy, and 8 with no subsidy). In 1 sub-county, we will randomly select 12 farmer groups (4 with high initial subsidy, 4 with low initial subsidy, and 4 with no subsidy). In 1 sub-county, we will randomly select 32 farmer groups (10 with high initial subsidy, 10 with low initial subsidy, and 12 with no subsidy). In the final sub-county, we will randomly select 27 farmer groups (9 with high initial subsidy, 8 with low initial subsidy, and 10 with no subsidy), yielding 136 clusters overall for our sample.
We will then randomly select 20 households per farmer organization cluster from among ACDP-eligible households, yielding a total sample size of 2160 households for our main sample of directly treated farmers.
Our minimum detectable effect calculations assume a net uptake rate of 75%. At the expected voucher uptake rate of 75%, the proposed design is powered to detect effects of less than half that magnitude (19%); even under our case of conservative assumptions, the study design could detect a minimum effect of 32%. As long as net uptake of the e-voucher is no lower than 50%, the proposed study design will have adequate power to detect program impacts if ACDP achieves its 50% yield growth goal. While this yield growth goal is high, it appears achievable in Uganda based on government estimates as well as estimates from experimental trials and on-farm trials of farm inputs in Uganda. We are thus confident that we have adequate power to pick up expected yield changes, as well as changes in the other, less variable, outcome indicators of interest. Moreover, we expect the actual power of the study to be higher than what is estimated here, because we will collect multiple (seasonal) rounds of follow-up data and will ensure high levels of data quality through carefully programmed electronic survey instruments.