Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We perform a power calculation using our baseline data to determine the minimum detectable effect size (MDE). The key parameters are as follows: α = 0.05 (standard type I error), κ = 0.80 (standard power), J = 110 (number of clusters (villages) per treatment arm), number of respondents under three treatment arms T1 = 1359 (control), T2 = 1374 (IVR only Treatment), and T3 = 1371 (IVR + Access to Doctor Treatment), and we used the low ICC = 0.15 (intra-cluster correlation coefficient). For the actual power analysis, we round off the latter two numbers to 1370.
Using one of our main outcome variables, the required number of Ante-Natal Care (ANC) visits, we compute the MDE. According to our baseline, only 2% of respondents in the three treatment arms have access to the required number of ANC visits. Therefore, our study is sufficiently powered to detect an effect size of 0.0331 or 3.31 percentage points. In case we experience very high attrition of roughly 30% (which would reduce the treatment sample to 959 and the control sample to 951), the minimum detectable effect size changes very little to 0.0359. For the MDE, we need less than 4 percentage points improvement in ANC visits, and given such a low base we expect the impact will be much higher.
We implement a similar exercise for a second outcome of interest, Post-Natal Care (PNC) visits. According to the baseline, only 5% of women (overall) have access to four required number of PNC visits. Our study is sufficiently powered to detect an effect size of 0.0491, which is 4.91 percentage points (or 0.0528, with 30% attrition). For the MDE, we need around 5 percentage points improvement in PNC visits, and given such a low base we expect a much higher treatment effect.
Similarly, we also compute the MDE of the treatment for some other outcome variables such as knowledge of five risk factors during pregnancy, providing additional food to the child after six months, breastfeeding duration, breastfeeding time, and going to Kabiraj (witch doctor) during complexity. Our study is sufficiently powered to detect an effect size of (i) 0.0381 for knowledge of five risk factors (or 0.0412 with 30% attrition); (ii) 0.0611 (or 0.0647) for knowledge of providing additional food to the child after six months of birth; (iii) 0.0882 (or 0.0939) for knowledge of breastfeeding duration (1 = if continue breastfeeding until the age of 6 months); (iv) 0.0775 (or 0.0823) for knowledge of breastfeeding time (=1 if started within an hour); and (v) 0.0865 (or 0.0823) for knowledge of going to Kabiraj (witch doctor) during complexity (=1 if say yes).