Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We primarily evaluate the impact of our intervention on well switching (i.e., a high-arsenic well household switching to a nearby low-arsenic well). According to the well listing conducted at baseline census in January 2020, on average each village has 89 private well owners. We set the cluster size to be 80, considering potential attrition. There are 36 control villages with no explicit risk-sharing agreement and 99 villages with an explicit risk-sharing agreement (T1+T2+T3). The baseline switching rate is set to be 0.28 based on Barnwal et al. (2017). Alpha and Beta are set conventionally to be 0.05 and 0.8. Then:
MDE = 0.182 when ICC = 0.1;
MDE = 0.250 when ICC = 0.2;
MDE = 0.303 when ICC = 0.3;
MDE = 0.348 when ICC = 0.4;
MDE = 0.388 when ICC = 0.5.
For comparison, Tarozzi et al. (2021) use the ICC for switching decision of 0.09 in their power calculation. The 0.09 ICC was calculated from the data collected by Bennear et al. (2013).
Note that the switching rate depends not only on the number of coupons villagers shared but also on the arsenic level of their own wells. To understand the magnitude of these detectable effects, we use a simulation exercise to calculate the combinations of the average number of agreements and the likelihood of having high arsenic well that achieves these effects. For example, if each villager makes 5 commitment contracts with others in a village of only 30% of wells are safe, then each villager still has over 80% chance to get at least one safe well among the households they exchanged coupons with and thus achieving a switching rate of 80%.
The simulation provides that to achieve an MDE of 0.388 (ICC 0.5), villagers need to make 5.4 agreements with others in a village with 20% of safe well, and about 1.7 agreements in a village with 50% of the safe well. Assuming that well owners make 5-6 agreements on average and the targeted villages are predicted to have over 30% of safe wells, the calculated MDE is achievable. These numbers are calculated by assuming that villagers are perfectly committed to the agreement. The required number of coupon-exchanges will increase marginally to achieve this MDE when the commitment to the agreement is near perfect.
If we consider ICC of 0.09, as in Tarozzi et al. (2021), our experiment is sufficiently powered to detect a much smaller treatment effect.
References:
Barnwal, P., van Geen, A., von der Goltz, J., & Singh, C. K. (2017). Demand for environmental quality information and household response: Evidence from well-water arsenic testing. Journal of Environmental Economics and Management, 86, 160-192.
Bennear, L., Tarozzi, A., Pfaff, A., Balasubramanya, S., Ahmed, K. M., & Van Geen, A. (2013). Impact of a randomized controlled trial in arsenic risk communication on household water-source choices in Bangladesh. Journal of environmental economics and management, 65(2), 225-240.
Tarozzi, A., Maertens, R., Ahmed, K. M., & Van Geen, A. (2021). Demand for Information on Environmental Health Risk, Mode of Delivery, and Behavioral Change: Evidence from Sonargaon, Bangladesh. The World Bank Economic Review, 35(3), 764-792.