Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Cluster-level randomized treatments
Public / private treatment: MDES of 0.17
The public-private treatment is randomized at the cluster level. There are 400 clusters, each with 10 households. In this treatment we will compare the overall use of mechanized desludging versus manual desludging. We estimate the necessary sample size assuming a standardized effect size of 0.2 and an intra-cluster correlation coefficient of 0.2, based on desludging data from a related study in Dakar. With 10 households per cluster, the cluster-corrected power calculation suggests that we need at least 364 clusters. This corresponds to a power level of 95% and confidence level of 5%. We have selected 400 project clusters.
We will also evaluate the effect of this treatment on the follow-through rate for those who commit to mechanized desludging. Because this outcome can only be measured for participants who sign up for a subsidized desludging through the project, there will be fewer participants per cluster. We adjust the power calculation assuming only 4 households per cluster. We expect a greater standardized effect size for this group of 0.25. Keeping the same power and confidence levels, we need at least 333 clusters.
Learning from others treatment: MDES of 0.17 (0.28 comparing treatments)
In half of the neighborhood clusters (200 clusters of 10 households), we will provide information about the first five households' desludging decisions to the last five participants. We will notify the last five of how many of their first five neighbors have signed up for a desludging subscription, with half of the treated group learning only the number of neighbors and half also learning the names of subscribing neighbors. We will then compare the takeup and follow-through rates of the first fives with the last fives, and also of the number-only treatment group with the names treatment group.
Within a cluster, the assignment of households to the first five or last five is randomized at the individual level. In order to detect a standardized effect size of 0.2 for the learning from others treatment, we will need a sample size of at least 1300 households for a 5% confidence level with 95% power. Conducting this experiment with 2000 households is therefore sufficient. We will be able to compare the 1000 last five households with control groups from the 1000 first five households, as well as with the 2000 households not included in this experiment, giving a total of 1000 treated and 3000 control.
We will also compare the two treated groups – whether the households are told how many or who among their neighbors sign up for a mechanized desludging. These treatments are randomized at the cluster level, with 5 treated households per cluster. Because this intervention is logistically difficult and costly, we are not able to increase the sample beyond 100 clusters (500 households) per treatment arm. Still, this sample size will be sufficient to detect a standardized effect of 0.25 with 82% power, or a standardized effect of 0.3 with 94% power.
Individual-level randomized treatments
Earmarking treatment: MDES of 0.17
This treatment will compare those who received a single savings account earmarked for desludging with those who received both a desludging account and a normal savings account. Because not all participants will opt into the payment plan, we have adjusted our sample size to account for an estimated 40% expected takeup (hence 1600 participants in the earmarking experiment). For the earmarking experiment, our outcomes of interest are both total levels of savings and final use of the desludging services. We will compare the proportion of households who follow through with mechanical desludging after signing up for a subscription plan.
We estimate that use of the desludging services (follow-through rate) will be about 70% for those with one account. In order to detect an increase to 80% or higher, we require a sample size of at least 962. We therefore believe our sample size of 1600 should be conservative for the earmarking treatment arms.
In terms of the savings level outcome for the earmarking experiment, we expect a standardized effect size of at least 0.2, which suggests that we need a sample size of at least 1300.
Commitment deposit treatment: MDES of 0.14
About 3500 households will be asked to pay a deposit toward a subsidized desludging at the time of the decider survey, and 500 will be offered a subsidized desludging with no deposit. We will compare the proportion of households who follow through with mechanical desludging among those who left a deposit and those who did not. This sample size is sufficient to detect a 0.2 standardized effect size at 5% confidence with over 98% power.
Note that this treatment is cross-randomized with the payment plans. Half of each deposit treatment arm will receive a subscription payment plan. We expect the effect of the commitment deposit treatment to be similar across the different subsidy levels.
Spillover estimation: MDES of 0.13
Two households (out of 12) within each cluster will not receive any treatment in order to measure spillover effects. We estimate the sample size necessary in order to have 95% power to calculate an effect that is statistically different from 0 at a 5% level of confidence, and still adjust for likely intra-cluster correlation of the two households.
Using a similar framework in estimating the effect of learning on adoption of mosquito nets, Dupas (2010) finds a standardized effect size of having all neighbors receive the maximum subsidy of 0.44. (The coefficient on the share of households with the maximum subsidy within 500 meters is 0.215, and the standard deviation of adoption is 0.489.) We estimate the necessary sample size assuming a more conservative standardized effect size of 0.2 and an intra-cluster correlation coefficient of 0.2. With 2 households per cluster, the cluster-corrected power calculation suggests that we need 780 spillover households to compare with 3900 participant households. We will therefore use all 400 of our project clusters, for a total of 800 spillover households.