AEA RC T Registr y

The American Economic Association's registry for randomized controlled trials

We evaluate the effects of an intervention designed to improve access to safe drinking water in rural Bangladesh. The intervention consists of a package of subsidies and technical advice to build new sources of water, which provide drinking water that is free of both arsenic and bacterial contamination. The new safe sources of water are deep tubewells, which draw water from aquifers that are sufficiently deep to be safe from both bacterial contamination and arsenic contamination.

The intervention is located in north-western Bangladesh, in Shibganj and Sonatala Upazilas in Bogra District and in Gobindaganj Upazila in Gaibandha District, and it is fully implemented by the Bangladeshi NGO "NGO Forum for Public Health". We enroll 171 communities in the study, of which 129 are randomly selected to receive the intervention. The subsidies range in value from 90% to 100% of the cost of water source installation. Required community contributions are either in cash or kind (labor for installation work). Communities decide the location of the new water sources by unanimous consensus in community meetings in the presence of NGO Forum staff.

2015-10-15

2017-08-31

Our first research question is: "What is the average effect of the program on household water quality?", with corresponding main outcomes of interest:
(1a) arsenic contamination in drinking water;
(1b) fecal contamination in drinking water.

Our second research question is: "How does the program change behavior with respect to obtaining and storing water for drinking and cooking?", with corresponding main outcomes of interest:
(2a) arsenic contamination of water source(s) used by households;
(2b) fecal contamination of water source(s) used by households;
(2c) distance walked to collect water;
(2d) household water storage practices.

Our third research question is: "What is the causal effect of the behavioral channels on household water quality?", with corresponding main outcomes of interest as for the first research question.

We test household drinking water and water obtained at the water source for arsenic contamination primarily using field arsenic tests. We use EZ Arsenic High Range Test Kit (Hach), which measures arsenic levels within the range of 0-500 ppb (parts per billion) with the following increments: 0, 10, 25, 50, 250, 500. We define arsenic contaminated water using both the WHO standard (10 ppb) and Bangladeshi standard (50 ppb).

We test household drinking water and water obtained at the water source for fecal bacteria contamination using hydrogen sulfide vials produced by NGO Forum for Public Health. We use dummy variables to indicate whether the test detected the presence of Escherichia coli in water.

We calculate distance walked to collect water using the difference between GPS recorded coordinates of households and used water source(s).

We primarily use enumerator observations of whether or not households retrieve drinking water from storage or directly from a source to measure storage practices. We will also compare these results to measures of self-reported habitual storage practice and additionally provide further descriptive evidence on the specific details of storage.

Throughout, we match households to the water source(s) used to collect drinking water. We verify correct matching showing to respondents the pictures of the selected water source(s), as collected during a baseline water source census. Where households use multiple sources, our primary analysis will aggregate the source quality and distance measures based on the fraction of drinking water drawn from each source.

We conducted the study in 171 communities eligible to receive the arsenic mitigation program. The mean community size is 153 households. The eligibility criteria were defined with respect to the baseline level of arsenic contamination: (i) communities where more than 25% of sources of drinking water are arsenic contaminated; or (ii) communities where fewer that 25% of sources are arsenic contaminated, but the arsenic-contaminated sources are highly geographically concentrated.

We randomly select 129 communities to receive the program by public lotteries. Treated communities are further randomly assigned to one of three requirements for co-funding the installation costs: no contributions; cash contributions; labor contributions.

We implemented the intervention using a Community Driven Development (CDD) approach. Communities are informed of the terms of the program, and select locations for the subsidized safe sources of drinking water and agree on how to divide the community contributions, where applicable. These decisions are taken by consensus at a community meeting with representation of women and the poor, and in the presence of project staff who act as moderators. Communities must then independently raise or coordinate the community contributions for the wells to be installed. Communities also take responsibility for maintenance, and if necessary repair, of the wells after they are installed.

To causally estimate changes in average household water quality and in behavior with respect to obtaining water for drinking and cooking, we primarily estimate reduced form “intent-to-treat" effects that exploit the random assignment of the program to treatment units.

In order to estimate the causal effects of water source quality and distance to safe drinking water on household water quality, we take two approaches. Our first analysis of mechanisms is a difference-in-difference approach where we evaluate how changes in household bacterial contamination vary with changes in source contamination, transport distance and storage practice. However, although assignment to the safe drinking water program is random, selection of locations for water source installation is endogenous, as it is determined by consensus at a community meeting. As a result, it is possible that changes in distance to water source, or water source contamination, are correlated with changes in other determinants of household drinking water contamination. These confounding factors might in principle bias the above analysis. To address this concern, we carry out a second, instrumental variables analysis.

The instrumental variables approach uses baseline data to predict where in the village a community will decide to install the new water source. We predict the location of the newly constructed water sources using two methods. First, we predict installation sites by visually inspecting the map of existing water sources in the community and selecting location(s) based on population density and existing source quality. Second, we program an algorithm to predict locations in treatment and control villages based on similar criteria. We then use the predicted source location, combined with household-level baseline data, to construct the following instruments: (i) predicted change in source fecal contamination; (ii) predicted change in distance to source of drinking water between baseline and follow-up. We use these predicted changes as instruments for observed changes in source fecal contamination and distance to drinking water.

We randomly assign communities to receive the original CDD program by public lottery at the presence of community representatives. The randomization is stratified by Union Parishad.

The randomization is performed at Treatment Unit level. Treatment Units consist of groups of 50-250 households, defined along natural geographic boundaries using administrative lists, GPS data and satellite imagery. We use the language "Treatment Unit" and community interchangeably in this document.

Yes

We carried out the CDD program in 171 communities, 129 selected to receive the intervention and 42 assigned to the control group.

Outcomes using household survey data: 7,145 households (target). Treatment Unit outcomes: 171.

The arsenic mitigation program was designed as follows: 42 control Treatment Units; 129 treated Treatment Units, evenly divided over three contribution requirements: no contributions; cash contributions; labor contributions.

We carry out power calculations by simulating follow-up data using baseline data, project implementation data, and plausible parameter values and assumptions about behavioral change (based on previous studies and our own experience). Intra-cluster correlation is modeled implicitly via the true intra-cluster correlation in the baseline data. Where relevant, we cluster standard errors by treatment unit. Our simulation is based on the following key assumptions: i) that absent our program, approximately 1/3 households would switch water source between baseline and follow-up; ii) that water quality in the new source would be a random draw from the baseline distribution in that treatment unit; iii) that distance to the new source is a random draw between the minimum distance to a source in the treatment unit and a 50% increase in walking distance. For simplicity, we assume that households rely on a single source, which is the case for 96% of observations at baseline. We calculate minimum detectable effects at the 5% level as 2.8 x the estimated standard deviation of coefficients. The minimum detectable effects of the intervention are: (i) 3.5% on arsenic contamination in household drinking water; (ii) 3.8% on fecal bacteria contamination in household drinking water; (iii) 2.4% on arsenic contamination at the water source used by the household; (iv) 2.8% on arsenic contamination at the water source used by the household; (v) 2.2 m on distance between household and water source used to collect drinking water; (vi) 3% on reported storage. In addition, we simulate the difference-in-difference analysis and the instrumental variables analysis. The estimated effect sizes for the difference-in-difference analyses compare favourably to plausible parameter values. Additionally, in these simulated power calculations, we obtain Sanderson-Windmeijer first stage F-statistics of more than 10 for both instruments used in the IV analysis in about 85% of simulations. However, the IV approach sacrifices considerable power: the IV approach has minimum detectable effects that are approximately ten times larger than the difference in difference approach. We will interpret the results from the IV analysis with these limitations in mind.

Pre-analysis Plan