|
Field
Last Published
|
Before
November 06, 2015 09:25 AM
|
After
January 31, 2016 05:54 AM
|
|
Field
Experimental Design (Public)
|
Before
The eligible population for this study is all residential households with access to piped water in the City of Cape Town. We are excluding commercial customers and only including those classified as “domestic” by the City. Furthermore, as we are limiting our sample to free standing houses, we are also excluding households with bulk meters. This obviates issues regarding flats where one meter number could be linked to multiple households. It further ensures that the recipient of the message is most likely one of the household members consuming water.
We are excluding those customers consuming only in Tariff Block 1, as this is the lowest consumption band (6 kilolitres/month) and is supplied free by the City. These households are already consuming the lowest possible amount of water each month and thus we assume they are either the most vulnerable residents in the City of Cape Town or are not residing in the household on a full-time basis (i.e. holiday home). If they are the most vulnerable residents in Cape Town, we want to safeguard against messaging them as it may be unethical (i.e. hazardous) to ask the lowest income residents to consume less water.
However, as we are interested in how different behavioural messages affect consumption across all income groups, we are including households with “indigent” status in the treatments - provided they are consuming above Tariff Block 1. Indigent households in Cape Town are those deemed most vulnerable and, according to the City’s policy, struggle to pay for public utilities. Given that those receiving an indigent grant receive varying levels of subsidization of utilities, or receive utilities free of charge, we chose not to include them in the financial incentives treatments (treatments 2, 3 and 4). We will not message them regarding financial savings as our code would significantly overestimate the financial savings associated with a reduction in water consumption.
The sample size is the population of the City of Cape Town excluding commercial and bulk accounts, apartment flats, and those consuming only within Tariff Block 1. This is a total expected sample size of 412,480 households: 113,312 are indigent and 299,168 are not indigent residents. We stratify the sample on suburbs (676) and tariff blocks (6 total). This leads to a total of 3213 distinct stratas. Using the stratas, we randomize the sample in slightly unequal numbers. The uneven allocation is due to i) inability to use indigent sample in the financial message treatments; ii) inflating the social recognition treatments due to the inability to message those households which receive estimated readings in the first month of the treatment; iii) inflating all treatments to account for households with meter readings higher than 35 days and also allow for those who fall into tariff block 1 to drop from the treatment.
|
After
The eligible population for this study is all residential households with access to piped water in the City of Cape Town. We are excluding commercial customers and only including those classified as “domestic” by the City. Furthermore, as we are limiting our sample to free standing houses, we are also excluding households with bulk meters. This obviates issues regarding flats where one meter number could be linked to multiple households. It further ensures that the recipient of the message is most likely one of the household members consuming water.
We are excluding those customers consuming only in Tariff Block 1, as this is the lowest consumption band (6 kilolitres/month) and is supplied free by the City. These households are already consuming the lowest possible amount of water each month and thus we assume they are either the most vulnerable residents in the City of Cape Town or are not residing in the household on a full-time basis (i.e. holiday home). If they are the most vulnerable residents in Cape Town, we want to safeguard against messaging them as it may be unethical (i.e. hazardous) to ask the lowest income residents to consume less water. However, as we are interested in how different behavioural messages affect consumption across all income groups, we are including households with “indigent” status in the treatments - provided they are consuming above Tariff Block 1.
The sample size is the population of the City of Cape Town excluding commercial and bulk accounts, apartment flats, and those consuming only within Tariff Block 1. This is a total expected sample size of 412,480 households: 113,312 are indigent and 299,168 are not indigent residents. We stratify the sample on suburbs (676) and tariff blocks (6 total).
|
|
Field
Randomization Method
|
Before
The randomization is done using Stata v13. First we create stratas across suburbs (676 total) and tariff blocks (6 total). This leads to a total of 3,213 distinct stratas. We generate random numbers to each household and then by strata create a rank of the random numbers. Then we randomise on two subsets: indigent, and non-indigent. By strata, we allocate the indigent sample equally across all six treatments and control. For the non-indigent, we create groups by strata and allow unequal allocation across treatments to account for households that will drop if they are in tariff block 1, have a meter reading greater than 35 days, and have an estimated reading for the first month (for social recognition treatments).
|
After
The randomization is done using Stata v13.
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
We used the most recent consumption data from November 2014 to April 2015 from the City of Cape Town’s municipal database to conduct our power calculations. For the power calculations. we chose to use the months for which our study will be conducted in order to allow for seasonality effects as consumption increases in the summer months.
We matched the municipal data with the list of contract accounts we received from the City's printers. We removed those consuming 6 kiloliters/month or below, as well as the 95th percentile to control for outliers due to measurement errors. We then calculated mean consumption over the treatment period last year (December-April).
We include two power calculations: one where we look at the mean consumption over the treatment period with an unbalanced panel and one where we use the balanced panel.
I) With our sample size, we are able to detect a 1.5% change in means per treatment. Assuming our standard deviation is 11.08, our mean is 21.47 kiloliters/month, alpha level is 0.05 and power of .8, the 1.5% detectable difference in means would be able to pick up an effect if the consumption decreases to 21.15 kiloliters/month (a difference of 0.32 kiloliters/month) with a minimum sample size of 18,579 per arm. We have tried various strategies for the power calculations, yet the strategy is not sensitive to changes in the detectable effect size.
We assume there will be high variability in the effect size across income groups. We will use property values and suburb as covariates in our regression models to decrease the variance.
II) Our power calculations are robust when using the balanced sample (those whose consumption we observe in each month)
With our sample size, we are able to detect a 1.5% change in means per treatment. Assuming our standard deviation is 9.5, our mean is 21.1 kiloliters/month, alpha level is 0.05 and power of .8, a 1.5% detectable difference in means would be able to pick up an effect if the consumption decreases to 20.8 kiloliters/month (a difference of 0.31 kiloliters/month) with a sample size of 14,104 households per arm.
|
After
We used the most recent consumption data from November 2014 to April 2015 from the City of Cape Town’s municipal database to conduct our power calculations. For the power calculations. we chose to use the months for which our study will be conducted in order to allow for seasonality effects as consumption increases in the summer months. We matched the municipal data with the list of contract accounts we received from the City's printers. We removed those consuming 6 kiloliters/month or below, as well as the 95th percentile to control for outliers due to measurement errors. We then calculated mean consumption over the treatment period last year (December-April).
We include two power calculations: one where we look at the mean consumption over the treatment period with an unbalanced panel and one where we use the balanced panel.
I) With our sample size, we are able to detect a 1.5% change in means per treatment. Assuming our standard deviation is 11.08, our mean is 21.47 kiloliters/month, alpha level is 0.05 and power of .8, the 1.5% detectable difference in means would be able to pick up an effect if the consumption decreases to 21.15 kiloliters/month (a difference of 0.32 kiloliters/month) with a minimum sample size of 18,579 per arm. We have tried various strategies for the power calculations, yet the strategy is not sensitive to changes in the detectable effect size.
We assume there will be high variability in the effect size across income groups. We will use property values and suburb as covariates in our regression models to decrease the variance.
II) Our power calculations are robust when using the balanced sample (those whose consumption we observe in each month)
With our sample size, we are able to detect a 1.5% change in means per treatment. Assuming our standard deviation is 9.5, our mean is 21.1 kiloliters/month, alpha level is 0.05 and power of .8, a 1.5% detectable difference in means would be able to pick up an effect if the consumption decreases to 20.8 kiloliters/month (a difference of 0.31 kiloliters/month) with a sample size of 14,104 households per arm.
|