Do Water Audits Promote Economic Welfare?: Evidence from a Natural Field Experiment

Last registered on June 04, 2022

Pre-Trial

Trial Information

General Information

Title
Do Water Audits Promote Economic Welfare?: Evidence from a Natural Field Experiment
RCT ID
AEARCTR-0009486
Initial registration date
May 30, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 04, 2022, 11:07 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
University of Southern California

Other Primary Investigator(s)

PI Affiliation
The Behavioralist
PI Affiliation
University of Southern California
PI Affiliation
University of Oxford

Additional Trial Information

Status
Completed
Start date
2018-12-01
End date
2019-02-28
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Water suppliers are showing greater interest in using non-price mechanisms that can help encourage conservation. One such mechanism is water audits, which involve assessing customer behavior and the water efficiency of their home. The audits also provide tailored suggestions regarding how to reduce water use. Yet, very little is known about the efficacy, efficiency, and cost-effectiveness of water audits. This paper helps fill this research gap by implementing a natural field experiment in the United Kingdom. We randomly allocate 45,000 water customers to a control group or to groups that are provided with different encouragements to take-up an online water audit. Our analysis of the data will allow us to both study how to promote uptake of online water audits, in addition to the economic effects of water audits.
External Link(s)

Registration Citation

Citation
Akesson, Jesper et al. 2022. "Do Water Audits Promote Economic Welfare?: Evidence from a Natural Field Experiment." AEA RCT Registry. June 04. https://doi.org/10.1257/rct.9486
Experimental Details

Interventions

Intervention(s)
There were six letter treatments. Treatment 1 (\emph{Vanilla}) informed customers that they can save water and money by using the free online platform. It also noted that many other customers had saved money with the platform, and told them how to access it. Treatment 2 (\emph{Simplified}) was similar to the \emph{Vanilla} communication but it simplified the content, making the main message and the call to action more salient. Treatment 3 (\emph{Altruism}) added to the message of the \emph{Simplified} mailer by reminding the consumers that water is a scarce resource, and asked them to help conserve it in their local area. Treatment group 4 (\emph{Moral Cost}) received a letter that complemented the \emph{Simplified} Mailer by telling customers that people in their region were making a change in an effort to save water, and invited them to join their neighbors. Furthermore, for consumers with relatively high water consumption, it informed them that they were in the top 50 per cent of consumption, whereas for the bottom 50 per cent, it congratulated them on being efficient. The final two treatment groups, Treatment 5 and Treatment 6, were offered pecuniary incentives (\emph{\textsterling 10 Incentive} and \emph{\textsterling 15 Incentive}) for completing the water audits. The former supplemented the \emph{Simplified} mailer by emphasizing monetary savings, and offered a \textsterling 10 incentive for using the platform, while the latter communication changed the incentive from \textsterling 10 to \textsterling 15.
Intervention Start Date
2018-12-08
Intervention End Date
2019-02-06

Primary Outcomes

Primary Outcomes (end points)
Water Consumption, Take up of the audit
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
In 2018, Northumbrian Water commissioned Save Water Save Money (SWSM) to provide its online water audit tool for Northumbrian’s customers. The tool, hosted on the company website, asked customers questions about their water use habits and homes. The main purpose of the tool was to help customers understand their water consumption, and identify ways in which they can save water and money. The tool also informed customers about free water-saving devices that NWL offers, and helped them book an in-home water audit if appropriate. The questionnaire on the platform took approximately ten minutes to complete. NWL was interested in getting its customers to take their online water audit, and understanding the impact of the audits on consumption. We were interested in helping NWL with these objectives, and, in addition, understanding the impact of different behavioral interventions on economic welfare. In order to encourage the use of the SWSM platform, we designed a set of customer communications using theories from behavioral science. We used one of NWL’s existing direct mailers as a template, and designed 5 new direct mailers. The only difference between the five communications was the application of different behavioral science ideas. We implemented an RCT to test the effectiveness of the redesigned letters, and to understand how the SWSM platform influences water consumption. This RCT included 45,000 NWL customers, spread across three post code areas. The customers that participated in the trial were randomly allocated to one of six treatment groups that received letters or a control group that received no letter. Subsequently, customers for whom NWL had email contact details were also randomly allocated to groups that either received or did not receive an email reminder about the online audit tool. The reminder emails followed the same theme as the initial letters that customers received. This design allows us to estimate the effects of particular letters and reminders on take-up of the audit. There were six letter treatments. Treatment 1 (Vanilla) informed customers that they can save water and money by using the free online platform. It also noted that many other customers had saved money with the platform, and told them how to access it. Treatment 2 (Simplified) was similar to the Vanilla communication but it simplified the content, making the main message and the call to action more salient. Treatment 3 (Altruism) added to the message of the Simplified mailer by reminding the consumers that water is a scarce resource, and asked them to help conserve it in their local area. Treatment group 4 (Moral Cost) received a letter that complemented the Simplified Mailer by telling customers that people in their region were making a change in an effort to save water, and invited them to join their neighbors. Furthermore, for consumers with relatively high water consumption, it informed them that they were in the top 50 per cent of consumption, whereas for the bottom 50 per cent, it congratulated them on being efficient. The final two treatment groups, Treatment 5 and Treatment 6, were offered pecuniary incentives (£10 Incentive and £15 Incentive) for completing the water audits. The former supplemented the Simplified mailer by emphasizing monetary savings, and offered a £10 incentive for using the platform, while the latter communication changed the incentive from £10 to £15. The data used to randomize the trial participants and to measure outcomes came from three anonymized sources: NWL’s administrative data on meter readings; the SWSM platform, which was used to code responses to the diagnostic questionnaire; and Customer Relationship Management (CRM) data identifying whether reminder emails were opened. The experiment took place over four months between December 2018 and March 2019. We collected baseline data for purposes of randomization and analysis of pre-treatment
consumption from January 2017. All direct mailers were posted on 8th December 2018, and email reminders were sent on 6th February 2019.
Experimental Design Details
Randomization Method
Simple Randomization
Randomization Unit
Individual customer
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1
Sample size: planned number of observations
45000
Sample size (or number of clusters) by treatment arms
7500 per treatment arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Assuming that the average in the comparison group is 0.7%, and we want a MDE of 0.5 percentage points, we need approximately 5.9k individuals per treatment arm to achieve 80% power. In our experiment, we have 7,500 per treatment arm.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials