Do Water Audits Promote Economic Welfare?: Evidence from a Natural Field Experiment

There were six letter treatments. Treatment 1 (\emph{Vanilla}) informed customers that they can save water and money by using the free online platform. It also noted that many other customers had saved money with the platform, and told them how to access it. Treatment 2 (\emph{Simplified}) was similar to the \emph{Vanilla} communication but it simplified the content, making the main message and the call to action more salient. Treatment 3 (\emph{Altruism}) added to the message of the \emph{Simplified} mailer by reminding the consumers that water is a scarce resource, and asked them to help conserve it in their local area. Treatment group 4 (\emph{Moral Cost}) received a letter that complemented the \emph{Simplified} Mailer by telling customers that people in their region were making a change in an effort to save water, and invited them to join their neighbors. Furthermore, for consumers with relatively high water consumption, it informed them that they were in the top 50 per cent of consumption, whereas for the bottom 50 per cent, it congratulated them on being efficient. The final two treatment groups, Treatment 5 and Treatment 6, were offered pecuniary incentives (\emph{\textsterling 10 Incentive} and \emph{\textsterling 15 Incentive}) for completing the water audits. The former supplemented the \emph{Simplified} mailer by emphasizing monetary savings, and offered a \textsterling 10 incentive for using the platform, while the latter communication changed the incentive from \textsterling 10 to \textsterling 15.

There were six letter treatments. Treatment 1 (\emph{Vanilla}) informed customers that they can save water and money by using the free online platform. It also noted that many other customers had saved money with the platform, and told them how to access it. Treatment 2 (\emph{Simplified}) was similar to the \emph{Vanilla} communication but it simplified the content, making the main message and the call to action more salient. Treatment 3 (\emph{Altruism}) added to the message of the \emph{Simplified} mailer by reminding the consumers that water is a scarce resource, and asked them to help conserve it in their local area. Treatment group 4 (\emph{Moral Cost}) received a letter that complemented the \emph{Simplified} Mailer by telling customers that people in their region were making a change in an effort to save water, and invited them to join their neighbors. Furthermore, for consumers with relatively high water consumption, it informed them that they were in the top 50 per cent of consumption, whereas for the bottom 50 per cent, it congratulated them on being efficient. The final two treatment groups, Treatment 5 and Treatment 6, were offered pecuniary incentives (\emph{\textsterling 10 Incentive} and \emph{\textsterling 15 Incentive}) for completing the water audits. The former supplemented the \emph{Simplified} mailer by emphasizing monetary savings, and offered a \textsterling 10 incentive for using the platform, while the latter communication changed the incentive from \textsterling 10 to \textsterling 15.

2018-12-08

2019-02-06

Water Consumption, Take up of the audit

In 2018, Northumbrian Water commissioned Save Water Save Money (SWSM) to provide its online water audit tool for Northumbrian’s customers. The tool, hosted on the company website, asked customers questions about their water use habits and homes. The main purpose of the tool was to help customers understand their water consumption, and identify ways in which they can save water and money. The tool also informed customers about free water-saving devices that NWL offers, and helped them book an in-home water audit if appropriate. The questionnaire on the platform took approximately ten minutes to complete. NWL was interested in getting its customers to take their online water audit, and understanding the impact of the audits on consumption. We were interested in helping NWL with these objectives, and, in addition, understanding the impact of different behavioral interventions on economic welfare. In order to encourage the use of the SWSM platform, we designed a set of customer communications using theories from behavioral science. We used one of NWL’s existing direct mailers as a template, and designed 5 new direct mailers. The only difference between the five communications was the application of different behavioral science ideas. We implemented an RCT to test the effectiveness of the redesigned letters, and to understand how the SWSM platform influences water consumption. This RCT included 45,000 NWL customers, spread across three post code areas. The customers that participated in the trial were randomly allocated to one of six treatment groups that received letters or a control group that received no letter. Subsequently, customers for whom NWL had email contact details were also randomly allocated to groups that either received or did not receive an email reminder about the online audit tool. The reminder emails followed the same theme as the initial letters that customers received. This design allows us to estimate the effects of particular letters and reminders on take-up of the audit. There were six letter treatments. Treatment 1 (Vanilla) informed customers that they can save water and money by using the free online platform. It also noted that many other customers had saved money with the platform, and told them how to access it. Treatment 2 (Simplified) was similar to the Vanilla communication but it simplified the content, making the main message and the call to action more salient. Treatment 3 (Altruism) added to the message of the Simplified mailer by reminding the consumers that water is a scarce resource, and asked them to help conserve it in their local area. Treatment group 4 (Moral Cost) received a letter that complemented the Simplified Mailer by telling customers that people in their region were making a change in an effort to save water, and invited them to join their neighbors. Furthermore, for consumers with relatively high water consumption, it informed them that they were in the top 50 per cent of consumption, whereas for the bottom 50 per cent, it congratulated them on being efficient. The final two treatment groups, Treatment 5 and Treatment 6, were offered pecuniary incentives (£10 Incentive and £15 Incentive) for completing the water audits. The former supplemented the Simplified mailer by emphasizing monetary savings, and offered a £10 incentive for using the platform, while the latter communication changed the incentive from £10 to £15. The data used to randomize the trial participants and to measure outcomes came from three anonymized sources: NWL’s administrative data on meter readings; the SWSM platform, which was used to code responses to the diagnostic questionnaire; and Customer Relationship Management (CRM) data identifying whether reminder emails were opened. The experiment took place over four months between December 2018 and March 2019. We collected baseline data for purposes of randomization and analysis of pre-treatment
consumption from January 2017. All direct mailers were posted on 8th December 2018, and email reminders were sent on 6th February 2019.

Simple Randomization

Individual customer

No

1

45000

7500 per treatment arm

Assuming that the average in the comparison group is 0.7%, and we want a MDE of 0.5 percentage points, we need approximately 5.9k individuals per treatment arm to achieve 80% power. In our experiment, we have 7,500 per treatment arm.