Behavior Change to Save Energy in Low-Income, Urban Households

Last registered on April 09, 2015

Pre-Trial

Trial Information

General Information

Title
Behavior Change to Save Energy in Low-Income, Urban Households
RCT ID
AEARCTR-0000666
Initial registration date
April 09, 2015

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 09, 2015, 6:25 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
UC Berkeley

Other Primary Investigator(s)

PI Affiliation
University of Maryland

Additional Trial Information

Status
On going
Start date
2014-10-20
End date
2015-07-06
Secondary IDs
Abstract
Low-income households allocate a disproportionately higher share of their disposable income to energy expenses than other households do. According to the Energy Information Administration, American households with an annual income below $20,000 spent over 7.9% of their income on energy. This number is below 0.3% for households earning more than $120,000 per year. As a result, policies that aim to internalize negative externalities in the energy sector are notoriously regressive (Grainger and Kolstad, 2010). Currently, this regressivity in the energy sector is addressed with two types of policies: increasing block pricing and means-tested subsidized tariffs. In practice, the redistribution gains from these policies are modest (Borenstein, 2012). Every year, thousands of low-income households are simply not able to afford energy services, fall into arrears, and end up at risk of being disconnected.

Across the country, institutions such as the Fuel Fund of Maryland and the Dollar Energy Fund help families struggling with their utility bills. Their mission is to provide small grants to low-income households that have received a disconnection notice from their utility. Though bill assistance plays an important role, it is a last resort solution to maintain energy services and does not address the long-term issue of how to help low-income households cope with high and often increasing energy bills. Policies that help families reduce their energy demand can thus potentially play an important role in this context. One such policy is energy education.

For decades, analysts have argued that there are opportunities to lower energy demand through low-cost behavior change (Diezt et al. 2009). Though the barriers to such opportunities are still hotly debated (Allcott and Greenstone, 2012), recent evidence suggests that providing energy information can lead to short-term and long-term reduction in energy usage (Allcott, 2011, Allcott and Rogers 2014, Houde et al. 2013). Yet important research questions still exist about the effectiveness of different technologies for providing information to customers and about specific impacts on low-income households. For instance, energy education that aims to engage households in active learning about low-cost behavior change has been called a promising solution (Wilson and Dowlatabadi, 2007), but its effectiveness is still mostly unknown.

Using a randomized experiment, we investigate the role of energy education provided to low-income households that face the risk of disconnection from their utility services. Our first goal is to evaluate whether energy education can lead to a decrease in energy demand. Our second goal is to investigate the effectiveness of different educational technologies, particularly comparing online education to in-person programs.

Registration Citation

Citation
Houde, Sebastien and Catherine Wolfram. 2015. "Behavior Change to Save Energy in Low-Income, Urban Households." AEA RCT Registry. April 09. https://doi.org/10.1257/rct.666-1.0
Former Citation
Houde, Sebastien and Catherine Wolfram. 2015. "Behavior Change to Save Energy in Low-Income, Urban Households." AEA RCT Registry. April 09. https://www.socialscienceregistry.org/trials/666/history/4055
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The Fuel Fund of Maryland (FF) is an NGO that has been providing financial assistance for over thirty years. When unable to honor their energy bills and at risk of having their energy service turned off by Baltimore Gas and Electric (BGE), low-income residents from Maryland can request financial assistance. The debt is split into thirds, shared by BGE, FF, and the customer. Since 2008, FF has also been running Watt Watchers (WW), a program that provides energy education and outlines demand-side energy reduction strategies for clients in need of bill assistance.

Watt Watchers is a heavily underutilized program that through Summer 2014 was optional to FF clients. Starting in the Fall 2014, completing the WW training became a requirement in order to qualify for bill assistance. WW also began rolling out an online tool around the same time. We took advantage of these two changes to design an extremely simple, valid, and low-cost intervention.

At registration, clients get randomly assigned to one of three groups: a control group for whom training is not required; a treatment group where clients are required to take the 45-minute online course; and a second treatment group where clients are required to participate in two 90-minute on-site sessions led by FF. We are not expecting full compliance in the study. FF cannot deny financial help even if the training is not completed. These clients are about to get their electricity and/or gas shut off, and FF’s mission is to help families afford energy, especially during winter. Because of this, a simple comparison of averages between control and treatments will solely measure ITT. Although the ITT is arguably interesting for policy reasons, we will also estimate a two-stage least squares regression, instrumenting on assignment to a group.

FF started randomly assigning clients to groups in October 2014. In March, we will start randomizing the contents of confirmation letter that WW sends after registration to measure its effects on compliance. Two additional randomizations will be put in place. First, to understand the most cost-effective way to generate compliance, we will randomize t reated clients (on-site/online) into three groups. After registering, some will receive a simple confirmation letter; some will receive this same letter, with a also informing them that they have the chance to enter a lottery for an autographed Ravens football if they complete their WW training; and a third group will receive the same letter, and inform them that they have the chance to enter a lottery for a gift card of approximately the same financial value as the Ravens football if they complete their WW training ($250).

Second, part of the WW curriculum involves rewarding on-site participants with a gift bag containing energy-related goods after the completion of the course. We want to understand the impact of this gift on energy consumption. For this, we will randomize online participants to receive or not the same gift bag after they complete the training.
Intervention Start Date
2014-10-20
Intervention End Date
2015-07-06

Primary Outcomes

Primary Outcomes (end points)
There are two outcome variables we are interested in measuring:
1. Compliance decision with the treatment
2. Energy/cost savings due to the treatment
Primary Outcomes (explanation)
We define our quantities of interest based on the following formulas. The data for month(s) where a household was disconnected will be treated as missing (not zero).

E[Btu/m | connected,online]- E[Btu/m| connected,control]
E[Btu/m| connected,onsite]- E[Btu/m | connected,control]

where Btu measures the combined gas and electricity consumption of a household (kWh and therms are converted to Btu using standard conversion factors). We will separately examine gas (therms) and electricity (kWh) consumption as a robustness check. We will also calculate the average daily usage level, where we assume consumption is uniform across a month and only measure days that the household was connected. This interpolation allows us to keep the number of days right around connection/disconnection. Additionally, we will see how consumption changes before and after connection for everyone. This is not experimental, but is interesting descriptive evidence.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This is a randomized controlled trial with two treatment arms (online and on site) and a control group. Group assignments are made on a week-location basis, where clients registered on the same location and same week are randomized to the same group. Completing the training is presented as mandatory to participants, but the Fuel Fund does not deny financial assistance to clients who complied with the financial requirements but did not complete the Watt Watchers training. This means that compliance is an issue that the analysis has to take into consideration.

Additionally, for clients who got assigned to one of the treatment arms, we have an encouragement component also by week-location. We are randomizing the registration letters that clients receive after they get assigned to the on site or online versions of the course. This means that one third of these clients will receive a simple confirmation letter; a third will receive the same letter with the chance to enter a lottery for an autographed Ravens football if they complete their WW training; the final third will also receive the same letter, and the chance to enter a lottery for a gift card of approximately the same value as the Ravens football ($250) if they complete their WW training.

Finally, part of the WW curriculum involves rewarding onsite participants with a gift bag containing energy-related goods after the completion of the course. We want to understand the impact of this gift on energy consumption. For this, we will randomize online participants to receive or not the same gift bag after they complete the training.
Experimental Design Details
Randomization Method
The randomization was done at a week-location level, prior to the beginning of the data collection. This means that all locations (main office and local agencies) will follow a pre-defined calendar of randomization. This calendar will inform the case worker in charge of the first interaction with the client in which group he/she should be placed. Every week the assignment changes, but all clients registering in the same location during one week will be placed in the same group. The schedule was developed such that the cycles in all location are neither predictable nor consistent (e.g., online-onsite-control-online-onsite-control...), although we guaranteed that every location receives at least one of each groups in every month. Each location follows an independent schedule. This means that, for instance, when the Salvation Army is on an “online week”, the main office might be on an “onsite week”.
Randomization Unit
The randomization units are week-locations: all participants registering on a specific week and location get assigned to the same group.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Standard errors will be clustered at the month-office level and at the office level as a robustness check. This intervention is composed by 7 locations (agencies disbursing Fuel Fund monies), over 10 months (37 weeks): 70 month-office clusters.
Sample size: planned number of observations
We are targeting around 6000 clients. The final number of observations depends the number of turn-off notices that BGE sent in the experimental period. The PIs have no control over this number.
Sample size (or number of clusters) by treatment arms
7 locations (agencies disbursing Fuel Fund monies), over 10 months (37 weeks). The final number of participants depends the number of turn-off notices that BGE sent in the experimental period. The PIs have no control over this number.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Our calculations suggest that we will be able to detect relatively small effects. Using the number of clients who requested assistance in 2013-14 (5,932) and with equal assignment between the three experimental conditions, we are able to detect an impact of the training on monthly energy usage as small as 2.5%. In the unlikely scenario of only 25% compliance to treatment, the minimum detectable effect is 10%, keeping us confident that we will have a large enough sample to establish generalizable results. Early indications suggest that after three months of program, compliance is at 32.9% for the onsite and 38.4% for the online group. These numbers were collected prior to implementing the encouragement letters, making it reasonable to expect compliance to increase in the next weeks. The Fuel Fund of Maryland expected the training to have about a 5-7% effect as suggested by previous, non-experimental evaluations.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Committee for Protection of Human Subjects - University of California, Berkeley
IRB Approval Date
2015-10-19
IRB Approval Number
UCB IRB Registration #IRB00000455 & IRB00005610. UCB FWA: #FWA00006252.
IRB Name
International Review Board - University of Maryland
IRB Approval Date
2014-10-23
IRB Approval Number
Relying Institution FWA: #FWA00005856. eProrotocol #: 2014-05-6328
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials