A residential electricity demand response field experiment

Last registered on February 24, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
A residential electricity demand response field experiment
RCT ID
AEARCTR-0008680
Initial registration date
February 19, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 24, 2022, 1:01 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Canada
Region

Primary Investigator

Name
Megan Bailey
Affiliation
University of Calgary
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
David Brown
PI Affiliation
University of Alberta
Contact Investigator
PI Name
Blake Shaffer
PI Affiliation
University of Calgary
Contact Investigator
PI Name
Frank Wolak
PI Affiliation
University of Calgary/Stanford
Contact Investigator

Additional Trial Information

Status
In development
Start date
2022-02-20
End date
2023-12-31
Keywords
Environment & Energy
Additional Keywords
electricity, demand response, residential, technology adoption
JEL code(s)
Q41, Q48, L94
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Electricity retailers must ensure that system demand is less than or equal to available generation capacity at all times. One way for the sector to achieve this outcome is through demand-side management actions that incentivize or require consumers to reduce electricity demand during peak times. In partnership with a large electricity retailer, we randomly offered residential electricity customers one of several treatments composed of combinations of smart home electricity management equipment and financial incentives for reducing their demand.. About 1100 households were ultimately enrolled in the experiment across all of the treatments. Our objectives are to evaluate: (1) the relative take-up rates and characteristics of consumers that accept offers for each treatment based on take-up incentives and customer demographic characteristics; (2) the average treatment effect on the treated in terms of (a) electricity demand reduced during peak load hours and (b) the reliability of the amount of electricity demand reduced, for those who accept; and (3) intent-to-treat estimates for each offer as a program, using (a) and (b). We also plan to estimate heterogenous treatment effects along a number of household characteristics from each intervention.
External Link(s)

Registration Citation

Citation
Bailey, Megan et al. 2022. "A residential electricity demand response field experiment." AEA RCT Registry. February 24. https://doi.org/10.1257/rct.8680-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
The intervention is composed of event days where customers notified that they will be compensated for demand reductions relative to a baseline level of demand during pre-selected periods of the day. These event days are randomly assigned across days of the month and customers. Event days are declared an average to three times per month for each participant in the experiment.
Intervention Start Date
2022-02-20
Intervention End Date
2023-06-30

Primary Outcomes

Primary Outcomes (end points)
(1) the relative take-up rates and characteristics of consumers that accept offers for each treatment group, according to take-up incentives; (2) the average treatment effect on the treated in terms of (a) electricity reduced during peak-load hours and (b) the reliability of electricity reduced for those who accept; (3) intent-to-treat estimates for each offer as a program, using (a) and (b); and (4) heterogenous treatment effects along a number of household characteristics.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We randomly offer one of six treatment or control offers to residential electricity customers in our partner’s service area. We followed a stratified random assignment approach in which we first used a machine learning algorithm (k-means) to identify clusters of households based on answers to survey questions and geographically matched Census data. We then randomly assigned households from clusters across treatment groups to obtain balance along observables with random assignment. Households were offered randomly assigned up-front incentives to participate in addition to the benefits of the treatment/control group to which they were assigned.

Treatments consist of combinations of smart home electricity use management equipment and incentives for reductions in peak demand time electricity usage. To estimate average treatment effects, we plan to use a selection model to account for customers’ selection into treatment offers.
Experimental Design Details
We hypothesize that large reductions in peak electricity demand are possible if a utility coordinates/conducts demand reductions on consumers’ behalf by manually switching off or turning down usage from appliances. However, we expect that a small share of customers will find remote control of appliances acceptable. Therefore, we are interested in comparing customers’ take-up rates and ultimate demand response in two programs: (1) a program that includes direct load control (DLC) equipment that the utility can use to reduce appliances’ electricity use during peak hours and incentives for allowing the utility to do so and (2) a program that offers the same equipment and incentives, but without any remote control of appliances by the utility. In our experiment, Groups A and B (described above), respectively, are these programs. To understand the role that such equipment (which the customer can also use to remotely control appliances) plays in demand response above and beyond incentives, we include Group C which does not receive any equipment, but receives the events notifications and incentives.

Because the equipment necessarily involves giving people real-time information about electricity usage, we include Group E as well, which received real-time information but no incentives. By comparing Groups C and E, we will understand how much incentives, on their own, work to alter behavior, controlling for the ability to observe real-time information. Finally, to further understand customers’ preferences about remote appliance control, we randomly offered a subset of households the ability to choose between the programs offered to Groups A, B, and C. This “Group D”, therefore, was offered a demand response incentive program as well as the option to accept DLC equipment and remote control. We elicited a full preference ranking from these households to obtain a complete picture of preferences and to ensure that subsequent offers for non-top-ranked programs were consistent with preferences.

Customers were recruited from a pool that had downloaded a mobile phone app to help manage their electricity usage. We randomly offered treatment offers to customers who indicated that they had appliances for which we could offer DLC equipment and who resided in areas our utility partner could offer installation. We stratified randomization according to several household-level and census dissemination area-level variables (including historic consumption) to ensure balance in observables across treatment and control groups.
Randomization Method
We randomized households using a STATA script.
Randomization Unit
Household
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1075 households
Sample size: planned number of observations
1075 households
Sample size (or number of clusters) by treatment arms
740 across 3 treatment groups, 335 across 2 control groups
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Alberta Research Ethics Board
IRB Approval Date
2021-05-28
IRB Approval Number
Pro00110333
IRB Name
University of Calgary Conjoint Faculties Research Ethics Board
IRB Approval Date
2021-06-17
IRB Approval Number
REB21-0161
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials