Energy Demand Flexibility at Scale: Business customers Randomized Encouragement Design

Last registered on February 16, 2024

Pre-Trial

Trial Information

General Information

Title
Energy Demand Flexibility at Scale: Business customers Randomized Encouragement Design
RCT ID
AEARCTR-0013031
Initial registration date
February 13, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 16, 2024, 3:59 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
Centre for Net Zero

Other Primary Investigator(s)

PI Affiliation
Centre for Net Zero
PI Affiliation
University of Southern California

Additional Trial Information

Status
On going
Start date
2023-11-21
End date
2024-09-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
See analysis plan.
External Link(s)

Registration Citation

Citation
Bernard, Louise, Robert Metcalfe and Andrew Schein. 2024. "Energy Demand Flexibility at Scale: Business customers Randomized Encouragement Design." AEA RCT Registry. February 16. https://doi.org/10.1257/rct.13031-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
See analysis plan
Intervention (Hidden)
The intervention is Saving Sessions, a demand reduction program led by our delivery partner, Octopus Energy Limited.

This subtrial, led by the Centre for Net Zero (CNZ), in partnership with Octopus Energy, builds on a larger randomized encouragement design nested in Octopus Energy’s second Saving Sessions (winter 2023 - 2024). In the main trial, we randomized domestic customers into a treatment group that Octopus Energy invited to join Octoplus – a membership that enables customers to participate in flexibility programmes like Saving Sessions – and a control group that did not receive the invitation (see: https://www.socialscienceregistry.org/trials/12451). This main trial focuses on domestic flexibility and does not involve business customers.

In this subtrial, we undertake further research by randomizing business customers into two groups: a main treatment group that will receive the invitation to join Octoplus and a control group that will not receive any invitation. The trial will give us an estimate of the impact of Octoplus sign-up and Session opt-in on business customers’ half-hourly electricity consumption during Saving Sessions. The estimates will come from comparing half-hourly electricity consumption of two groups:
1. A business group with high Octoplus sign-up (the treatment group from this subtrial);
2. A business group with low Octoplus sign-up (the control group from this substrial).

This subtrial will allow us to gain insight on business customers' engagement and ability to provide energy demand reduction as part of Saving Sessions.
Intervention Start Date
2023-11-21
Intervention End Date
2024-09-30

Primary Outcomes

Primary Outcomes (end points)
Sign-Up to Octoplus: We will analyze whether the customers in the treatment and control group have signed up to Octoplus by each Saving Session (exact Session dates and times will be determined during winter 2023-2024 by National Grid ESO and are not yet known).

Electricity Consumption in kWh per half-hour: We will analyze the electricity consumption (kWh per half-hour) of participating customers during Saving Sessions.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Electricity Consumption in kWh per half-hour: We will analyze the electricity consumption (kWh per half-hour) of participating customers in the half-hours before and after Saving Sessions.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
See analysis plan.
Experimental Design Details
This trial builds on a Randomized Encouragement Design to assess the difference between domestic and business customers on sign-up rates and electricity consumption behaviors. A total of 2.7 million eligible participants were initially categorized into two groups: those who were invited to participate in Octopus Energy's new flexibility program, Octoplus, launched in October 2023 (treatment group), and those who were not (control group).

In this subtrial, we randomized 8,964 eligible business customers into a treatment group who received an invitation to join Octoplus on November 21st 2023 and a control group that didn’t.

The subtrial creates two groups for analysis:
1. The treatment group who received the invitation: 7,136 out of 8,964 customers (80% of the sample)
2. The control group that did not receive the invitation email: 1,828 of 8,964 customers (20% of the sample)

As we are particularly interested in spatial heterogeneity, we stratified the sample by Great Britain’s 14 GSP region groups. Contrary to the main trial, we did not stratify by prepayment as it is less relevant for business accounts.

The primary objectives of the experiment are:

To evaluate the effectiveness of the invitation email in increasing the sign-up rate.

To assess the impact of the invitation on electricity consumption reduction among those influenced to sign up by the invitation, and instrument this invitation to understand the impact of program sign-up and event opt-in on consumption

As previously, the sign up process will be via Octoplus (via the Octopus app or website), a new customer reward scheme that Octopus Energy is launching. Octoplus offers customers benefits/rewards within and outside of Saving Sessions. This differs from the sign up process for Saving Sessions 2022-2023, whereby customers received an invitation to participate in the Saving Sessions program only.

We have already conducted the randomization and launched this trial. However, we have not looked at the underlying data between launch in November 2023 and pre-registration today.
Randomization Method
Stratified randomization (see above) using R.
Randomization Unit
Customer. Observations are clustered at the customer level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
8,964 customers
Sample size: planned number of observations
It depends on the number of flexibility events (a figure that is not known in advance). Assuming 20 events of 1 hour in length, there would be 20*2*8,964 = 358,560 customer*half-hours during Saving Sessions.
Sample size (or number of clusters) by treatment arms
The treatment group who received the invitation: 7,136 out of 8,964 customers (80% of the sample)
The control group that did not receive the invitation email: 1,828 of 8,964 customers (20% of the sample)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
0.07 kWh per half-hour (18% of half hourly consumption during events)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials