Energy Demand Flexibility at Scale: Sign Up Chaser Emails

See analysis plan.

2024-01-17

2024-09-30

Sign-Up to Octoplus
Electricity Consumption in kWh per half-hour

Sign-Up to Octoplus: We will analyze whether the customers in the hold out group (invitation but no chaser) and chaser group (invitation and chaser email) have signed up to Octoplus by each Saving Session (exact Session dates and times will be determined during winter 2023-2024 by National Grid ESO and are not yet known).
Electricity Consumption in kWh per half-hour: We will analyze the electricity consumption (kWh per half-hour) of participating customers during and around Saving Sessions in the hold out group (invitation but no chaser) and chaser group (invitation and chaser email).

See analysis plan.

Not available

Simple randomisation (using R).

Customer

Yes

680k customers received the chaser (treatment group).
70k customers did not (control group)

There are approximately 10-20 Saving Sessions we expect to happen between the sending of the chaser, on 17 January 2024, and the end of the 2023-2024 Saving Sessions, which will occur in March or April 2024. The exact number (and length of each event) depends on the needs of Great Britain's Electricity System Operator. Assuming 10 events of 1 hour in length, there would be 10*2*680k = 13,600,000 customer*half-hours during Saving Sessions in the treatment (chased) group; and 10*2*70k = 1,400,000 customer*half-hours during Saving Sessions in the control group.

See above regarding Planned Number of Clusters and Planned Number of Observations.

We use a random sample of 8,298 Octopus Energy smart meter customers to produce the standard errors on the impact of the variable “Saving Session” of 10 “placebo” Saving Sessions. We then use the rule of 2.8 to estimate the Minimum Detectable Effect Size by multiplying the standard errors by 2.8, giving us an estimate of what effect size we could detect for a 5% significance level and 80% power. We then predict the standard errors for a larger sample where each group has 70,000 customers (note that this is slightly conservative, as one of our groups has much more than 70,000 customers). We find a MDES for the ITT of 0.003 kWh. This is ~ 0.7% of half hourly consumption for our sample at peak hour (5pm to 6pm) in winter 2022-2023. We expect an average treatment effect of 0.1 kWh for sign-up through the season based on Jacob et al (2023) Table AT.4. We also expect the chaser group to increase signup by 10 percentage points. Assuming a 10 percentage point higher sign-up to Octoplus between groups, we have an expected treatment effect of 0.1 (additional sign-up from the chaser) * 0.1 (kWh per signup) = 0.01 kWh ITT effect. In this case, we are powered to detect the ITT effect, as 0.01 > MDES of 0.003 kWh. We do the same exercise for the LATE analysis: we do a placebo 2SLS regressions of a placebo invitation on sign-up. We obtain a MDES of 0.014 kWh (about 4% of half hourly consumption). In this case, we are not powered to 80% to detect this LATE, as 0.014 kWh is greater than the 0.1 kWh per half-hour effect we expect from signing up to Octoplus.