Energy Demand Flexibility at Scale: Late Invitations

See analysis plan.

2024-02-26

2024-09-30

Sign-Up to Octoplus
Electricity Consumption in kWh per half-hour

Sign-Up to Octoplus: We will analyze whether the customer has signed up to Octoplus by each Saving Session (exact Session dates and times will be determined during winter 2023-2024 by National Grid ESO and are not yet known). We will compare sign-up rates between the main treatment group, the late invitation group and the rest of the control group.
Electricity Consumption in kWh per half-hour: We will analyze the electricity consumption (kWh per half-hour) of participating customers during and around Saving Sessions.

See analysis plan.

Not available

Simple randomisation (using R).

Customer

Yes

There are approximately 5-10 Saving Sessions we expect to happen between the sending of the late invitation, on or just after 21 February, and the end of the 2023-2024 Saving Sessions, which will occur in March or April 2024. The exact number (and length of each event) depends on the needs of Great Britain's Electricity System Operator.

Assuming 5 events of 1 hour in length, there would be
1. Control group: 79,999 * 2 * 5 = 799,990 customer*half-hours during Saving Sessions
2. Late invitation group: 40,000 * 2 * 5 = 400,000 customer*half-hours during Saving Sessions
3. Original treatment group: 2,571,287 * 2 * 5 = 25,712,870 customer*half-hours during Saving Sessions

As stated above: 1. Control group: 79,999 of the 119,999 customers in the original control group that did not receive an invitation to join Octoplus and will stay uncontacted. 2. Late invitation group: 40,000 customers of the 119,999 customers in the original control group to receive a late invitation. The goal is to estimate the impact of the first event and subsequent events while removing time specific effects. 3. Original treatment group: the original 2,571,287 customers that received the first invitation to join Octoplus.

See above regarding Planned Number of Clusters and Planned Number of Observations.

We use a random sample of 8,298 Octopus Energy smart meter customers to produce the standard errors on the impact of the variable “Saving Session” of 10 “placebo” Saving Sessions. We then use the rule of 2.8 to estimate the Minimum Detectable Effect Size by multiplying the standard errors by 2.8, giving us an estimate of what effect size we could detect for a 5% significance level and 80% power. We then predict the standard errors for a larger sample where each group has 40,000 customers (note that this is slightly conservative, as one of our groups has much more than 40,000 customers). ITT For a late invitation group of 40,000 customers, we find a MDES of 0.003 kWh ~ 0.1% of half hourly consumption for our sample during the winter evening peak (5pm to 6pm in winter 2022-2023). We expect an average invitation treatment effect of 0.1 kWh per half-hour lower consumption during Saving Sessions among customers signed up to Octoplus based on values from Jacob et al (2023) Table AT.4. This average is based on what the authors note appears to be a slight decay in the ITT effect over time. We expect the late invitation group to reach similar levels of sign-up to Octoplus as the original treatment group; in other words, we expect both to have similar “compliance” with the encouragement to sign up to Octoplus. Assuming a 0.07 kWh difference in demand between those signed up in the original treatment group versus those signed up in the late invitation group (decay effect) and a difference of 15 percentage points in signup to Octoplus due to the late invitation (compliance) compared to signup to Octoplus in the control group (~30% vs ~45%), we have an expected ITT of 0.15 * 0.07 = 0.0105 kWh. For the comparison between the late invited and the rest of the control group, we expect a similar treatment effect of on average 0.1 kWh (invitation effect) and a difference of 15 percentage points in signup due to the late invitation (compliance); thus an expected ITT of 0.015 kWh. We should be well powered for both comparisons.