Energy Demand Flexibility at Scale: a mini-RCT involving a higher incentive group of customers

See analysis plan.

The treatment group email text was:
Octoplus Saving Session alert: use less electricity TODAY from 6:00 PM to 7:00 PM on 8th February. You'll earn 1800 Octopoints - that's £2.25 worth - per kilowatt hour for any power you save compared to normal.
Lucky you: most customers are getting 1400 Octopoints per unit in this Session, but you're getting a special higher reward rate this time. You'll be rewarded at the standard 1400 Octopoints rate online, but we'll add bonus Octopoints to your account to take your reward up to your higher 1800 Octopoint rate.

In contrast, the control group email text began:
Octoplus Saving Session alert: use less electricity TODAY from 6:00 PM to 7:00 PM on 8th February. You'll earn 1400 Octopoints - that's £1.75 worth - per kilowatt hour for any power you save compared to normal.

For both groups, the emails then continued with explanatory text about Octoplus, its “perks”, and Saving Sessions more specifically:
Opt in now and go in the draw to win a pair of Odeon Cinema tickets – we're giving away 300 pairs to kick off your exclusive new Octoplus perks (read more below). On a Saving Session streak? You get 8 extra chances to win and a 100 Octopoint bonus.
Opt in in two clicks
Even more reason to earn Octopoints: introducing some shiny new perks for Octoplus customers:
Claim a free hot drink from Greggs every week
Get great deals on Odeon Cinema tickets – right now, they're 2 for £10!
Spend Octopoints on a super-discounted day out with National Trust.
Head to Octoplus in our app to see more.

2024-02-08

2024-09-30

Opt-In: We will analyze if the customers that received the extra incentive are more likely to opt-in to the Session on 08 February 2024.
We will also examine if these customers are more likely to opt in to subsequent Sessions.

Electricity Consumption in kWh per half-hour: We will analyze the electricity consumption (kWh per half-hour) of participating customers during the 08 February 2024 Saving Session in the control and treatment group (normal incentive, and extra incentive).

We will also examine if these have different opt-in and consumption in subsequent Sessions.

See analysis plan.

This subtrial, led by the Centre for Net Zero (CNZ), in partnership with Octopus Energy, builds on a larger randomized encouragement design nested in Octopus Energy’s second Saving Sessions (winter 2023 - 2024). In the main trial, we randomized domestic customers into a treatment group that Octopus Energy invited to join Octoplus – a membership that enables customers to participate in flexibility programmes like Saving Sessions – and a control group that did not receive the invitation (see: https://www.socialscienceregistry.org/trials/12451). This main trial focuses on domestic flexibility and does not involve business customers.

In this subtrial, we randomize a group of 5,000 domestic customers to receive a higher incentive than a control group, for a specific Session that occurred on 08 February 2024.

The trial will give us an estimate of the impact of the extra incentive on opt-in and half hourly consumption during the Saving Session on 08 February 2024, and in subsequent Sessions.

The subtrial creates two groups for analysis:
1. The treatment group who received the higher incentive of £2.25/kWh: 5,000 out of 591,643 domestic accounts who were signed up to Octoplus and were likely to receive the opt-in notice as an email rather than a push notification on their app.
2. The control group that received the normal incentive for that Saving Session of £1.75/kWh: 591,643 domestic accounts that have signed up for Octoplus and were likely to receive the opt-in notice as an email rather than a push notification on their app.
The reason that the sample is domestic customers who “were likely to receive the opt-in notice as an email rather than a push notification on their app” is that only the email noted the higher incentive level.

It was possible that the sample selection was imperfect, in that some customers in both groups could have received push notifications. In these cases, the push notification did not differ between the two groups. Push notifications involve many fewer characters, and there was not enough space to specify the incentive level in these push notifications. We have no reason to believe that the rate of push notifications versus email notifications would differ between the treatment and control group. If the data on push notification receipt is available to us, we will conduct exploratory analyses where we restrict our sample (in both treatment and control) to customers who received the opt-in email only and did not receive the opt-in push notification.

The opt-in notice – whether email or push notification – was sent during the afternoon of 08 February 2024 for a Session 18:00 to 19:00 on the same day.

We used simple randomisation – we did not stratify randomisation on any variables.

We have already conducted the randomization and launched this trial. However, we have not looked at the underlying data between launch in November 2023 and pre-registration today.

We used simple randomisation – we did not stratify randomisation on any variables. We used R to do so.

Customer

Yes

The treatment group contains 5,000 domestic customers.
The control group contains 591,643 domestic customers.

There are 2 half-hours in the Saving Session. Thus there are 2*(5,000+591,643) = 1,193,286 customer*half-hour observations in the primary analysis of consumption in the Session on 08 February 2024. There are more clusters for the secondary analysis of consumption in subsequent Sessions. The exact number is unknown because it will depend on how many flexibility events Great Britain's Electricity System Operator calls during the rest of 2024.

See above.
The treatment group contains 5,000 domestic customers.
The control group contains 591,643 domestic customers.

The MDES is approximately 0.014476 kWh per half-hour. Based on an average half-hourly electricity consumption of 0.38 kWh, this is a change of 3.8%.