CrowdFlex: Beta Utilisation Payments Mini-Trial 1

See pre-analysis plan.

2024-05-07

2024-07-12

Electricity consumption during the event window (kWh per half-hour)

Our primary outcome of interest is customers’ electricity consumption during the event window. This will primarily be measured using customers’ smart meters, which record electricity consumption in 30-minute intervals, though asset level data may also be utilised where available.

Electricity consumption 24 hrs before/after the event window (event window specified in the event timetable)

Our secondary outcome of interest is customers’ electricity consumption outside the event window (24 hours before and after the event). This is to understand whether customers shift their energy consumption to half hours outside the event window versus whether the energy demand is created or destroyed. This will also be measured using smart meters and asset data where available.

See pre-analysis plan

As discussed above, we recommend a wait-list randomisation to create a pure control group, with the rest of the customers randomly allocated into one of 5 groups.
From a marketing and comms perspective, if customers are signed-up into the CrowdFlex programme, they would expect to take part in at least some events. Thus, we have recommended inviting people to sign up to CrowdFlex but note in the invitation that the first set of events may be oversubscribed, and a waitlist will form. Customers in the control group may have the possibility of joining later trials. Note that the DSRSP does not have to use the terminology of a “waitlist”.
We suggest equal sized groups of at least 9,600 customers in each group. The waitlisted control group may be larger than the other groups if a DSRSP’s recruitment resulted in greater than 48,000 signups.
What is the unit of randomisation? Household
What is the unit of analysis? Household * half-hour combination
Will we stratify randomisation? We suggest using only one variable for stratification: GSP Group (14 levels).
As discussed above, DSRSPs should aim to have at least 600 participants per GSP group to ensure that the results are representative at the GSP group level.

Randomization done in office by a computer

Household

Yes

48,000 households

Given that there are 31 events comprising 107 half-hours, there will be 107 * 48,000 = 5,136,000 household*half-hours. However, note that there are 18 turn-down events (comprising 65 half-hours) and 11 turn-up events (comprising 37 half-hours). There are also 2 events (comprising 5 half-hours) where two out of 13 regions turn-up and 11 out of 13 events turn-down. Turn-down and turn-up will always be analysed separately.

9,600 households (clusters) per treatment arm
With 107 half-hours in events, that entails 1,027,200 household*half-hours per treatment arm.

See pre-analysis plan for details. We expect an MDES of 0.03 kWh (<1% of baseline consumption) for the ~20 turn-down events where we have 9,600 customers per trial arm. We expect and MDES of 0.035 (just over 1%) for the ~10 turn-up events.