CrowdFlex: Winter Trials Design Summary

Last registered on February 21, 2025

Pre-Trial

Trial Information

General Information

Title
CrowdFlex: Winter Trials Design Summary
RCT ID
AEARCTR-0015339
Initial registration date
February 13, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 21, 2025, 6:44 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Centre for Net Zero

Other Primary Investigator(s)

PI Affiliation
Centre for Net Zero
PI Affiliation
Centre for Net Zero
PI Affiliation
National Energy System Operator
PI Affiliation
National Energy System Operator
PI Affiliation
National Energy System Operator
PI Affiliation
Ohme Technology
PI Affiliation
OVO Energy
PI Affiliation
OVO Energy
PI Affiliation
Ohme Technology

Additional Trial Information

Status
On going
Start date
2024-08-22
End date
2025-04-27
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
CrowdFlex is an innovation project aimed at exploring how domestic flexibility can be harnessed to support the management of the electricity grid. It involves social science trials to better understand consumer behaviour and how households respond to incentives to adjust their energy use. While previous services have explored domestic flexibility, CrowdFlex rigorously tests domestic flexibility through randomised controlled trials (RCTs), examining variables like incentive structures, notice periods, times of day, and event durations. This rigorous causal identification will help policymakers and other stakeholders to assess the role domestic flexibility can play in supporting the UK’s goal of achieving a net-zero energy system by 2030.

Building on the insights from CrowdFlex’s Summer trials, CrowdFlex will conduct Winter trials from September 2024 through April 2025. In two of the trials, a randomised controlled trial (RCT) methodology will continue to be employed to ensure rigorous evaluation of the outcomes, allowing for robust comparisons between different consumer groups and behaviours. The third will involve a matched control group and randomised treatment groups.

The rewards for participants over the trials will align with payments that could be obtained for the volume of response from stacking of demand turn-up and turn-down markets, services and related capacity payments. The exact reward structure will differ between trial groups to generate evidence on customers’ sensitivity to differing incentive levels and structures. The trial is designed to investigate domestic demand response performance and classify response by key flexibility characteristics. This includes both turn-up and turn-down events. The trial will also consider where specific customer archetypes are more likely than others to display key flexibility characteristics.

These trials are designed to provide behavioural insights and learnings for demand side response service providers, particularly in understanding how customers react to price signals and domestic flexibility. The data collected will also inform decision-making in the Control Room regarding grid management at various times of the day and across different seasons. This information will offer a deeper understanding of system challenges, such as peak demand periods, network constraints, and the potential of distributed domestic assets to provide effective balancing solutions.

The Winter Trials are designed to assess domestic demand response performance by testing both availability and utilisation payments. The study will categorise responses according to key flexibility characteristics, including both turn-up and turn-down events. The trial will also examine whether certain customer archetypes are more likely than others to demonstrate particular flexibility characteristics.

Different trial groups will be created to test:
* The impact of availability payments on availability and response performance
* The impact of utilisation price on response performance
* The impact of additional rewards, such as consistency rewards, on response performance

The Winter Trials will also continue to serve the purpose of collecting data on domestic demand and flexible response to further inform and improve Demand Side Response Service Providers (DSRSPs)’ and NESO’s domestic demand and flexibility models respectively.

Two types of price structures are to be tested in the winter trials:
* Utilisation payments – payments to households per kWh for flexibility (turn-down or turn-up compared to an industry-agreed calculation of the customer’s counterfactual consumption (their ‘baseline’)) during flexibility events.
* Availability payments – payments to households with Electric Vehicles (EVs) and home chargers that allow third parties to operate them.

OVO Energy will conduct a trial of each type, while Ohme will focus on an availability payments trial only.
External Link(s)

Registration Citation

Citation
Atherton, Sanna et al. 2025. "CrowdFlex: Winter Trials Design Summary." AEA RCT Registry. February 21. https://doi.org/10.1257/rct.15339-1.0
Sponsors & Partners

Sponsors

Partner

Type
private_company
Type
private_company
Type
private_company
Experimental Details

Interventions

Intervention(s)
Customers will receive different incentive levels / structures.
Intervention Start Date
2024-09-16
Intervention End Date
2025-04-27

Primary Outcomes

Primary Outcomes (end points)
Availability Payments Trial: Weekly plug-ins
Utilisation Payments Trial: kWh consumption
Primary Outcomes (explanation)
Availability Payments Trial: Primary outcome: weekly plug-ins by trial group.
Utilisation Payments Trial: Primary outcome: half-hourly electricity consumption (kWh) during event windows.

Secondary Outcomes

Secondary Outcomes (end points)
Availability Payments Trial: kWh consumption during event windows
Utilisation Payments Trial: kWh consumption outside event windows
Secondary Outcomes (explanation)
Availability Payments Trial: kWh – used in analyses to investigate the extent to which utilisation-like performance (kWh turn-down and turn-up) also varies by trial group – i.e., the extent to which higher plug-in time does, in fact, deliver more genuinely useful availability.

Utilisation Payments Trial: Assess spillover effects of treatments on electricity consumption outside event windows, measuring half-hourly consumption in the 24 hours before and after the events. Spillovers may be positive (e.g., habit formation) or negative (e.g., shifting consumption).

Experimental Design

Experimental Design
Parallel-group RCT
Experimental Design Details
Not available
Randomization Method
Randomization done in office by a computer
Randomization Unit
Customer
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
OVO availability payments trial: 7.5k customers
Ohme availability payments trial: 12.5k customers
OVO utilisation payments trial: 55k customers
Sample size: planned number of observations
Analysis is clustered at the customer level. We have hundreds of repeated observations for each customer: * Observations are at the half-hourly level * Utilisation Payments Trial: approximately 35 turn-down and 35 turn-up events * Availability Payments Trial: over 100 turn-down and over 100 turn-up events * Events are 1-4 hours
Sample size (or number of clusters) by treatment arms
See # clusters.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
See analysis plan. Utilisation Payments Trial: 0.0057 kWh, or approximately 3% of typical control group consumption per half-hour. Availability Payments Trial: A change in plug-ins per week of 5-7% of the control group average.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information