Smart Thermostats, Automation, and Time-Varying Pricing

Last registered on August 02, 2021

Pre-Trial

Trial Information

General Information

Title
Smart Thermostats, Automation, and Time-Varying Pricing
RCT ID
AEARCTR-0008019
Initial registration date
July 29, 2021
Last updated
August 02, 2021, 12:48 PM EDT

Locations

Region

Primary Investigator

Affiliation
Georgia Institute of Technology

Other Primary Investigator(s)

PI Affiliation
Federal Reserve Board
PI Affiliation
Resources for the Future
PI Affiliation
University of Maryland

Additional Trial Information

Status
Completed
Start date
2018-07-01
End date
2020-01-31
Secondary IDs
Abstract
We evaluate an experiment in which randomly encouraged households activate a smart-thermostat feature that automates responsiveness to time-of-use electricity pricing. The thermostat feature reduces electricity use by raising indoor temperatures, thus increasing thermal discomfort in some households during peak periods. Changes in discomfort are small, concentrated among households who spend the most time at home, and do not prompt them to adjust the feature’s intensity or deactivate it. Using energy cost savings and experienced indoor temperatures, we calculate households’ revealed preference trade-off between comfort and cooling expenditure and find that households are willing to trade off small monetary savings for small increases in discomfort. Automation thus provides a low-cost opportunity to make small changes in energy demand at the household level, with potentially large electricity supply-cost reductions at scale.
External Link(s)

Registration Citation

Citation
Blonz, Joshua et al. 2021. "Smart Thermostats, Automation, and Time-Varying Pricing." AEA RCT Registry. August 02. https://doi.org/10.1257/rct.8019-1.0
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
Randomized encouragement into suite of smart thermostat features, including a feature that automates responsiveness to time-of-use prices.
Intervention Start Date
2019-08-05
Intervention End Date
2019-12-31

Primary Outcomes

Primary Outcomes (end points)
HVAC compressor run-time and discomfort.
Primary Outcomes (explanation)
HVAC compressor run-time is reported in the thermostat data.
Discomfort will be constructed as the difference between experienced indoor temperature and preferred indoor temperature multiplied by the number of minutes experienced.
Both outcomes will be measured at the thermostat-by-hour level.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Randomized encouragement into eco+ (a suite of smart thermostat features) that includes a feature that automates responsiveness to time-of-use pricing. We are conducting the randomization in June 2019 and Ecobee will begin the full rollout of the eco+ product to all of their customers in August 2019. During this rollout, Ecobee will use email, push notifications on cell phones, and the thermostat itself to encourage customers to turn on the eco+ product. We randomly selected 1,500 customers as a control group who will not receive encouragement to enroll in eco+ out of the 3,945 customers who are eligible for analysis in our experimental sample. Control group customers are able to sign up for eco+ if they find the feature on their own, but they will not be directly encouraged by Ecobee to sign up for at least 6 months after the initial rollout (e.g., until January 2020). Eligibility for inclusion in our experiment is restricted to customers that: (a) reside in Ontario, Canada; (b) were enrolled in the Donate Your Data (DYD) program in June 2019; (c) were not exposed to any eco+ pilot programs; (d) have only one thermostat per user account; (e) do not have multistage cooling systems; and (f) have had an Ecobee thermostat for at least 12 months.
Experimental Design Details
Randomization Method
Randomization performed by Stata on a computer in Casey Wichman's former office at RFF in Washington, DC, with a replicable seed (8675309).
Randomization Unit
Unit of randomization is the thermostat (or household containing a thermostat).
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
3,945 households.
Sample size: planned number of observations
5-minute interval observations for each of the experimental households for at least two years (including 6-months of post-treatment data). Data will be aggregated to the hourly level.
Sample size (or number of clusters) by treatment arms
1500 households in not-encouraged group. 2445 households in encouraged group. Anticipated 50-80 compliance with encouragement.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We assume 1500 control households and calculate minimum detectable effect (MDE) sizes for simple pre-post mean comparisons for compressor run-time (our primary outcome variable) for each hour of the day. Our control mean and standard deviation estimates come from all Ontario households present in the DYD database in June 2017. These data are our best approximation of pre-treatment means, as we have not yet analyzed household data for June 2018 or June 2019. For all calculations, we assume a power level of 0.8 and statistical significance of 0.05. MDE sizes for assumptions of 80% compliance with treatment range from 0.8 minutes per hour at 8AM to 1.93 minutes per hour at 7PM. These MDEs represent changes of approximately 0.095 standard deviations. For 50% compliance, results are similar, with MDEs ranging from 0.72 minutes per hour between 8 and 9AM to 1.72 minutes per hour at 7PM. For this more conservative estimate of compliance with treatment, MDEs represent changes of approximately 0.108 standard deviations for all hours of the day. Thus, our study design will allow us to measure small effects of treatment on hourly compressor usage. In our analysis, we will control for additional covariates, which will further increase the precision of our treatment effects, in addition to leveraging the rich panel nature of the DYD data set.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials