Smart Thermostats, Automation, and Time-Varying Pricing

Randomized encouragement into suite of smart thermostat features, including a feature that automates responsiveness to time-of-use prices.

2019-08-05

2019-12-31

HVAC compressor run-time and discomfort.

HVAC compressor run-time is reported in the thermostat data.
Discomfort will be constructed as the difference between experienced indoor temperature and preferred indoor temperature multiplied by the number of minutes experienced.
Both outcomes will be measured at the thermostat-by-hour level.

Randomized encouragement into eco+ (a suite of smart thermostat features) that includes a feature that automates responsiveness to time-of-use pricing. We are conducting the randomization in June 2019 and Ecobee will begin the full rollout of the eco+ product to all of their customers in August 2019. During this rollout, Ecobee will use email, push notifications on cell phones, and the thermostat itself to encourage customers to turn on the eco+ product. We randomly selected 1,500 customers as a control group who will not receive encouragement to enroll in eco+ out of the 3,945 customers who are eligible for analysis in our experimental sample. Control group customers are able to sign up for eco+ if they find the feature on their own, but they will not be directly encouraged by Ecobee to sign up for at least 6 months after the initial rollout (e.g., until January 2020). Eligibility for inclusion in our experiment is restricted to customers that: (a) reside in Ontario, Canada; (b) were enrolled in the Donate Your Data (DYD) program in June 2019; (c) were not exposed to any eco+ pilot programs; (d) have only one thermostat per user account; (e) do not have multistage cooling systems; and (f) have had an Ecobee thermostat for at least 12 months.

Randomization performed by Stata on a computer in Casey Wichman's former office at RFF in Washington, DC, with a replicable seed (8675309).

Unit of randomization is the thermostat (or household containing a thermostat).

No

3,945 households.

5-minute interval observations for each of the experimental households for at least two years (including 6-months of post-treatment data). Data will be aggregated to the hourly level.

1500 households in not-encouraged group. 2445 households in encouraged group. Anticipated 50-80 compliance with encouragement.

We assume 1500 control households and calculate minimum detectable effect (MDE) sizes for simple pre-post mean comparisons for compressor run-time (our primary outcome variable) for each hour of the day. Our control mean and standard deviation estimates come from all Ontario households present in the DYD database in June 2017. These data are our best approximation of pre-treatment means, as we have not yet analyzed household data for June 2018 or June 2019. For all calculations, we assume a power level of 0.8 and statistical significance of 0.05. MDE sizes for assumptions of 80% compliance with treatment range from 0.8 minutes per hour at 8AM to 1.93 minutes per hour at 7PM. These MDEs represent changes of approximately 0.095 standard deviations. For 50% compliance, results are similar, with MDEs ranging from 0.72 minutes per hour between 8 and 9AM to 1.72 minutes per hour at 7PM. For this more conservative estimate of compliance with treatment, MDEs represent changes of approximately 0.108 standard deviations for all hours of the day. Thus, our study design will allow us to measure small effects of treatment on hourly compressor usage. In our analysis, we will control for additional covariates, which will further increase the precision of our treatment effects, in addition to leveraging the rich panel nature of the DYD data set.