An Experimental Analysis of Adoption, Savings and Information Diffusion for Industrial Energy Management Systems
Last registered on August 23, 2018

Pre-Trial

Trial Information
General Information
Title
An Experimental Analysis of Adoption, Savings and Information Diffusion for Industrial Energy Management Systems
RCT ID
AEARCTR-0003227
Initial registration date
August 15, 2018
Last updated
August 23, 2018 7:01 PM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
UC Berkeley
Other Primary Investigator(s)
PI Affiliation
University of Chicago
PI Affiliation
MIT
Additional Trial Information
Status
On going
Start date
2016-09-16
End date
2020-12-31
Secondary IDs
Abstract
This study will examine whether there is under-investment in diffusion of new technologies due to information spillovers. We will explore potential market failure attributable to within-industry spillovers of unilateral marketing and education activities. If technology suppliers cannot internalize all of the producer surplus from marketing and education, government subsidization of these activities could lead to more efficient diffusion rates.

In particular, we will analyze information spillovers for Lightapp’s industrial energy efficiency technology for air compressors. We will first randomize marketing and subsidies for the technology across industrial facilities in Pacific Gas and Electric and Southern California Edison service territories. We will then use a stepped wedge randomized design to estimate the software’s energy and bill impacts. After removing references to Lightapp, we will share these results with a random selection of previously uncontacted firms. We will elicit willingness to pay for the Lightapp software from these firms and from a set of control firms that receive no information. This will enable us to compare the effects of education about one energy efficiency product on willingness to pay for a similar energy efficiency product, potentially offered by a competitor.

External Link(s)
Registration Citation
Citation
Greenstone, Michael, Christopher Knittel and Catherine Wolfram. 2018. "An Experimental Analysis of Adoption, Savings and Information Diffusion for Industrial Energy Management Systems." AEA RCT Registry. August 23. https://www.socialscienceregistry.org/trials/3227/history/33427
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2017-03-12
Intervention End Date
2019-11-30
Primary Outcomes
Primary Outcomes (end points)
Take-up of free and subsidized energy management system, energy savings from energy management system, information spillovers to subsequent customers.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We are using a stepped-wedge design with randomized recruitment and treatment. Details are below.

Step 1: Sampling and recruitment

Excluding the pilot sites, the sampling frame consists of 1,124 facilities. After establishing this sampling frame, we randomly assigned facilities into two groups:
ˆ FREE group: 70 percent of facilities were o ered free Lightapp services
ˆ DISCOUNT group: 30 percent of facilities were o ered a 25 percent discount for Lightapp services
For both FREE and DISCOUNT groups, we fully subsidize the one-time installation of Lightapp
software devices and any subsequent maintenance or repair costs incurred during the duration of the
project.
We also cross-randomized the timing in which Lightapp's sales team will contact each facility to o ffer
its services. We randomly sorted the facilities in our sampling frame into an ordered list, and divided
facilities into 50 sequential tranches according to this randomized ordering. Each week, we released an
additional tranche of facilities to Lightapp's sales team. Information included company names, addresses,
and contact information. The release date initiated a rolling 30- or 40-day recruitment window, during
which Lightapp was encouraged to contact potential program participants and convert them into Lightapp
customers. After this tranche-speci fic recruitment window closed, Lightapp agreed not to recruit any
facility in the tranche that had not signed a contract for the remainder of the experiment.
We made three adjustments to the experimental design near the beginning of the recruitment phase.
We expanded the recruitment window from 30 days to 40 days after discovering that this change would
give more facilities sufficient time to gain internal approval and sign contracts with Lightapp. We also
increased the tranche size from 12 to 14 facilities due to the unexpectedly large number of facilities
Lightapp deemed ineligible. We maintained the same randomization order for determining the facilities in
each tranche. At the same time, we removed water and sewage treatment facilities from the experiment.
These facilities initially comprised approximately 20 percent of our sampling frame, yet they were very unlikely to have compressed air onsite. When we removed water
and sewage treatment facilities, we preserved the randomization order among all other facilities. We
had released eight tranches of facility names to Lightapp when we introduced these three experimental
design changes. We will perform sensitivity analysis on key results excluding facilities recruited prior to
instituting these changes.
Following the approach by Hussey and Hughes (2006) for power calculations in stepped wedge designs,
we pre-specifi ed a target sample size of 100 Lightapp participants. Due to funding constraints, we made
this target a ceiling, so recruitment would end after 100 facilities signed participation contracts. Since
Lightapp successfully recruited 100 customers before we released all facilities in the sampling frame, we
hit this ceiling and stopped recruitment. At this time, there were 117 uncontacted facilities remaining.

Step 2: Installation and Validation
After the end of the recruitment window, Lightapp and the other installers have 80 days to install the
EMS software in the new participating facilities. This process requires connecting the Lightapp system
to air compressors. A group of installation subcontractors perform these installations. If the lead installer, Ingersoll
Rand, deems an installation more expensive than authorized in their contract, Lightapp or another
subcontractor, Compressor IQ, performs the installation. At the time of registering this pre-analysis
plan, two individuals from Ingersoll Rand had overseen every installation, ensuring that installations
were performed correctly and consistently.
After installation is complete, Lightapp validates the data by comparing readings from the newly
installed meters to the PG&E or SCE electricity meter readings. Lightapp explores and addresses any
data discrepancies.

Step 3: Baseline data collection
Following meter installation and validation, each facility in the project enters into a baseline data
collection period of no less than 90 days. During this time, no one at the participating facility has
access to the Lightapp software platform or receives any performance reports or alerts. This establishes
a "business as usual" compressed air energy usage baseline for each facility. Note that we will test for
the presence of Hawthorne eff ects.

To ensure that the facilities begin the treatment phase (Step 4) in the established randomization
order, we occasionally extend baseline periods for some facilities. If one facility experiences a delay in
treatment period commencement, Lightapp withholds data from all facilities later in the randomization
order. We plan to look for the existence of heterogeneous impacts with respect
to baseline period length.

Step 4: Treatment period data collection
After this baseline period is over, facilities begin receiving real-time Lightapp data. This treatment
period lasts 12 months. During this time, facility personnel have access to all EMS data, scheduled reports,
and noti cations with tips and recommended actions for improving compressor eciency. Noti cations
include performance summaries and alerts for energy use data anomalies that cannot be explained by
changes in production. Participants can also compare their Compressed Air System Efficiency (CAS)
scores, which reflect the energy required to produce 100 cubic feet per minute (CFM), to other sites in
the project. All data are anonymized, but users can sort based on the general industry. Companies with
multiple plants that have Lightapp access can directly compare detailed data across these sites. We limit
the number of facilities that can begin the treatment period to three facilities per week.
Experimental Design Details
Not available
Randomization Method
Random number generator on a computer.
Randomization Unit
Establishment.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
NA
Sample size: planned number of observations
100 establishments
Sample size (or number of clusters) by treatment arms
We randomized the timing of treatment, as described above.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Committee for Protection of Human Subjects, UC Berkeley
IRB Approval Date
2015-08-20
IRB Approval Number
2015-08-7838