Addressing Hassle Costs: Man versus Machine

Our intervention consists of providing an Energy Coach program to reduce the hassle costs associated with heat pump adoption. The program has two versions: one consists of a human-assisted program, and the other consists of a digital assistant.

2023-08-07

2023-11-30

The primary outcome is the compensation amount households require to be paid or are willing to pay, respectively the WTA or WTP for the Energy Coach program. We elicit those values with an auction procedure in the survey.

Using the WTA/WTP values, we will first perform a reduced-form analysis to assess whether the treatments impact those values. The treatments are the two versions of the energy coach program and the front-end delay in implementing the energy coach program.

The WTA/WTP values are also used to compute the Marginal Hassle Cost for each household. The Marginal Hassle Cost is the ratio of the difference in the compensation amounts for the same energy coach program but for different durations, divided by the difference in the expected duration of the program.

As part of the analysis, we are also planning to compute the Value of Information associated with the Energy Coach program.

Our experimental procedure has two main components. First, we designed a randomized mailer campaign with several treatment arms and a control arm. The mailer campaign is used to recruit households in the second stage of the experiment. The mailer provides a link to an online survey. The online survey is a way to invite households to participate in an Energy Coach program. The nature of the Energy Coach program is randomized, and we use an incentive-compatible procedure to elicit the willingness to accept (WTA) for the program. The online survey also asks additional questions that we plan on using to validate our approach and explore heterogeneity.

Not available

There are three randomization procedures. First, we randomize the mailer campaign. Using the database of addresses, we use an Excel number generator number to assign a value of 1, 2, or 3 to each address. These numbers correspond to the three arms of the mailer campaign.

For households that decide to participate in the online survey using the QR code provided on the mailer, Qualtrics randomizes survey participants between the two versions of the Energy Coach program. We use a 50/50 split.

Finally, we use a lottery to determine whether and how much participants will be compensated for participating in either version of the Energy Coach program. For participants randomized into the human arm, we ask them to state the minimum amount that will convince them to spend 1 hour with the Energy Coach. After they have completed the survey, Qualtrics generates a random number between $0 and $100 at random (just like a lottery). If this random number is higher than the amount they entered, they will be entered into a pool of people eligible to work with an energy coordinator. Then, we will randomly select 30 people from this group who will be contacted by email to schedule a 1-hour session with the energy coordinator. After the session, they will receive the dollar amount chosen by the computer, plus the $10 gift card reward. If the random number is less than the amount they entered, they will only get the base $10 gift card for completing this survey.

For participants randomized into the machine arm, we also ask them to state the minimum amount that will convince them to spend 1 on the Energy Coach digital tool. After they have completed the survey, Qualtrics generates a number between $0 and $100 at random (just like a lottery). If this random number is higher than the amount they entered, they all will be rewarded the dollar amount chosen by the computer after completing the energy online training tool. If the random number is lower than the amount they entered, they will only be rewarded with the base $10 gift card.

The randomization of the intervention is at the individual level. We also perform a randomization for each recipient selected to receive the intervention to determine the compensation amount.

No

Our target population is up to 25000 households. We are not clustering to conduct the randomization.

Our target population is 25000 households, and we are expecting a response rate ranging from 2.5% to 20% to the mailer invitation. Our planned number of observations is thus approximately between 500 and 4000 survey respondents.

For the mailer campaign, our sample size was determined to max out our budget for this item. For the online survey, we have a 50/50 split between the treatment arms to maximize our power. Given that our planned number of observations is between 500 and 4000 survey respondents, each version of the energy coach program will receive 50% of this expected number.

Our sample size allocation was designed to max out our budget. Printing and postage costs per mailer are approximately $2. Assuming that half of our respondents will state they need to be compensated to work with an Energy Coach or digital assistant. To compute our MDE, we have taken a uniform prior regarding the distribution of the WTA. In particular, we have assumed that the WTA is uniformly distributed between 0 and 100, the range we provide to the participants in the survey. This uniform distribution of WTA results in a standard deviation of about 30. Based on the assumption that our budget will result in 1,250 heat pump mailer survey responders, we have further assumed that half of these respondents will be incentive compatible. With these assumptions, the Minimum Detectable Effect is a 12% difference in the WTA for the Energy Coach versus the digital assistant.