Experimental Design Details
Background for particular experimental design:
Our experimental design is largely driven by the fact that electricity consumption data of most German households is only available on an annual basis. The reason for this lack of real-time consumption data is that the vast majority of German households does not have smart electricity meters. Most utility providers sent an employee to every household once a year in order to read the meter manually. Any experimental design that requires consumption data on a more frequent basis has to involve alternative ways of gathering consumption data.
For this purpose, we use a new mobile application feature that recognizes and automatically reads the electricity meter as soon as the app user takes a picture of the meter with her mobile phone. The electricity consumption data can then be uploaded to a server of the utility provider. For the user, this has the advantage that she does not need to organize manual meter readings with the utility. For the researcher, this new function facilitates the collection of electricity consumption data more than once a year if the user is willing to take photos of her meter on a more frequent basis.
We integrate this new “photo meter reading” function into the mobile application of a large utility provider. The app is used by around 100,000 individuals and involves functions such as a local news feed, real-time information on changes in bus schedules, notifications of free parking spots downtown, etc. For the purpose of our experiment, the utility advertises the meter reading feature and offers a lottery to app users who sign up to test the new function. To recruit participants, about 50,000 utility customers are directly mailed to receive advertisement on the meter reading and the lottery. Further, a local newspaper is used to distribute 18,000 flyers to households. Finally, an ad in a local newspaper, radio spots, flyers and posters should encourage participation. The marketing strategy leaves both the experimental character of the study and the cooperation with researchers unmentioned.
The experiment runs for a period of 4 months. Upon registration for the experiment, subjects are randomized into one of three groups: 1) adjustable goal, 2) non-adjustable goal, 3) control group. The first month constitutes the baseline period, in which there is no difference between the three groups. All subjects are asked to make a photo of their meter at the beginning and at the end of this month. As of month 2, subjects in the control group are asked to continue making a photo of their meter every four weeks. In addition to taking these pictures, subjects in group 1) and 2) are asked to set themselves monthly electricity consumption goals as specified below. The only difference between group 1) and group 2) is that, subjects in group 2) experience an additional feature which allows them to readjust the chosen goal during the month.
At the beginning of month 4, randomly chosen subjects either receive A) an additional financial incentive to conserve energy or B) no further financial incentive. The probability to be assigned to subgroup A or B is 50%. Thus, month 4 involves 6 experimental groups: 1.A, 1.B, 2.A, 2.B, 3.A, 3.B.
At the end of the fourth month, subjects are invited to participate in a survey in which we elicit time-preferences and other relevant parameters.
Upon registration for testing the new feature, subjects provide information on the type of the electricity meter they own and give informed consent to the conditions of participation. Next, the app randomizes subjects into three experimental groups. All subjects are then asked to make a photo of their meter using the new function in the mobile app. After 4 weeks the app asks every subject to take another photo of the meter. The difference between the meter reading in week 4 and the reading in week 1 indicates the baseline electricity consumption. This baseline period also allows subjects to become more familiar with their electricity consumption.
For every taken picture requested by the app, the subject receives a lottery ticket and thereby increases her chances of winning a prize. We sent a reminder to take the photo always two days before and two days after the day at which the photo is due. If the subject fails to take a picture two days after the due day, she does not gather the lottery ticket (but continues to participate in the experiment).
After subjects have taken their second photo in week 4, they are provided with additional information on how energy service consumption (e.g. using a light bulb or doing the laundry) translates into kilowatt hour consumption. Once subjects have read this information, they are treated differently depending on their group:
1) Group with non-adjustable goal:
Subjects are asked to set themselves a goal for how much electricity they plan to consume in the next 4 weeks. While choosing the goal in kilowatt hours, the app also tells the consumer how much these kilowatt hours translate into percentage savings relative to the baseline period.
Subjects set three goals in total: at the beginning of month 2, 3 and 4.
2) Group with adjustable goal:
Just as in the first group, subjects must set themselves an energy consumption goal. Immediately after they have set the first goal, they are informed that they may re-adjust the level of the goal at any point in time during the following four weeks. The idea behind this re-adjustment function is that it gives future selves the possibility to change the reference point induced by the goal. We hypothesize that if consumers are present-biased, future selves should re-adjust the goal in order to avoid the disciplining effect of the commitment device. In this case, electricity consumption would be higher for subjects in the adjustable than in the non-adjustable goal group.
We acknowledge that this difference may be mitigated if re-adjusting the goal is too costly. E.g., subjects affected by self-image concerns may avoid changing the goal if this is associated with a (costly) feeling of being undisciplined.
Another channel that could confound our interpretation of the treatment effect is uncertainty. The possibility to change the level of the goal could be beneficial (even from the perspective of “self zero”) as subjects can adjust to unforeseen events. Alternatively, subjects could be initially uncertain of their energy-savings potential and learning over time causes them to re-adjust and set a more adequate goal. We plan to address these alternative explanations in an explorative analysis.
3) Control Group:
Subjects in this group do not receive the possibility to set electricity consumption goals and are only asked to make a photo of their meter every 4 weeks.
Additional financial incentive to conserve energy:
At the beginning of month 4, subjects in every group are randomly assigned to an additional financial incentive to conserve energy. With a probability of 50% the subject is informed that she participates in an additional lottery. If she wins the lottery, she receives 1 Euro per kilowatt hour saved in month 4 relative to her electricity consumption in month 3. The chances to win in the lottery are calculated based on the current number of app users and are communicated to the subject. The total amount she may receive is limited to 100 Euros. Prizes are paid out in the form of vouchers for the online shop Amazon.com. This additional monetary incentive allows to compare the effectiveness of our non-price intervention (goal setting) to an increase (in expectation) in the price of electricity. Moreover, we intent to estimate own-price elasticities using this exogenous variation in prices.
At the end of the experiment, subjects are invited to participate in an online survey. The survey elicits time preferences, loss aversion, price beliefs and further subject-specific characteristics.
While every subject is only part of the experiment for four months and group 1) and 2) subjects experience the intervention for three months, the total period of our trial is five months due to a one-month registration period. This implies that subjects who sign up directly at the beginning of the trial finish the experiment four months after trial start. Subjects who sign up one month after the beginning of the trial finish the experiment five months after trial start.
Control group outside of the mobile app:
To get an indication of monthly energy consumption of households not using the mobile application, we gather consumption data of an additional sample. In July and August 2017, randomly selected households in the city of Münster were invited by mail to participate in a research project. For the purpose of our experiment, these households will receive an e-mail invitation to answer two surveys. The first survey takes place at the same date when the experiment in the mobile application starts and asks subjects to report their meter reading. The second survey is sent out four months later and again asks for the meter reading. The second survey also includes the same questions as the survey answered by the subjects in the mobile application. The difference between the meter reading in survey 1 and 2 provides us with an indication of the electricity consumption of households not using the mobile application.