Home Energy Upgrades survey

A range of different interventions will be used throughout the 5 trials.

2024-05-16

2024-06-30

Trial 1 - the proportion of highly efficient houses selected across the 8 choice sets.

Trial 2 - whether their priority upgrade is an energy upgrade (1 = energy upgrade, 0 = another upgrade).

Trial 3 - whether a participant clicks-through to the website.

Trial 4 - whether the participant clicked on an upgrade rather than an energy efficiency tip or no action (binary, 1 = clicked on an upgrade, 0 = did not click on an upgrade).

Trail 5 - whether the participant intends to make upgrades on their home this year (1 = Definitely or likely, 0 = Unsure, unlikely or definitely not).

Trial 1 - Individual level outcomes will be averaged within treatment groups to give the average proportion of highly efficient homes selected by arm. High home energy ratings will be those at 60/100 or higher.

Trial 1 - The importance ranking the participant gives to ‘energy rating’ which is among a selection of 7 options.
- The average rating participants choose for homes that don’t display a rating. (In the mandatory condition participants are asked to imagine an unrated home.)

Trial 2 - Whether the participant thinks that knowing their home energy rating before choosing home upgrades and renovations is useful (binary, 1 = ‘extremely useful’ or ‘very useful’).
- The number of home energy upgrades selected by the participant when they are asked to select 3 upgrades. This will be averaged within treatment groups to give the mean number of upgrades within each arm.

Trial 3 - none

Trial 4 - none

Trial 5 - Whether the participant is confident about the upgrades to make.

Trial 1 - The survey question ‘Please order these property features from most to least important by dragging them up or down’. Response options are 1 (number of bedrooms), 2 (number of bathrooms), 3 (size of land), 4 (nice interior), 5 (nice garden and landscaping), 6 (home energy rating), 7 (price).
- Numeric response options are 0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100


Trial 5 - This will be constructed from two items: ‘How confident are you that you know which energy efficient upgrades to make to your home’ and ’How confident are you in planning energy upgrades for your home’. Each item will be scored 0 – 3 with 0 representing not at all confident. The composite outcome measure will be a mean of these two scores

The survey will have 5 embedded trials.

The survey will have 5 embedded trials:

Trial 1 - A discrete choice experiment and randomised controlled trial to test mandatory vs voluntary disclosure of home energy ratings.
Participants will be first presented with a vignette which describes the process of searching for houses to inspect. They are shown a mock search result page for a real estate website (with 2 properties displayed as options, as well as a third option of ‘neither of these options’.). Participants are then asked to choose which property they would like to inspect each weekend, for the next 8 weeks.

Trial 2 - A 2x2 factorial trial to test the impact of mandatory vs voluntary disclosure of home energy ratings on buyer and seller home energy upgrades behaviour. Participants will be randomised into one of four hypothetical scenarios: they will be a buyer or a seller in the market, and in a voluntary or a mandatory disclosure setting. They will then be shown the same single, detailed real estate property listing. The listing says that home is 2003 build (i.e. pre-minimum energy standards) and that home is in all original conditions (i.e. implies that appliances are likely to be very inefficient). The condition of the house and their financial circumstances will remain constant across the scenarios. They are then asked to choose 3 upgrades from a list that includes home energy performance upgrades and non-energy related upgrades.

Trial 3 - A four-arm randomised controlled trial to test messages to encourage click-through to the yourhome.gov.au website. Participants will be randomised to 1 of 4 end of survey messages with a link to https://www.yourhome.gov.au/ . This design can help us determine which messages are most likely to promote information-seeking behaviour.

Trial 4 - A 4x2 factorial design to test whether elements of a home energy certificate, or offering a free updated certificate, increase home energy upgrade behaviour. Participants will be presented with 1 of 4 customised hypothetical scenarios, based on their own situation. In each, they are asked to imagine they own a house, which gets very cold in winter and warm in summer. They were offered a government-funded home energy performance assessment. They will be presented with a mock certificate that contains the results of the home energy assessment and invited to choose what actions they would take in response to the certificate.

Trial 5 - A four-arm trial to test the impact of decision support tools on home upgrade behaviour. Participants will be randomised to 1 of 4 treatment groups. Afterwards they will be asked a range of questions, including about their interest in getting an energy upgrade assessment, and their intention to make upgrades to their home this year.

For the trials participants will be randomised to treatment groups within the survey platform, Qualtrics, using the inbuilt randomisation functionality, with roughly equal probability of assignment across the groups (using the ‘evenly present elements’ option in Qualtrics).
The rest of the survey will be non-randomised.
Only participants who live in a detached home or townhouse that they own will be assigned to take Trials 4 & 5.

Individual survey respondent

No

Not applicable.

approximately 13,000 survey respondents.

We expect that Trials 1, 2, 4, 5 will each have roughly 6,500 participants, while Trial 3 will have roughly 13,000 participants. Within each trial participants will be randomly allocated with roughly probability assignment to the treatment groups.

Trial 1 - This RCT is designed to observe a minimal detectable effect of 2.2 percentage points. As we do not have a baseline understanding of the proportion of times that participants would pick the high energy rating option, we have made the most conservative assumptions for the power calculation. Therefore, we assumed that the average proportion for the control was 50%, and thus the treatment group’s average proportion was 52.2%. We will use conventional settings for type I error with alpha at 5% and 80% power. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. To achieve 80% power to detect an effect size of 2.2 percentage points (Cohen’s h = -0.04) we will recruit 3,250 per arm for a total sample size of 6,500. We conducted power calculations in R version 4.4.0 using the ‘pwr’ package version 1.3.2. Trial 2 - The sample for the survey experiment will be roughly 6,500 individuals. With this sample size we will have approximately 90% power to detect an effect of 4 percentage points for both hypothesis. That is for a 4 percentage point main effect for the voluntary vs mandatory disclosure hypothesis, and a 4 percentage point change in the buyer vs seller. If there was an interaction between the effects we are powered with 88% power for a 5 percentage point main effect, with either an antagonistic or synergistic interaction effect of 2 percentage points. We are not powered to measure the interaction. We have set alpha at 5% and power at 90%. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. We conducted power calculations in R version 4.4.0 using simulation. Trial 3 - We designed this study for a minimum detectable effect of 0.6 percentage points (a click-through rate of 0.9% in the intervention arms compared with 0.3% in the control arm). These numbers are based on similar work encouraging clicks. We set alpha at 5% and power at 90%. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. To achieve 90% power to detect an effect size of 0.6 percentage points (Cohen’s h = 0.08) we will recruit 3250 per arm for a total sample size of 13,000. We conducted power calculations in R version 4.4.0 using the ‘pwr’ package version. Trial 4 - The sample for the survey experiment will be roughly 6,500 individuals. With this sample size we will have approximately 90% power to detect an effect of 4.5 percentage points in the main effect for the upgrades action prioritisation experiment and an effect of 3 percentage points in the main effect for the incentives experiment. We do not expect these experiments to interact and we are not powered to detect an interaction. We set alpha at 5% and power at 90%. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. We conducted power calculations in R version 4.4.0 using simulation. Trial 5 - We designed this study for a minimum detectable effect of approximately 5 percentage points. We will set alpha at 5% and power at 90%. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. We conducted power calculations in R version 4.4.0 using the ‘pwr’ package version 1.3-0.