Home Energy Upgrades survey

Last registered on May 21, 2024

Pre-Trial

Trial Information

General Information

Title
Home Energy Upgrades survey
RCT ID
AEARCTR-0013623
Initial registration date
May 16, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 21, 2024, 11:31 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Behavioural Economics Team of the Australian Government

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2024-05-16
End date
2024-06-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project aims to test a range of intervention options to encourage home upgrade behaviour using survey experiments.
External Link(s)

Registration Citation

Citation
Team Registration, BETA. 2024. "Home Energy Upgrades survey." AEA RCT Registry. May 21. https://doi.org/10.1257/rct.13623-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
A range of different interventions will be used throughout the 5 trials.
Intervention (Hidden)
Trial 1 - Participants will be randomised to one of two different conditions. One will mimic the voluntary disclosure of home energy ratings by only showing rating for houses with a high rating (60/100 or higher) and not showing any information for those with low ratings. The other arm will mimic mandatory disclosure where the energy rating of the home is displayed regardless of its level.

Trial 2 - Participants will be randomised into one of four hypothetical scenarios: they will be a buyer or a seller in the market, and in a voluntary or a mandatory disclosure setting. In the mandatory disclosure condition the house energy rating is displayed for them to see, while in the voluntary condition the house’s energy rating is not displayed.

Trial 3 - Participants will be randomly allocated to see one of 4 different messages designed to encourage them to visit yourhome.gov.au. One message will be an attention control (a very basic message), and the other 3 will be new messages incorporating behavioural insights.

Trial 4 - For upgrade prioritisation there are 4 different interventions with suggested upgrade actions prioritised in different ways. The section ‘Suggested energy efficiency upgrades for your home’ (bottom right of the certificate) will have one of 4 different subheadings, and number of suggested actions. For the Incentive conditions - 2 different interventions, people will see a certificate either with or without a message encouraging them to upgrade to get free updated rating certificate.

Trial 5 - There is a control group, plus 3 intervention groups where people receive either a decision support tool, a 'one-stop shop' website, or both the decision support tool and the 'one-stop shop' website.

Intervention Start Date
2024-05-16
Intervention End Date
2024-06-30

Primary Outcomes

Primary Outcomes (end points)
Trial 1 - the proportion of highly efficient houses selected across the 8 choice sets.

Trial 2 - whether their priority upgrade is an energy upgrade (1 = energy upgrade, 0 = another upgrade).

Trial 3 - whether a participant clicks-through to the website.

Trial 4 - whether the participant clicked on an upgrade rather than an energy efficiency tip or no action (binary, 1 = clicked on an upgrade, 0 = did not click on an upgrade).

Trail 5 - whether the participant intends to make upgrades on their home this year (1 = Definitely or likely, 0 = Unsure, unlikely or definitely not).
Primary Outcomes (explanation)
Trial 1 - Individual level outcomes will be averaged within treatment groups to give the average proportion of highly efficient homes selected by arm. High home energy ratings will be those at 60/100 or higher.

Secondary Outcomes

Secondary Outcomes (end points)
Trial 1 - The importance ranking the participant gives to ‘energy rating’ which is among a selection of 7 options.
- The average rating participants choose for homes that don’t display a rating. (In the mandatory condition participants are asked to imagine an unrated home.)

Trial 2 - Whether the participant thinks that knowing their home energy rating before choosing home upgrades and renovations is useful (binary, 1 = ‘extremely useful’ or ‘very useful’).
- The number of home energy upgrades selected by the participant when they are asked to select 3 upgrades. This will be averaged within treatment groups to give the mean number of upgrades within each arm.

Trial 3 - none

Trial 4 - none

Trial 5 - Whether the participant is confident about the upgrades to make.
Secondary Outcomes (explanation)
Trial 1 - The survey question ‘Please order these property features from most to least important by dragging them up or down’. Response options are 1 (number of bedrooms), 2 (number of bathrooms), 3 (size of land), 4 (nice interior), 5 (nice garden and landscaping), 6 (home energy rating), 7 (price).
- Numeric response options are 0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100


Trial 5 - This will be constructed from two items: ‘How confident are you that you know which energy efficient upgrades to make to your home’ and ’How confident are you in planning energy upgrades for your home’. Each item will be scored 0 – 3 with 0 representing not at all confident. The composite outcome measure will be a mean of these two scores

Experimental Design

Experimental Design
The survey will have 5 embedded trials.
Experimental Design Details
The survey will have 5 embedded trials:

Trial 1 - A discrete choice experiment and randomised controlled trial to test mandatory vs voluntary disclosure of home energy ratings.
Participants will be first presented with a vignette which describes the process of searching for houses to inspect. They are shown a mock search result page for a real estate website (with 2 properties displayed as options, as well as a third option of ‘neither of these options’.). Participants are then asked to choose which property they would like to inspect each weekend, for the next 8 weeks.

Trial 2 - A 2x2 factorial trial to test the impact of mandatory vs voluntary disclosure of home energy ratings on buyer and seller home energy upgrades behaviour. Participants will be randomised into one of four hypothetical scenarios: they will be a buyer or a seller in the market, and in a voluntary or a mandatory disclosure setting. They will then be shown the same single, detailed real estate property listing. The listing says that home is 2003 build (i.e. pre-minimum energy standards) and that home is in all original conditions (i.e. implies that appliances are likely to be very inefficient). The condition of the house and their financial circumstances will remain constant across the scenarios. They are then asked to choose 3 upgrades from a list that includes home energy performance upgrades and non-energy related upgrades.

Trial 3 - A four-arm randomised controlled trial to test messages to encourage click-through to the yourhome.gov.au website. Participants will be randomised to 1 of 4 end of survey messages with a link to https://www.yourhome.gov.au/ . This design can help us determine which messages are most likely to promote information-seeking behaviour.

Trial 4 - A 4x2 factorial design to test whether elements of a home energy certificate, or offering a free updated certificate, increase home energy upgrade behaviour. Participants will be presented with 1 of 4 customised hypothetical scenarios, based on their own situation. In each, they are asked to imagine they own a house, which gets very cold in winter and warm in summer. They were offered a government-funded home energy performance assessment. They will be presented with a mock certificate that contains the results of the home energy assessment and invited to choose what actions they would take in response to the certificate.

Trial 5 - A four-arm trial to test the impact of decision support tools on home upgrade behaviour. Participants will be randomised to 1 of 4 treatment groups. Afterwards they will be asked a range of questions, including about their interest in getting an energy upgrade assessment, and their intention to make upgrades to their home this year.
Randomization Method
For the trials participants will be randomised to treatment groups within the survey platform, Qualtrics, using the inbuilt randomisation functionality, with roughly equal probability of assignment across the groups (using the ‘evenly present elements’ option in Qualtrics).
The rest of the survey will be non-randomised.
Only participants who live in a detached home or townhouse that they own will be assigned to take Trials 4 & 5.
Randomization Unit
Individual survey respondent
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Not applicable.
Sample size: planned number of observations
approximately 13,000 survey respondents.
Sample size (or number of clusters) by treatment arms
We expect that Trials 1, 2, 4, 5 will each have roughly 6,500 participants, while Trial 3 will have roughly 13,000 participants. Within each trial participants will be randomly allocated with roughly probability assignment to the treatment groups.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Trial 1 - This RCT is designed to observe a minimal detectable effect of 2.2 percentage points. As we do not have a baseline understanding of the proportion of times that participants would pick the high energy rating option, we have made the most conservative assumptions for the power calculation. Therefore, we assumed that the average proportion for the control was 50%, and thus the treatment group’s average proportion was 52.2%. We will use conventional settings for type I error with alpha at 5% and 80% power. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. To achieve 80% power to detect an effect size of 2.2 percentage points (Cohen’s h = -0.04) we will recruit 3,250 per arm for a total sample size of 6,500. We conducted power calculations in R version 4.4.0 using the ‘pwr’ package version 1.3.2. Trial 2 - The sample for the survey experiment will be roughly 6,500 individuals. With this sample size we will have approximately 90% power to detect an effect of 4 percentage points for both hypothesis. That is for a 4 percentage point main effect for the voluntary vs mandatory disclosure hypothesis, and a 4 percentage point change in the buyer vs seller. If there was an interaction between the effects we are powered with 88% power for a 5 percentage point main effect, with either an antagonistic or synergistic interaction effect of 2 percentage points. We are not powered to measure the interaction. We have set alpha at 5% and power at 90%. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. We conducted power calculations in R version 4.4.0 using simulation. Trial 3 - We designed this study for a minimum detectable effect of 0.6 percentage points (a click-through rate of 0.9% in the intervention arms compared with 0.3% in the control arm). These numbers are based on similar work encouraging clicks. We set alpha at 5% and power at 90%. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. To achieve 90% power to detect an effect size of 0.6 percentage points (Cohen’s h = 0.08) we will recruit 3250 per arm for a total sample size of 13,000. We conducted power calculations in R version 4.4.0 using the ‘pwr’ package version. Trial 4 - The sample for the survey experiment will be roughly 6,500 individuals. With this sample size we will have approximately 90% power to detect an effect of 4.5 percentage points in the main effect for the upgrades action prioritisation experiment and an effect of 3 percentage points in the main effect for the incentives experiment. We do not expect these experiments to interact and we are not powered to detect an interaction. We set alpha at 5% and power at 90%. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. We conducted power calculations in R version 4.4.0 using simulation. Trial 5 - We designed this study for a minimum detectable effect of approximately 5 percentage points. We will set alpha at 5% and power at 90%. We have chosen these settings because the intervention is low risk and it would be worse to reject a possible real effect than to accept a potentially spurious one. We conducted power calculations in R version 4.4.0 using the ‘pwr’ package version 1.3-0.
IRB

Institutional Review Boards (IRBs)

IRB Name
Macquarie University Human Research Ethics Committee
IRB Approval Date
2024-05-14
IRB Approval Number
Project ID: 15629
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials