Banking survey experiments

Last registered on April 10, 2025

View Trial History

Pre-Trial

Trial Information

General Information

Title

Banking survey experiments

RCT ID

AEARCTR-0015747

Initial registration date

April 06, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

April 10, 2025, 7:33 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Australia

Region

Primary Investigator

Name

BETA Team Registration

Affiliation

Behavioural Economics Team of the Australian Government

Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status

On going

Start date

2025-03-17

End date

2025-08-29

Keywords

Finance & Microfinance

Additional Keywords

Behaviour, Finance & Microfinance, Firms & Productivity, Prompts, Switching, Bank

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

The Treasury is advising the government on its responses to Australian Competition and Consumer Commission (ACCC)’s Retail deposits inquiry report (ACCC 2023) and Home loan price inquiry report (ACCC 2020). Both reports find a lack of consumer engagement in these markets, despite the significant potential financial gain that could be realised by many customers through regular engagement. Treasury has asked BETA to conduct the consumer testing recommended by ACCC’s reports, to determine the optimal design for prompts aimed at increasing engagement.

BETA is conducting a suite of research activities to support this work: field trial, interviews, surveys. There are two aspects to the suite of work. The first is identifying the best prompt design for enabling customers to engage with the market using an RCT design. The second is understanding frictions and barriers that make it hard for customers to switch to help identify other potential solutions, aside from prompts (separate analyses not PAP relevant). This project focuses on the first part of the work only as prompt designs will be testing using RCTs embedded in the surveys.

As part of this work, BETA is conducting two RCTs embedded in surveys (one on mortgages and one on retail deposits) to test prompts and collect data on consumer experiences with switching. These two RCTs are the focus of this project.

External Link(s)

Registration Citation

Citation

Team Registration, BETA. 2025. "Banking survey experiments." AEA RCT Registry. April 10. https://doi.org/10.1257/rct.15747-1.0

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

Each RCT (one for mortgage, one for savings account) will be a three-arm survey experiment. The effects of prompts (attention control + 2 treatments) will be tested via vignettes. Participants are asked to provide advice to their friend in the vignette. A mock-up banking app showing their friend’s financial position is shown and the app screen includes one of the three prompts.
The text in the prompts are as below.
Mortgage stream
Attention control: Check out our great range of products! We have a wide range of products to suit your financial needs. Visit banking.com.au/all-products for more information. Eligibility requirements may apply.
Treatment 1 (simple prompt): A lower interest rate will save you money on your home loan. Visit moneysmart.gov.au/mortgagecalculator to see how much you could save with a lower interest rate. This information is required by the Australian Government.
Treatment 2 (detailed prompt): Your home loan interest rate is higher than average. 7.8% your current rate, 6.5% average rate for similar home loans. Visit moneysmart.gov.au/mortgagecalculator to see how much you could save with a lower interest rate. This information is required by the Australian Government.
Savings stream
Attention control: Check out our great range of products! We have a wide range of products to suit your financial needs. Visit banking.com.au/all-products for more information. Eligibility requirements may apply.
Treatment 1 (simple prompt): Grow your savings with a higher interest rate Visit moneysmart.gov.au to check if you are getting the best interest rate for your savings. This information is required by the Australian Government.
Treatment 2 (detailed prompt): Grow your savings with a higher interest rate A competitive savings account will offer an interest rate of 4% or higher. Visit moneysmart.gov.au to check if you are getting the best interest rate for your savings. This information is required by the Australian Government.

Intervention (Hidden)

Intervention Start Date

2025-03-17

Intervention End Date

2025-03-28

Primary Outcomes

Primary Outcomes (end points)

Savings stream primary outcome: A binary variable based on recommendations provided by survey respondent about engaging with the market where 1 is engaging with the market (e.g. switching to a higher interest savings account or looking for a better interest rate savings account) and the rest is coded 0.
Mortgage stream primary outcome: A binary variable based on recommendations provided by survey respondents about engaging with the market where 1 is engaging with the market (e.g. looking for a better mortgage product, asking for a reprice, switching banks to get better interest rate on home loan) and the rest is coded 0.
The question asked to participants for the primary outcome is ‘After looking at your friend's banking app screen below, what are some of the actions you suggest that he takes to improve his financial position?’

Primary Outcomes (explanation)

See Pre-analysis Plan for further details and a copy of the script.

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

This project includes 2 RCTs (one for mortgage stream, one for savings stream) embedded in surveys. Each RCT will be a 3-arm randomised controlled field trial. All participants will be shown a vignette asking them to recommend actions to help their friend improve their financial position. Part of the vignette includes a banking app screen which contains prompts (interventions).

Experimental Design Details

Randomization Method

Participants will be randomised into one of the three cells in each RCT within the survey platform, Qualtrics, using the inbuilt randomisation functionality, with roughly equal probability of assignment across the groups (using the ‘evenly present elements’ option in Qualtrics).

Randomization Unit

Individual level

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

No clusters

Sample size: planned number of observations

approximately 4200-4500 participants for each RCT

Sample size (or number of clusters) by treatment arms

We expect approximately 4200-4500 participants for each RCT (one for mortgage and one for savings) with approximately 1400-1500 participants in each arm of the RCT

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Based on previous work and a pilot study, we estimate the effect size for this study will be around 10-15 percentage points. We will set the alpha at 5% and power at 90% as this is a survey experiment and we do not want to miss a true effect. With a sample size of around 4200 we will have 90% power to detect an effect size of 5pp or approximately 0.1 Cohen’s h. Power calculations were conducted in R, version 4.4.1 using ‘pwr’ package version 1.3-0.

Supporting Documents and Materials

IRB