Banking prompts trial

Last registered on December 20, 2024

View Trial History

Pre-Trial

Trial Information

General Information

Title

Banking prompts trial

RCT ID

AEARCTR-0015045

Initial registration date

December 17, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

December 20, 2024, 1:53 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Australia

Region

Primary Investigator

Name

BETA Team Registration

Affiliation

Behavioural Economics Team of the Australian Government

Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status

On going

Start date

2024-12-03

End date

2025-08-29

Keywords

Behavior, Finance & Microfinance, Firms & Productivity, Lab

Additional Keywords

Prompts, Switching, Bank

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

The Treasury is advising the government on its responses to the ACCC’s Retail Deposits Inquiry and Home Loan Price Inquiry. Despite the financial benefits that can result from switching products, the reports found a lack of consumer engagement in these markets. For both markets the ACCC recommended that prompts be considered encouraging consumers to see if they could benefit from switching to alternative products that might better suit their needs.

BETA is working with the Treasury to: (1) identify the relevant behavioural factors in consumer financial decision-making; (2) understand the utility of prompts; (3) determine the optimal design for the prompts proposed in the ACCC reports; and (4) understand the barriers and enablers consumers experience when switching or repricing to identify additional interventions which may complement, or be an alternative to, the use of prompts.

BETA is conducting a suite of research activities to support this work, including a field trial (‘trial’) in partnership with a large retail bank. This trial will specifically test a single prompt designed to encourage disengaged customers to engage with their bank to negotiate a better deal on their home loan. This will provide evidence informing the effectiveness of prompts to encourage engagement in a relatively disengaged cohort.

Trial Aims
The aim of the trial is to test the effectiveness of a prompt to encourage disengaged customers to call the bank to see if they are eligible to receive a lower interest rate on their loan.

External Link(s)

Registration Citation

Citation

Team Registration, BETA. 2024. "Banking prompts trial ." AEA RCT Registry. December 20. https://doi.org/10.1257/rct.15045-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Participants in the trial will see a prompt surfaced in their banking app if they are in the intervention arm. Control participants will not see the prompt. We will use bank administrative data to measure the primary and secondary outcomes.
The text of the prompt reads “Interested in reducing your mortgage repayments? Check whether you can get a lower interest rate on your home loan”. Customers can call the bank directly from the prompt or through their usual channels.

Intervention (Hidden)

Intervention Start Date

2024-12-03

Intervention End Date

2024-12-17

Primary Outcomes

Primary Outcomes (end points)

Customer contact: Measured at both the cluster and individual level. The cluster-level analysis will examine whether any customer within each cluster contacted the bank, while the individual-level analysis examines whether each individual customer contacted the bank.

Primary Outcomes (explanation)

Cluster-level customer contact rate: The primary outcome variable is a binary indicator at the cluster level, taking the value of 1 if at least one customer telephoned the bank on their general contact line or their specialist home loan line in the trial period, and 0 otherwise. This definition captures cluster-level engagement with the intervention, recognising that for shared loans, one borrower making contact is sufficient. The variable will be constructed from administrative data measuring the count of calls made to the bank in the trial period. This will be averaged within the treatment and control groups to give the proportion of clusters in which at least one person made contact with the bank in each arm. The outcome will be measured at T1.

Individual customer contact rate: At an individual level the outcome will be a binary indicator, flagging whether a customer telephoned the bank on their general contact line or their specialist home loan line in the trial period (0 = did not make contact, 1 = made contact). The variable will be constructed from administrative data measuring the count of calls made to the bank in the trial period. This will be averaged within treatment and control group to give the proportion of customers who made contact with the bank by arm. This outcome will be measured at T1.

Secondary Outcomes

Secondary Outcomes (end points)

Interest rate, repricing, financial stress, financial wellbeing

Secondary Outcomes (explanation)

Interest rate: A variable at the cluster level, the interest rate outcome will be a continuous variable representing the mean interest rate of all loans in a cluster expressed as a percentage. This will be averaged within treatment groups to give the mean interest rate by arm. This outcome will be measured at T1.

Repricing: A binary variable at the cluster level of whether at least one loan in a cluster was repriced during the intervention period. This will be constructed from administrative data, with 1 indicating at least one loan was re-priced and 0 if it was not. For shared loans, the outcome captures whether the shared loan was re-priced regardless of which borrower initiated the change. This will be averaged within treatment and control groups to give the proportion of clusters with reduced rates by arm. This outcome will be measured at T1.

Financial stress: A binary indicator at the cluster level of whether at least one person in a cluster has experienced financial stress (0 = no financial stress, 1 = financial stress). This will be any of:
• Hardship support requests, OR
• Late payment fees for any of their loans, OR
• Arrears on any of their loans
This will be averaged within treatment groups to give the proportion of clusters in financial stress by arm. This outcome will be measured at T2 and T3.

Financial wellbeing: At the cluster level we will measure the total balance of available accounts such as deposit accounts, investment accounts, and shares held by each individual in the cluster at T2 and T3. The construction of this variable will depend on account information that is shared by the bank partner. This will be averaged within treatment groups to give the mean balance in each arm.

Experimental Design

This project will be a 2-arm cluster randomised controlled field trial. Clusters will be defined in the administrative data as people who share loans, defined through inter-related person-loan pairs. This is because the intervention will be delivered to individuals, and the outcomes will relate to both people (e.g. calling the bank) and loans (e.g. rate reductions).
Clusters will be randomly assigned to either the treatment group or control group with approximately equal number in each arm. The randomisation is performed at the cluster level to ensure that all members of a cluster receive the same treatment condition.
After initial randomisation, stakeholders will remove 10% of individuals from the treatment group as part of their automated system.
The primary analysis will be conducted at the cluster level; i.e., whether at least one individual in a cluster takes action in response to the intervention. Analysis at the individual level represents an upper bound.
Outcomes will be measured at three time points. Timepoint 1 (T1) will be 7 days after the close of the prompt on 10 December 2024 in the mobile application (17 December 2024). Timepoint 2 (T2) will be 3 months post T1. Timepoint 3 (T3) will be 3 months post T2. T3 outcomes will explore longer-term effects of the intervention on financial wellbeing of participants. T2 outcomes will be exploratory.

Experimental Design Details

Randomization Method

Randomisation will be at the cluster level. Clusters of connected customers will be randomly assigned to treatment and control groups in a 1:1 ratio. The clusters are flexible structures able to manage a multitude of individual arrangements and the many-to-many relationship between people and loans.
After initial treatment assignment, 10% of customers from the treatment group will be randomly selected to not receive the intervention as part of the stakeholder’s automatic program implementation. The random selection will occur at the individual customer level, not the cluster level. For single-customer clusters, the entire network will not receive the intervention if selected. For multi-customer clusters, only the selected customers will not receive the intervention. All people and clusters will be analysed as initially allocated to maintain the integrity of clusters post-randomisation.

Randomization Unit

Randomisation will be at the cluster level.

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

The planned number of clusters for this trial is 11,922 (defined as unique groups of individuals and their shared loans).

Sample size: planned number of observations

18,769 person-loan pairs, representing the total number of unique observations in the dataset

Sample size (or number of clusters) by treatment arms

Treatment group: 5,961 clusters
Control group: 5,961 clusters

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

This study has a fixed sample size and will analyse 11,922 loan clusters. Using a joint decision framework at the cluster level and accounting for the 10% of individuals that will be randomly removed from receiving the intervention, our estimation approach (which treats a loan cluster as treated if at least one individual remains in the treatment group) indicates we will have a minimum detectable effect of 0.78 percentage points. Historical click-through rates for interventions using this type of notification shell are approximately 1%. Therefore, with our sample size, we will be able to detect an increase from 1% to 1.78%, representing a 78.2% relative increase. This trial will be adequately powered to detect these differences using a one-sided test with a conventional alpha level of 5% and 90% power.

Supporting Documents and Materials

IRB