Banking prompts trial

Last registered on December 20, 2024

Pre-Trial

Trial Information

General Information

Title
Banking prompts trial
RCT ID
AEARCTR-0015045
Initial registration date
December 17, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 20, 2024, 1:53 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Behavioural Economics Team of the Australian Government

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2024-12-03
End date
2025-08-29
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The Treasury is advising the government on its responses to the ACCC’s Retail Deposits Inquiry and Home Loan Price Inquiry. Despite the financial benefits that can result from switching products, the reports found a lack of consumer engagement in these markets. For both markets the ACCC recommended that prompts be considered encouraging consumers to see if they could benefit from switching to alternative products that might better suit their needs.

BETA is working with the Treasury to: (1) identify the relevant behavioural factors in consumer financial decision-making; (2) understand the utility of prompts; (3) determine the optimal design for the prompts proposed in the ACCC reports; and (4) understand the barriers and enablers consumers experience when switching or repricing to identify additional interventions which may complement, or be an alternative to, the use of prompts.

BETA is conducting a suite of research activities to support this work, including a field trial (‘trial’) in partnership with a large retail bank. This trial will specifically test a single prompt designed to encourage disengaged customers to engage with their bank to negotiate a better deal on their home loan. This will provide evidence informing the effectiveness of prompts to encourage engagement in a relatively disengaged cohort.

Trial Aims
The aim of the trial is to test the effectiveness of a prompt to encourage disengaged customers to call the bank to see if they are eligible to receive a lower interest rate on their loan.
External Link(s)

Registration Citation

Citation
Team Registration, BETA. 2024. "Banking prompts trial ." AEA RCT Registry. December 20. https://doi.org/10.1257/rct.15045-1.0
Experimental Details

Interventions

Intervention(s)
Participants in the trial will see a prompt surfaced in their banking app if they are in the intervention arm. Control participants will not see the prompt. We will use bank administrative data to measure the primary and secondary outcomes.
The text of the prompt reads “Interested in reducing your mortgage repayments? Check whether you can get a lower interest rate on your home loan”. Customers can call the bank directly from the prompt or through their usual channels.
Intervention Start Date
2024-12-03
Intervention End Date
2024-12-17

Primary Outcomes

Primary Outcomes (end points)
Customer contact: Measured at both the cluster and individual level. The cluster-level analysis will examine whether any customer within each cluster contacted the bank, while the individual-level analysis examines whether each individual customer contacted the bank.
Primary Outcomes (explanation)
Cluster-level customer contact rate: The primary outcome variable is a binary indicator at the cluster level, taking the value of 1 if at least one customer telephoned the bank on their general contact line or their specialist home loan line in the trial period, and 0 otherwise. This definition captures cluster-level engagement with the intervention, recognising that for shared loans, one borrower making contact is sufficient. The variable will be constructed from administrative data measuring the count of calls made to the bank in the trial period. This will be averaged within the treatment and control groups to give the proportion of clusters in which at least one person made contact with the bank in each arm. The outcome will be measured at T1.

Individual customer contact rate: At an individual level the outcome will be a binary indicator, flagging whether a customer telephoned the bank on their general contact line or their specialist home loan line in the trial period (0 = did not make contact, 1 = made contact). The variable will be constructed from administrative data measuring the count of calls made to the bank in the trial period. This will be averaged within treatment and control group to give the proportion of customers who made contact with the bank by arm. This outcome will be measured at T1.

Secondary Outcomes

Secondary Outcomes (end points)
Interest rate, repricing, financial stress, financial wellbeing
Secondary Outcomes (explanation)
Interest rate: A variable at the cluster level, the interest rate outcome will be a continuous variable representing the mean interest rate of all loans in a cluster expressed as a percentage. This will be averaged within treatment groups to give the mean interest rate by arm. This outcome will be measured at T1.

Repricing: A binary variable at the cluster level of whether at least one loan in a cluster was repriced during the intervention period. This will be constructed from administrative data, with 1 indicating at least one loan was re-priced and 0 if it was not. For shared loans, the outcome captures whether the shared loan was re-priced regardless of which borrower initiated the change. This will be averaged within treatment and control groups to give the proportion of clusters with reduced rates by arm. This outcome will be measured at T1.

Financial stress: A binary indicator at the cluster level of whether at least one person in a cluster has experienced financial stress (0 = no financial stress, 1 = financial stress). This will be any of:
• Hardship support requests, OR
• Late payment fees for any of their loans, OR
• Arrears on any of their loans
This will be averaged within treatment groups to give the proportion of clusters in financial stress by arm. This outcome will be measured at T2 and T3.

Financial wellbeing: At the cluster level we will measure the total balance of available accounts such as deposit accounts, investment accounts, and shares held by each individual in the cluster at T2 and T3. The construction of this variable will depend on account information that is shared by the bank partner. This will be averaged within treatment groups to give the mean balance in each arm.

Experimental Design

Experimental Design
This project will be a 2-arm cluster randomised controlled field trial. Clusters will be defined in the administrative data as people who share loans, defined through inter-related person-loan pairs. This is because the intervention will be delivered to individuals, and the outcomes will relate to both people (e.g. calling the bank) and loans (e.g. rate reductions).
Clusters will be randomly assigned to either the treatment group or control group with approximately equal number in each arm. The randomisation is performed at the cluster level to ensure that all members of a cluster receive the same treatment condition.
After initial randomisation, stakeholders will remove 10% of individuals from the treatment group as part of their automated system.
The primary analysis will be conducted at the cluster level; i.e., whether at least one individual in a cluster takes action in response to the intervention. Analysis at the individual level represents an upper bound.
Outcomes will be measured at three time points. Timepoint 1 (T1) will be 7 days after the close of the prompt on 10 December 2024 in the mobile application (17 December 2024). Timepoint 2 (T2) will be 3 months post T1. Timepoint 3 (T3) will be 3 months post T2. T3 outcomes will explore longer-term effects of the intervention on financial wellbeing of participants. T2 outcomes will be exploratory.
Experimental Design Details
Not available
Randomization Method
Randomisation will be at the cluster level. Clusters of connected customers will be randomly assigned to treatment and control groups in a 1:1 ratio. The clusters are flexible structures able to manage a multitude of individual arrangements and the many-to-many relationship between people and loans.
After initial treatment assignment, 10% of customers from the treatment group will be randomly selected to not receive the intervention as part of the stakeholder’s automatic program implementation. The random selection will occur at the individual customer level, not the cluster level. For single-customer clusters, the entire network will not receive the intervention if selected. For multi-customer clusters, only the selected customers will not receive the intervention. All people and clusters will be analysed as initially allocated to maintain the integrity of clusters post-randomisation.
Randomization Unit
Randomisation will be at the cluster level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
The planned number of clusters for this trial is 11,922 (defined as unique groups of individuals and their shared loans).
Sample size: planned number of observations
18,769 person-loan pairs, representing the total number of unique observations in the dataset
Sample size (or number of clusters) by treatment arms
Treatment group: 5,961 clusters
Control group: 5,961 clusters
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
This study has a fixed sample size and will analyse 11,922 loan clusters. Using a joint decision framework at the cluster level and accounting for the 10% of individuals that will be randomly removed from receiving the intervention, our estimation approach (which treats a loan cluster as treated if at least one individual remains in the treatment group) indicates we will have a minimum detectable effect of 0.78 percentage points. Historical click-through rates for interventions using this type of notification shell are approximately 1%. Therefore, with our sample size, we will be able to detect an increase from 1% to 1.78%, representing a 78.2% relative increase. This trial will be adequately powered to detect these differences using a one-sided test with a conventional alpha level of 5% and 90% power.
IRB

Institutional Review Boards (IRBs)

IRB Name
Macquarie University Humanities and Social Sciences Human Research Ethics Committee
IRB Approval Date
2024-11-18
IRB Approval Number
18484
Analysis Plan

Analysis Plan Documents

20241217_field_trial_PAP.docx

MD5: fbcfebfcc05325dcf2c0d73f75b83b36

SHA1: 4e5a03ecb4b962598994ef21840cd0ca70812673

Uploaded At: December 17, 2024