Experimental evidence on Educational Tips and Warnings within a Digital Payment App

Last registered on January 08, 2025

Pre-Trial

Trial Information

General Information

Title
Experimental evidence on Educational Tips and Warnings within a Digital Payment App
RCT ID
AEARCTR-0015071
Initial registration date
December 20, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 20, 2024, 3:16 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
January 08, 2025, 11:56 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Central Bank of Ireland

Other Primary Investigator(s)

PI Affiliation
Central Bank of Ireland
PI Affiliation
Central Bank of Ireland
PI Affiliation
Trinity College Dublin
PI Affiliation
Trinity College Dublin

Additional Trial Information

Status
In development
Start date
2024-12-21
End date
2025-05-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Financial scams impose huge costs on consumers and the financial industry. Consumer education is viewed as one part of a wider toolkit to address fraud, particularly given the growth in authorised push payment fraud (APP), where consumers unwittingly authorise digital payments to fraudsters (BPFI, UK Finance). Through an online trial, we intend to test the causal effects of providing generic or scam specific fraud education tips, or a visual fraud warning, just before the payment confirmation step within a simulated banking app. We will disentangle which, if any, of these interventions assist consumers to detect fraudulent scenarios, and whether any of the methods are more or less effective. Additionally, we will test which methods are associated with greater consumer comprehension and overconfidence, as well as whether effectiveness varies with participants online risky payment behaviour. Our results will aid policymakers in their efforts to protect consumers from payment fraud.
External Link(s)

Registration Citation

Citation
Gulesci, Selim et al. 2025. "Experimental evidence on Educational Tips and Warnings within a Digital Payment App." AEA RCT Registry. January 08. https://doi.org/10.1257/rct.15071-1.1
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2024-12-23
Intervention End Date
2025-02-28

Primary Outcomes

Primary Outcomes (end points)
1. Proportion of participants who confirm a payment for legitimate and illegitimate scenarios separately.
2. Proportion of participants who confirm a payment for each scenario.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
1. Comprehension in identifying scams – a fraud comprehension module which measures overall fraud detection literacy of participants.
2. Confidence in Payment Decisions - A self-reported measure of how confident participants are in their payment decisions
3. Measure of the proportion of participants who successfully cancelled all payments for illegitimate scenarios and confirmed all payments for legitimate scenarios.

Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Control Group (C) - Standard payment journey - Participants in this group will complete a standard mobile payment journey consisting of no more than 2–3 payment steps for each scenario. This group serves as the baseline comparison, allowing for the measurement of differences between the control and treatment groups.
Treatment 1 (T1) - Generic fraud tips within the mobile payment journey - Participants will complete a standard mobile payment journey similar to the control group but with the addition of generic fraud tips embedded within the journey. These tips will be displayed just before the payment confirmation step. The generic fraud tips have been designed based on fraud prevention material available on the Central Bank of Ireland’s website and provide broad guidance on identifying fraudulent activities.
Treatment 2 (T2) - Fraud specific tips within the mobile payment journey - Participants in this group will receive fraud tips tailored to the specific type of fraud presented in each scenario. These scenario specific tips are embedded within the payment journey and are shown just before the payment confirmation step. The content of these tips has been developed by reviewing the tell-tale signs of common fraud types, ensuring relevance and specificity for each scenario.
Treatment 3 (T3) – Visual Fraud Warning within the mobile payment journey - In this group, participants will complete a standard mobile payment journey similar to the control group but with the addition of a visual fraud warning. This warning embeds behavioural principles (Wang et al., 2022) and is designed to alert participants that a specific payment may be fraudulent. There will be a small element of text, warning participants that they could lose their money but without any specific capacity building tips. Unlike the generic or scenario-specific fraud education tips, the visual warning employs a visual approach to emphasise the risk associated with certain payments.

Experimental Design Details
Not available
Randomization Method
Participants are randomised to either a control group or one of the three treatment groups based on randomisation done by the Gorilla experiment builder tool.
Randomization Unit
The unit of randomisation is at individual level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
None
Sample size: planned number of observations
3000 nationally representative individuals
Sample size (or number of clusters) by treatment arms
750 in each of the experimental arms
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With a sample of 3,000 nationally representative individuals, which are equally split between one control and three treatment groups (resulting in 750 participants in each of the four study groups), the study is powered enough to detect a minimum effect size of 3.53 percentage points or 16%.
IRB

Institutional Review Boards (IRBs)

IRB Name
Central Bank of Ireland
IRB Approval Date
2024-11-22
IRB Approval Number
N/A
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information