Back to History Current Version

Using Emails Effectively for Sharing Cyber Security Advice

Last registered on March 17, 2020

Pre-Trial

Trial Information

General Information

Title
Using Emails Effectively for Sharing Cyber Security Advice
RCT ID
AEARCTR-0005501
Initial registration date
February 26, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 02, 2020, 3:49 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
March 17, 2020, 7:59 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Behavioural Economics Team of the Australian Government

Other Primary Investigator(s)

PI Affiliation
Lead analyst

Additional Trial Information

Status
Completed
Start date
2020-02-27
End date
2020-03-05
Secondary IDs
Abstract
The Australian Cyber Security Centre (ACSC) provides cyber security advice to help make Australia the safest place to connect online. BETA is working with the ACSC to test novel ways to present information about cyber security advice in a way that best helps people and organisations to enact that advice.
The Stay Smart Online alert service is an email subscription that sends alerts whenever a major cyber incident has occurred. The aim of the alert service is to provide timely advice to members of the public either to prevent them from falling victim to the incident, or helping them to rectify the problem if they have been a victim, by providing advice and next steps.
This trial aims to test the effect of different behavioural concepts to see email design elements increase the likelihood that people will engage with the advice, and share it with their friends and family.
External Link(s)

Registration Citation

Citation
Bennetts-Kneebone, Laura and BETA Team Registration. 2020. "Using Emails Effectively for Sharing Cyber Security Advice." AEA RCT Registry. March 17. https://doi.org/10.1257/rct.5501-1.1
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This trial is an individually randomised email experiment conducted online as part of a cyber-security alert service.
Intervention Start Date
2020-02-27
Intervention End Date
2020-03-05

Primary Outcomes

Primary Outcomes (end points)
We will measure two primary outcomes.
1. Sharing: the rate at which individuals share the email
2. Engagement: the rate at which individuals engage with content within the email.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Rates of printing/saving copies of the email.
Secondary Outcomes (explanation)
All treatment arms have new functionality to print and/or save the email as a PDF version. We will apply the specified hypothesis tests and models to a secondary outcome, which looks at what effects our treatments have upon the rate of printing or saving the emails.

Experimental Design

Experimental Design
The trial is a 2x3 factorial design with two independent variables. One with two levels relating to altruism, the other with three relating to salience.
Experimental Design Details
All subscribers to the alert service will receive an email outlining prominent online security threats from 2019. The trial is a 2x3 factorial design, thus we will have two independent variables (IV), one with two levels, the other with three:
IV. A: Call to Share (Binary) – emails either will or will not have a banner calling the recipient to share the email alert with acquaintances
IV. B: Visual Design – emails will have one of three variants of visual design.
- No icon: a BAU form of the email with no icon at the top
- Timing: a salient icon indicating the urgency required to protect oneself against the threat described by the alert
- Action: a salient icon indicating the action the recipient need take to protect themselves against the threat described by the alert.
Randomization Method
Seeded random number allocation at an individual unit level, performed by STATA.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
60,508 (roughly 25,900 after expected exclusion of missing data)
Sample size (or number of clusters) by treatment arms
A0B0: 10084
A0B1: 10085
A0B2: 10085
A1B0: 10085
A1B1: 10084
A2B2: 10085
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
BETA
IRB Approval Date
2019-09-11
IRB Approval Number
N/A
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials