NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Using Emails Effectively for Sharing Cyber Security Advice
Last registered on April 08, 2020


Trial Information
General Information
Using Emails Effectively for Sharing Cyber Security Advice
Initial registration date
February 26, 2020
Last updated
April 08, 2020 8:22 PM EDT
Primary Investigator
Behavioural Economics Team of the Australian Government
Other Primary Investigator(s)
PI Affiliation
Lead analyst
Additional Trial Information
Start date
End date
Secondary IDs
The Australian Cyber Security Centre (ACSC) provides cyber security advice to help make Australia the safest place to connect online. BETA is working with the ACSC to test novel ways to present information about cyber security advice in a way that best helps people and organisations to enact that advice.
The Stay Smart Online alert service is an email subscription that sends alerts whenever a major cyber incident has occurred. The aim of the alert service is to provide timely advice to members of the public either to prevent them from falling victim to the incident, or helping them to rectify the problem if they have been a victim, by providing advice and next steps.
This trial aims to test the effect of different behavioural concepts to see email design elements increase the likelihood that people will engage with the advice, and share it with their friends and family.
External Link(s)
Registration Citation
Bennetts Kneebone, Laura and BETA Team Registration. 2020. "Using Emails Effectively for Sharing Cyber Security Advice." AEA RCT Registry. April 08. https://doi.org/10.1257/rct.5501-1.2000000000000002.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
This trial is an individually randomised email experiment conducted online as part of a cyber-security alert service.
Intervention Start Date
Intervention End Date
Primary Outcomes
Primary Outcomes (end points)
We will measure two primary outcomes.
1. Sharing: the rate at which individuals share the email
2. Engagement: the rate at which individuals engage with content within the email.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Rates of printing/saving copies of the email.
Secondary Outcomes (explanation)
All treatment arms have new functionality to print and/or save the email as a PDF version. We will apply the specified hypothesis tests and models to a secondary outcome, which looks at what effects our treatments have upon the rate of printing or saving the emails.
Experimental Design
Experimental Design
The trial is a 2x3 factorial design with two independent variables. One with two levels relating to altruism, the other with three relating to salience.
Experimental Design Details
All subscribers to the alert service will receive an email outlining prominent online security threats from 2019. The trial is a 2x3 factorial design, thus we will have two independent variables (IV), one with two levels, the other with three: IV. A: Call to Share (Binary) – emails either will or will not have a banner calling the recipient to share the email alert with acquaintances IV. B: Visual Design – emails will have one of three variants of visual design. - No icon: a BAU form of the email with no icon at the top - Timing: a salient icon indicating the urgency required to protect oneself against the threat described by the alert - Action: a salient icon indicating the action the recipient need take to protect themselves against the threat described by the alert.
Randomization Method
Seeded random number allocation at an individual unit level, performed by STATA.
Randomization Unit
Was the treatment clustered?
Experiment Characteristics
Sample size: planned number of clusters
Sample size: planned number of observations
60,508 (roughly 25,900 after expected exclusion of missing data)
Sample size (or number of clusters) by treatment arms
A0B0: 10084
A0B1: 10085
A0B2: 10085
A1B0: 10085
A1B1: 10084
A2B2: 10085
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Post Trial Information
Study Withdrawal
Is the intervention completed?
Is data collection complete?
Data Publication
Data Publication
Is public data available?
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)