Stay smart: Helping consumers choose cyber secure smart devices

Last registered on August 04, 2022

Pre-Trial

Trial Information

General Information

Title
Stay smart: Helping consumers choose cyber secure smart devices
RCT ID
AEARCTR-0008044
Initial registration date
August 12, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 13, 2021, 11:58 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
August 04, 2022, 11:42 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Behavioural Economics Team of the Australian Government

Other Primary Investigator(s)

PI Affiliation
BETA

Additional Trial Information

Status
Completed
Start date
2021-07-27
End date
2021-08-11
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
’Smart devices’ are products with extra functionality to connect to the internet (like smart lights, TVs, and watches). Internationally, cyber security labels have been introduced to help consumers choose smart devices that are more cyber secure. BETA is collaborating with the Department of Home Affairs to evaluate the effectiveness of cyber security labels in the Australian context, to test how they influence people’s purchasing decisions, and how they increase awareness about the risks of insecure smart devices.
External Link(s)

Registration Citation

Citation
Team Registration, BETA and Hanne Watkins. 2022. "Stay smart: Helping consumers choose cyber secure smart devices." AEA RCT Registry. August 04. https://doi.org/10.1257/rct.8044
Experimental Details

Interventions

Intervention(s)
We are testing the influence of cyber security labels on people’s ‘purchasing’ decisions in an online shopping scenario.
Intervention Start Date
2021-07-27
Intervention End Date
2021-08-11

Primary Outcomes

Primary Outcomes (end points)
Details of the outcome variables are kept private until the trial is complete.
Primary Outcomes (explanation)
Details of the outcome variables are kept private until the trial is complete.

Secondary Outcomes

Secondary Outcomes (end points)
Details of the outcome variables are kept private until the trial is complete.
Secondary Outcomes (explanation)
Details of the outcome variables are kept private until the trial is complete.

Experimental Design

Experimental Design
Details of the experimental design are kept private until the trial is complete.
Experimental Design Details
assigned to shop for one of four product categories – smart light bulb, smart watch, home hub or smart TV. When participants “shop” for the smart devices, they will be shown three devices (e.g., three TVs, three watches, etc) in a choice set. The devices will vary in price, features, and level of cyber security. Participants will be asked to choose the one they prefer, making trade-offs between the devices’ attributes. (They can also respond ‘none of these’.) Then they will be shown a new choice set, with three different devices. They will repeat this task ten times.

Some of the devices will be displayed with a cyber security label. Participants will be randomly assigned to see one of the three cyber security labels. The type of label is a between-subjects variable.
Randomization Method
The RCT component of this trial is an individually randomised online experiment, with repeated measures. Participants will be randomised to 1 of 3 cells (corresponding to three different label types). Randomisation will be done by Qualtrics (the survey software), by giving each participant a 1/3 probability of being assigned to each trial arm. We will use an option that Qualtrics provides, to prevent cell sizes becoming too uneven.
This study also has a Discrete Choice Experiment component, included in our internal pre-registration document (see attached).
Randomization Unit
The randomisation unit for the RCT component is the individual.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
6,000 individuals
Sample size: planned number of observations
60,000 choice sets (10 per individual)
Sample size (or number of clusters) by treatment arms
2,000 (individuals are randomly assigned to 1 of 3 treatments)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Here, we present power calculations for the RCT component of the research project only. Due to resource and timing constraints, our sample is fixed at around 6,000 individuals, which will provide 2,000 individuals per group. Each individual will respond to 10 choice sets. To account for repeated measures, we will cluster our standard errors by individual. For the sake of these power calculations, we assume an ICC of 1. This is very conservative with any reduction in this correlation reducing the minimum detectable effect at a given power level/sample size. For this study, alpha is set to 0.05, and beta to 0.2, and hypothesis tests will be one-sided. With these assumptions in place, for H1a and H1b, which both relate to the RCT component, we estimate that our design can detect a standardised effect of 0.08 (Cohen’s h) with 80% power, this corresponds to approximately a 4 percentage point increase from a conservative 50% baseline.
IRB

Institutional Review Boards (IRBs)

IRB Name
Macquarie University Human Research Ethics Committee
IRB Approval Date
2021-06-30
IRB Approval Number
NA
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
August 11, 2021, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
August 11, 2021, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
5,943 individuals
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
5,943 individuals
Final Sample Size (or Number of Clusters) by Treatment Arms
1987 in plain text expiry condition; 1971 in icon expiry condition, 1985 in graded shield condition
Data Publication

Data Publication

Is public data available?
No

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials