Stay smart: Helping consumers choose cyber secure smart devices

Last registered on August 13, 2021

View Trial History

Pre-Trial

Trial Information

General Information

Title

Stay smart: Helping consumers choose cyber secure smart devices

RCT ID

AEARCTR-0008044

Initial registration date

August 12, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

August 13, 2021, 11:58 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Australia

Region

Primary Investigator

Name

BETA Team Registration

Affiliation

Behavioural Economics Team of the Australian Government

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Hanne Watkins

PI Affiliation

BETA

Contact Investigator

Additional Trial Information

Status

On going

Start date

2021-07-27

End date

2022-08-16

Keywords

Behavior, Governance

Additional Keywords

cyber security, discrete choice experiment, purchasing decisions, regulation

JEL code(s)

C91, Laboratory, Individual Behavior; D91, Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making; D18, Consumer Protection;

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

’Smart devices’ are products with extra functionality to connect to the internet (like smart lights, TVs, and watches). Internationally, cyber security labels have been introduced to help consumers choose smart devices that are more cyber secure. BETA is collaborating with the Department of Home Affairs to evaluate the effectiveness of cyber security labels in the Australian context, to test how they influence people’s purchasing decisions, and how they increase awareness about the risks of insecure smart devices.

External Link(s)

Registration Citation

Citation

Team Registration, BETA and Hanne Watkins. 2021. "Stay smart: Helping consumers choose cyber secure smart devices." AEA RCT Registry. August 13. https://doi.org/10.1257/rct.8044-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

We are testing the influence of cyber security labels on people’s ‘purchasing’ decisions in an online shopping scenario.

Intervention Start Date

2021-07-27

Intervention End Date

2021-08-16

Primary Outcomes

Primary Outcomes (end points)

Details of the outcome variables are kept private until the trial is complete.

Primary Outcomes (explanation)

Details of the outcome variables are kept private until the trial is complete.

Secondary Outcomes

Secondary Outcomes (end points)

Details of the outcome variables are kept private until the trial is complete.

Secondary Outcomes (explanation)

Details of the outcome variables are kept private until the trial is complete.

Experimental Design

Details of the experimental design are kept private until the trial is complete.

Experimental Design Details

assigned to shop for one of four product categories – smart light bulb, smart watch, home hub or smart TV. When participants “shop” for the smart devices, they will be shown three devices (e.g., three TVs, three watches, etc) in a choice set. The devices will vary in price, features, and level of cyber security. Participants will be asked to choose the one they prefer, making trade-offs between the devices’ attributes. (They can also respond ‘none of these’.) Then they will be shown a new choice set, with three different devices. They will repeat this task ten times.

Some of the devices will be displayed with a cyber security label. Participants will be randomly assigned to see one of the three cyber security labels. The type of label is a between-subjects variable.

Randomization Method

The RCT component of this trial is an individually randomised online experiment, with repeated measures. Participants will be randomised to 1 of 3 cells (corresponding to three different label types). Randomisation will be done by Qualtrics (the survey software), by giving each participant a 1/3 probability of being assigned to each trial arm. We will use an option that Qualtrics provides, to prevent cell sizes becoming too uneven.
This study also has a Discrete Choice Experiment component, included in our internal pre-registration document (see attached).

Randomization Unit

The randomisation unit for the RCT component is the individual.

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

6,000 individuals

Sample size: planned number of observations

60,000 choice sets (10 per individual)

Sample size (or number of clusters) by treatment arms

2,000 (individuals are randomly assigned to 1 of 3 treatments)

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Here, we present power calculations for the RCT component of the research project only. Due to resource and timing constraints, our sample is fixed at around 6,000 individuals, which will provide 2,000 individuals per group. Each individual will respond to 10 choice sets. To account for repeated measures, we will cluster our standard errors by individual. For the sake of these power calculations, we assume an ICC of 1. This is very conservative with any reduction in this correlation reducing the minimum detectable effect at a given power level/sample size. For this study, alpha is set to 0.05, and beta to 0.2, and hypothesis tests will be one-sided. With these assumptions in place, for H1a and H1b, which both relate to the RCT component, we estimate that our design can detect a standardised effect of 0.08 (Cohen’s h) with 80% power, this corresponds to approximately a 4 percentage point increase from a conservative 50% baseline.

Supporting Documents and Materials

IRB