Scalable Cyber Interventions Accelerating Productivity Practice for SMEs' programme (Cyber Well) explores whether the use of an online game can enhance the productivity of Dorset SMEs through encouraging the adoption of better data management and cyber security practices.

Scalable Cyber Interventions Accelerating Productivity Practice for SMEs' programme (Cyber Well) is a Randomised Control Trial to test whether the use of online gamification can enhance the productivity of Dorset SMEs through encouraging the adoption of better data management and cyber security practices. Cyber Well is an integrated training and self-delivered programme building on four Proof of Concept studies in Dorset. The studies showed evidence that SMEs' growth and productivity can suffer directly from poor digital data management and careless cyber security behaviour. Furthermore, evidence shows that SMEs are unlikely to win new business contracts that increasingly require national security and data management accreditation. Conversely, those that have good practices through a combination of new contracts, compliance certification and organisational behaviour change increase productivity.

The primary outcomes being evaluated are whether the deployment of a cyber game with nudge learning increase the cyber and data knowledge resilience in SMEs, and whether this style of teaching encourages attitudinal changes and increases productivity in SMEs related to cyber behaviours and certification in a more effective way than typical cyber ‘push-learning’ training.

The trial design is a parallel group, two-arm superiority trial with 1:1 allocation ratio of Control: Intervention (aim: 150 SMEs in each). The project team ran three batches of delivery. The first batch started in March 2020 and the final batch in August 2020. Participant eligability requirements included that they were: South England based; an SME; and had enough de Minimis allocation.

The main comparison will be between the Intervention and the Control groups. This between groups comparison will take place following each quantitative data collection phase, i.e. at baseline, at initial follow-up and at two-month follow-up. Due to the use of block randomisation the team will also be able to compare performance across different sized businesses and across different job roles (IT or non-IT role) within each arm. In addition, within group comparisons across the three time points of baseline, initial follow-up and two-month follow-up and for each arm will be conducted. This may be done as a factorial analysis, depending on if the data meets the requirements for such a test (see below).

The analysis to be done will be determined by the normality and distribution of the data. In the requirements for ANOVA are met then the survey items with continuous outcomes such as most of the survey attitudes questions will be analysed using a 2 x 3 mixed ANOVA (intervention vs control/ baseline to initial follow-up to two—month follow-up). Post-hoc tests main effects and interactions will be conducted as appropriate.If the requirements for ANOVA are not met, then the Friedman test will be used to determine changes on continuous responses within each arm over the three quantitative data collection time points. Mann-Whitney will be used to make comparisons between each arm at each data collection point, with Bonferroni corrections applied as appropriate. Chi-square analysis will be used for any outcomes with binary choices, such as some of the true/ false items within the baseline and follow-up surveys. An appropriate variation on a chi-square such as McNemar’s test will be used to establish the statistical significance in any changes from baseline to follow-up.

Descriptive analysis will be used to report outcomes from any response items that use ranking, such as the attitude item on perception of barriers to implementing good cyber security practice. Due to COVID restrictions, the numbers of participants has been much lower than anticipated. As such, the project team decided to interview 20 of the participants who completed the training. This will provide insight into the training and rich evidence to substantiate the quantitative data. The team are currently in the final phase of data collection and will begin writing the final report in the next month.