Digital Compass: Supporting the ethical development of young people when using communications technology

Last registered on May 02, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
Digital Compass: Supporting the ethical development of young people when using communications technology
RCT ID
AEARCTR-0006173
Initial registration date
July 20, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 20, 2020, 11:36 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 02, 2022, 9:13 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
Australia
Region
New South Wales

Primary Investigator

Name
Vera Newman
Affiliation
Behavioural Insights Team
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Erin Howard
PI Affiliation
Behavioural Insights Team
Contact Investigator
PI Name
Sheridan Hartley
PI Affiliation
Behavioural Insights Team
Contact Investigator
PI Name
Alex Gyani
PI Affiliation
Behavioural Insights Team
Contact Investigator
PI Name
Rory Gallagher
PI Affiliation
Behavioural Insights Team
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2020-07-27
End date
2022-04-01
Keywords
Behavior, Health
Additional Keywords
young people, ethical development, social media, online behaviour, antisocial behaviour, prosocial behaviour, high school, year 9
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Young people use technology to communicate, test out and develop their identities, create and join spaces to reflect their interests and passions, and feel a sense of community with others. Despite these many great possibilities, there is a need for young people to be better equipped to navigate their connected, constantly changing, digital world.
Several programs have been developed and implemented in schools to address this need. The concern with these programs, however, is that many are not evidence-based, nor has their impact been robustly evaluated. Digital Compass is an 8-week evidence-based program designed to support the ethical development of teenagers in a world of technological change. The program has been developed by The Behavioural Insights Team in collaboration with the Vincent Fairfax Family Foundation and The Alannah and Madeline Foundation.
The aim of the current project is to understand whether the Digital Compass program can help to support the ethical development of teenagers, particularly in the online environment. To do so, we have designed a study to evaluate the effectiveness of Digital Compass on teenagers’ self-reported online ethical behaviour. Our primary outcome variables are self-reported prosocial behaviour online, antisocial behaviour online, and attainment of goals related to ethical development identified during the program.
External Link(s)

Registration Citation

Citation
Gallagher, Rory et al. 2022. "Digital Compass: Supporting the ethical development of young people when using communications technology." AEA RCT Registry. May 02. https://doi.org/10.1257/rct.6173-2.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Digital Compass is an 8 week program, targeting Year 9 students, delivered in schools weekly during a 40 minute workshop, by an external facilitator. Digital Compass aims to change online ethical behaviour by helping young people to; self reflect, determine acceptable online social norms, develop their self efficacy to intervene in online conflicts, control their online environment and identify and change online behaviours. The workshops involve online and offline evidence-based exercises, facilitated discussion and personalised online behaviour data compared to their peers to help them identify areas of change.

Digital Compass is designed to provide developmentally and cognitively appropriate support for young people to improve their ethical online behaviour, and build the skills required to navigate the online environment on their own.

Intervention (Hidden)
Intervention Start Date
2020-08-03
Intervention End Date
2020-09-30

Primary Outcomes

Primary Outcomes (end points)
There are two primary outcomes:

1. Did the Digital Compass program impact the students’ prosocial online behaviour?
2. Did the Digital Compass program impact the students’ anti-social online behaviour?
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
The secondary outcomes are listed below:

Did the Digital Compass program impact the students’ self-reflection?
Did the Digital Compass program impact the students’ self-efficacy in relationships?
Did the Digital Compass program impact the students’ empathy?
Did the Digital Compass program impact the students’ judgment of moral vignettes?
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This study will be conducted as a cluster Randomised Controlled Trial, randomised at the class level. Year 9 classes within participating schools will receive the intervention. We will conduct pre- and post-intervention at all schools, to index change in the intervention group, and compare with any change in the control group. This design will allow us to understand the causal impact of the Digital Compass intervention on our key outcomes, but will not be powered to estimate whether the causal impact differs across different demographic groups.

We will randomise at the classroom level, with randomisation taking place before the launch of the intervention. BIT will be responsible for the randomisation. Participants will not be blind to the condition they are in, as those in the intervention group will receive the Digital Compass workshops and those in the control group will not.
Experimental Design Details
Randomization Method
Randomisation will take place via randomisation functions built into R statistical software
Randomization Unit
The randomisation will take place at the class-level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
We plan to implement the intervention in 20 classes, and all additional classes within the schools recruited will serve as control classes (completing data collection activities only) and will make up the bulk of the comparison group.
Sample size: planned number of observations
We plan to implement this intervention in 20 classes, which will be approximately 500 individuals. Assuming that there are approximately three year 9 classes within each school, we will have approximately 1000 individuals in the control group.
Sample size (or number of clusters) by treatment arms
We plan to have 20 classes in the treatment, and with the assumptions detailed above, approximately 40 classes in the control.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We have conducted a power calculation for a number of scenarios for our primary outcome measures, which are continuous survey measures. We believe that this measure will be the most difficult to power of our outcome measures, given the potential for floor effects for these measures (particularly the antisocial measures). Below are the assumptions for our power calculations: ● Significance level: 0.05 ● Power: 80% ● Number of participants: 500 in treatment, and 1000 in control ● Number of treatment arms: 2 (intervention, control) ● Standard deviation of measure: 4, 8, 10 ● ICC: 0.1 With 2 arms, we will be powered to detect between a 1 and 3 point difference in the average value of the scale given our assumptions. Given that the scales are out of 40 or 44, this represents a 2 - 7% increase in the scale. We believe that this trial is adequately powered to investigate our research question, as a difference smaller than this is not policy relevant.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Bellberry
IRB Approval Date
2020-02-24
IRB Approval Number
2019-12-1148
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
September 30, 2021, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
November 30, 2021, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials

Description
Evaluation report
Citation
Gallagher, Rory et al. 2022. "Digital Compass: Supporting the ethical development of young people when using communications technology." AEA RCT Registry. May 02. 2022. "Registration Entry Title: Evaluation report." AEA RCT Registry. May 02 https://doi.org/10.1257/rct.6173-2.0
File
2022-02 - VFFF - Digital Compass Brochure - Experiment Results.pdf

MD5: 641fd6cf3f13033538520e6bf232719f

SHA1: a2b99806f42a208ea0943620252de6da1ebe82df

Uploaded At: May 02, 2022