Digital well-being (Benessere Digitale). Testing an integrated training package aiming at improving digital competence in high schools

Last registered on May 05, 2021
View Trial History

Pre-Trial

Trial Information

General Information

Title
Digital well-being (Benessere Digitale). Testing an integrated training package aiming at improving digital competence in high schools
RCT ID
AEARCTR-0002990
Initial registration date
May 23, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 23, 2018, 5:01 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 05, 2021, 11:21 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
Italy
Region
Lombardy

Primary Investigator

Name
Marco Gui
Affiliation
University of Milano-Bicocca
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Gianluca Argentin
PI Affiliation
Catholic University of Milan
Contact Investigator
PI Name
Tiziano Gerosa
PI Affiliation
University of Milano-Bicocca
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2017-10-30
End date
2018-10-31
Keywords
Education, Other
Additional Keywords
ICT, media education, digital competence, digital well-being, digital citinzenship, schools, teachers, students, high-school, upper secondary education, digitial inclusion, Internet use, smartphone, digital overuse
JEL code(s)
I2 - Education and Research Institutions
Secondary IDs
Abstract
Recent literature on media education and European policy documents point out that there is an urgent need to develop digital literacy interventions for adolescents. This is particularly true for Italy, were the new digital agenda for school developed by the Ministry of Education (Piano Nazionale Scuola Digitale), as well as recent OECD research reports, underline the absence of solid and shared strategies for the development of digital competence in the school system, and the need to develop new theoretical and methodological educational standards.
Drawing on the European digital competence framework for citizens (DigComp 2.0) and on the support of a team of expert from different disciplines and fields (sociologists, psychologists, media educators, experienced teachers and school principals), the "Digital well-being" project provides secondary school teachers the opportunity to participate in a free training initiative aimed at developing digital competence among their students.
The intervention is delivered in a blended learning environment and deals with the four most relavent themes in digital competence research: 1) searching and evaluating information online, 2) managing online identity and relationships, 3) creating effective digital content; 4) managing time online within today's communicative overabundance.
To evaluate the impact of this training course on students’ digital competence and well-being, we enrolled all 10th grade classes (N=171) within 18 self-selected high schools located in the provinces of Milano and Monza-Brianza (Lombardy, Italy). Following a cluster randomization approach based on 33 blocks of classes defined by course of studies within schools, we identified a total of 41 treated units out of 171. Teachers of the treated classes were left free to participate in the first session of the digital well-being training course (2017-2018 school year), with a limit of two participants per class. Teachers working in the remaining 130 classes (control group) were excluded from the first training session, but they will be given the opportunity to receive our intervention later, in the following school year (2018-2019).
Our RCT is designed to evaluate whether teacher participation to the first session of the digital well-being training course significantly affect one or more of the following primary outcomes measured at the student level: digital knowledge and skills, smartphone pervasiveness in daily life, problematic digital media use, types of internet use and perceived outcomes of internet use. Moreover, we also test the additional hypotheses that higher levels of digital competence and digital well-being could contribute to increase students’ general well-being and academic achievement. To do so, we collect a secondary set of outcomes measuring students’ happiness, satisfaction with life, average grade points and standardized test scores.
External Link(s)
Website with a description of the trial in italian targeted to teachers
Website of the Research Center of Quality of Life in the Digital Society. The report of the project is published in the "publications" section

Registration Citation

Citation
Gui, Marco, Gianluca Argentin and Tiziano Gerosa. 2021. "Digital well-being (Benessere Digitale). Testing an integrated training package aiming at improving digital competence in high schools." AEA RCT Registry. May 05. https://doi.org/10.1257/rct.2990-2.0
Former Citation
Gui, Marco, Gianluca Argentin and Tiziano Gerosa. 2021. "Digital well-being (Benessere Digitale). Testing an integrated training package aiming at improving digital competence in high schools." AEA RCT Registry. May 05. https://www.socialscienceregistry.org/trials/2990/history/91148
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The digital well-being intervention follows the guidelines offered by the European framework for the development of educational initiatives aimed at fostering citizens’ digital competence (DigComp 2.0). This reference framework has been translated into a training proposals for Italian high school teachers by a steering group made up of some of the most prominent experts in the field of media education at the national level, together with a selected group of experienced teachers and school principals.

The resulting training package is organized in four learning modules: digital well-being (online reputation, privacy management, economic value of personal information, awareness of the time spent with digital devices and media); communication and cooperation (management of conflicts in social networks, digital identity, online cooperation, online behavior styles); information research and management (how to search for information, reliability of sources, knowledge online management); creation of digital contents (contents production and sharing, authorship and copyright, news and fake news dissemination).

Each of the training modules is delivered to teachers in a blended learning environment, within which they are in charge of carrying out the following activities.
Preparation (online). It consists in an initial moment of self-training based on introductory materials made available on the online platform of the course (video clips, docs and other open educational resources).
In-presence lesson (in training poles). Interactive lesson in which experts of the field deepen the module contents and offer practical examples on how to translate them into educational activities to be carried out with students in class.
Deepening (online). Self-study of the materials needed to design the intervention plan for the activities with students in class (optional – based on personal needs).
Classroom activity (at school). Teachers are required to carry out a 3-hour classroom activity for each module and plan a series of small tasks that students must perform in their leisure time.
Habit establishment (at school). At the end of the classroom activity, teachers define together with students a series of good habits related to the learning module to be observed since then inside and outside the school. Teachers must also implement habits reinforcement strategies throughout the entire school year (reflection, practical applications, etc.)
Feedback (online). Finally, in line with the action-research framework in which the digital well-being project has been developed, teachers are requested to fill a feedback questionnaire to evaluate and express their opinions on the quality, feasibility and effectiveness of the training materials and the classroom activity.
Intervention Start Date
2017-12-04
Intervention End Date
2018-04-30

Primary Outcomes

Primary Outcomes (end points)
Measured two times, at the beginning and the end of the first school year (before and after the intervention):
- Digital knowledge and skills
- Smartphone pervasiveness in students’ daily life
- Problematic digital media use
- Types of internet use
- Tangible outcomes of internet use
- Perceived teacher support in digital media use (Manipulation check)
Primary Outcomes (explanation)
Outcomes description
- Digital knowledge and skills. Multiple-choice test aimed at measuring the cognitive domain of competence (knowledge and skills) in four strictly interrelated areas: information, communication, creation and safety.
- Smartphone pervasiveness in students’ daily life. Measure derived by a battery of items asking students how frequently they usually make use of their smartphone in relevant moments of the day.
- Problematic digital media use. A battery of items focused on students’ perceptions of the cognitive, relational and physiological consequences of their digital media usage habits.
- Types of internet use. Derived from a battery of items aimed at identify students’ internet usage habits in terms of frequency over time.
- Tangible outcomes of internet use. A battery of items aimed at measuring student’s perceived ability in translating engagement with the web into tangible outcomes.
- Teacher support in digital media use. A manipulation check index based on a battery of item measuring student’s perceived support from teachers in digital media use.
All the outcomes are measured either by means of validated measures found in the literature or with newly constructed measures based on existing theory and evidence.

Scaling procedures
Dimensionality and local independence of the Digital knowledge and skills outcome are tested through structural equation modeling techniques, while its calibration and validation are conducted through Item Response Theory (IRT).
All the other outcomes are obtained from batteries of items following two alternative scaling methods. Outcomes derived from previously developed scales are validated on the respondents’ sample through the Confirmatory Factor Analysis technique (CFA), while the newly developed measures are validated following a two-step procedure, comprising Exploratory Factor Analysis (EFA) and, subsequently, CFA. Moreover, we evaluate internal reliability and measurement invariance of all the construct across groups of treatment.

Secondary Outcomes

Secondary Outcomes (end points)
Measured two times, at the beginning and the end of the first school year (before and after the intervention):
- General well-being

Measured only at the end of the first school year:
- Students’ achievement (administrative data on students’ average grades)
- Students’ performance in national standardized assessment (administrative data from INVALSI)

Secondary Outcomes (explanation)
Outcomes description
- General well-being is measured through a global index of satisfaction with life and indicators of happiness toward specific life domains.
- Students’ achievement is measured through average grades in Italian language and math coming from administrative data collected at the national level.
- The INVALSI standardized test consists in a written test that aims at evaluating students’ achievement in Italian language and math of all the Italian students at different school grades.

Scaling procedures
General well-being measures derives from batteries of item and are validated through confirmatory factor analysis techniques, while students’ average grades and standardized test scores are managed following the guidelines offered by data providers.

Experimental Design

Experimental Design
All high schools located in three different districts of the provinces of Milano and Monza-Brianza (school districts 23, 27 and 28) were invited to participate in the project. At the end of the recruitment stage, 18 schools out of 42 accepted to be involved in the initiative. All together, these schools count a total of 171 10th grade classes, belonging to 5 main courses of study (COS):
- lyceum (scientific area);
- lyceum (other areas);
- technical institute (economic area);
- technical institute (technological area);
- vocational institutes.
Some of the schools only have one COS, while others offer two or more COS. To guarantee a direct involvement of all the selected schools in the first year of intervention and, at the same time, ensure an equal distribution of the 10th grade classes from each of the above mentioned COS between treated and control groups, we opted for a cluster randomization approach based on blocks of classes defined by COS within each of the participant schools.
We then defined 31 intra-school blocks (COS within each school) and, in addition, we created other 2 inter-school blocks to cover also COS that counts only 1 class within a specific school.
Classes have been randomly assigned to the treated and control groups within each of the 33 blocks using the following rules:
- draw 1 treated class if the block counts 6 classes or less;
- draw 2 treated classes if the block counts 7 classes or more.
After randomization, schools were asked to organize separate meetings for teachers of each of the treated classes, during which they had the opportunity to collegially select the two candidates taking part to the training since the beginning of the 2017-2018 school year. On the other side, teachers of the control classes will simply have the opportunity to receive our intervention one year later, in the 2018-2019 school year.
Experimental Design Details
Randomization Method
Randomization done in office with a computer
Randomization Unit
Classes within intra-school and inter-school blocks defined by COS
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
171 classes within 33 blocks (31 intra-school + 2 inter-school)
Sample size: planned number of observations
Around 3420 10th grade students
Sample size (or number of clusters) by treatment arms
41 treated classes (around 820 students)
130 control classes (around 2600 students)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Conservative estimates for students’ primary outcomes around 0,106 expressed in terms of effect size
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethical Committee - University of Milano-Bicocca
IRB Approval Date
2017-06-26
IRB Approval Number
CE 372
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
April 30, 2018, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
May 31, 2018, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
171 classes recruited and randomized: 41 treated classes and 130 control classes
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
2,997 students (171 classes)
Final Sample Size (or Number of Clusters) by Treatment Arms
695 treated students (41 classes) 2,302 control students (130 classes)
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials