x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Digital well-being (Benessere Digitale). Testing an integrated training package aiming at improving digital competence in high schools
Last registered on May 23, 2018

Pre-Trial

Trial Information
General Information
Title
Digital well-being (Benessere Digitale). Testing an integrated training package aiming at improving digital competence in high schools
RCT ID
AEARCTR-0002990
Initial registration date
May 23, 2018
Last updated
May 23, 2018 5:01 PM EDT
Location(s)
Region
Primary Investigator
Affiliation
University of Milano-Bicocca
Other Primary Investigator(s)
PI Affiliation
University of Milano-Bicocca
PI Affiliation
Catholic University of Milan
Additional Trial Information
Status
On going
Start date
2017-10-30
End date
2018-10-31
Secondary IDs
Abstract
Recent literature on media education and European policy documents point out that there is an urgent need to develop digital literacy interventions for adolescents. This is particularly true for Italy, were the new digital agenda for school developed by the Ministry of Education (Piano Nazionale Scuola Digitale), as well as recent OECD research reports, underline the absence of solid and shared strategies for the development of digital competence in the school system, and the need to develop new theoretical and methodological educational standards.
Drawing on the European digital competence framework for citizens (DigComp 2.0) and on the support of a team of expert from different disciplines and fields (sociologists, psychologists, media educators, experienced teachers and school principals), the "Digital well-being" project provides secondary school teachers the opportunity to participate in a free training initiative aimed at developing digital competence among their students.
The intervention is delivered in a blended learning environment and deals with the four most relavent themes in digital competence research: 1) searching and evaluating information online, 2) managing online identity and relationships, 3) creating effective digital content; 4) managing time online within today's communicative overabundance.
To evaluate the impact of this training course on students’ digital competence and well-being, we enrolled all 10th grade classes (N=171) within 18 self-selected high schools located in the provinces of Milano and Monza-Brianza (Lombardy, Italy). Following a cluster randomization approach based on 33 blocks of classes defined by course of studies within schools, we identified a total of 41 treated units out of 171. Teachers of the treated classes were left free to participate in the first session of the digital well-being training course (2017-2018 school year), with a limit of two participants per class. Teachers working in the remaining 130 classes (control group) were excluded from the first training session, but they will be given the opportunity to receive our intervention later, in the following school year (2018-2019).
Our RCT is designed to evaluate whether teacher participation to the first session of the digital well-being training course significantly affect one or more of the following primary outcomes measured at the student level: digital knowledge and skills, smartphone pervasiveness in daily life, problematic digital media use, types of internet use and perceived outcomes of internet use. Moreover, we also test the additional hypotheses that higher levels of digital competence and digital well-being could contribute to increase students’ general well-being and academic achievement. To do so, we collect a secondary set of outcomes measuring students’ happiness, satisfaction with life, average grade points and standardized test scores.
Registration Citation
Citation
Argentin, Gianluca, Tiziano Gerosa and Marco Gui. 2018. "Digital well-being (Benessere Digitale). Testing an integrated training package aiming at improving digital competence in high schools." AEA RCT Registry. May 23. https://doi.org/10.1257/rct.2990-1.0.
Former Citation
Argentin, Gianluca et al. 2018. "Digital well-being (Benessere Digitale). Testing an integrated training package aiming at improving digital competence in high schools." AEA RCT Registry. May 23. http://www.socialscienceregistry.org/trials/2990/history/29974.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
The digital well-being intervention follows the guidelines offered by the European framework for the development of educational initiatives aimed at fostering citizens’ digital competence (DigComp 2.0). This reference framework has been translated into a training proposals for Italian high school teachers by a steering group made up of some of the most prominent experts in the field of media education at the national level, together with a selected group of experienced teachers and school principals.

The resulting training package is organized in four learning modules: digital well-being (online reputation, privacy management, economic value of personal information, awareness of the time spent with digital devices and media); communication and cooperation (management of conflicts in social networks, digital identity, online cooperation, online behavior styles); information research and management (how to search for information, reliability of sources, knowledge online management); creation of digital contents (contents production and sharing, authorship and copyright, news and fake news dissemination).

Each of the training modules is delivered to teachers in a blended learning environment, within which they are in charge of carrying out the following activities.
Preparation (online). It consists in an initial moment of self-training based on introductory materials made available on the online platform of the course (video clips, docs and other open educational resources).
In-presence lesson (in training poles). Interactive lesson in which experts of the field deepen the module contents and offer practical examples on how to translate them into educational activities to be carried out with students in class.
Deepening (online). Self-study of the materials needed to design the intervention plan for the activities with students in class (optional – based on personal needs).
Classroom activity (at school). Teachers are required to carry out a 3-hour classroom activity for each module and plan a series of small tasks that students must perform in their leisure time.
Habit establishment (at school). At the end of the classroom activity, teachers define together with students a series of good habits related to the learning module to be observed since then inside and outside the school. Teachers must also implement habits reinforcement strategies throughout the entire school year (reflection, practical applications, etc.)
Feedback (online). Finally, in line with the action-research framework in which the digital well-being project has been developed, teachers are requested to fill a feedback questionnaire to evaluate and express their opinions on the quality, feasibility and effectiveness of the training materials and the classroom activity.
Intervention Start Date
2017-12-04
Intervention End Date
2018-04-30
Primary Outcomes
Primary Outcomes (end points)
Measured two times, at the beginning and the end of the first school year (before and after the intervention):
- Digital knowledge and skills
- Smartphone pervasiveness in students’ daily life
- Problematic digital media use
- Types of internet use
- Tangible outcomes of internet use
- Perceived teacher support in digital media use (Manipulation check)
Primary Outcomes (explanation)
Outcomes description
- Digital knowledge and skills. Multiple-choice test aimed at measuring the cognitive domain of competence (knowledge and skills) in four strictly interrelated areas: information, communication, creation and safety.
- Smartphone pervasiveness in students’ daily life. Measure derived by a battery of items asking students how frequently they usually make use of their smartphone in relevant moments of the day.
- Problematic digital media use. A battery of items focused on students’ perceptions of the cognitive, relational and physiological consequences of their digital media usage habits.
- Types of internet use. Derived from a battery of items aimed at identify students’ internet usage habits in terms of frequency over time.
- Tangible outcomes of internet use. A battery of items aimed at measuring student’s perceived ability in translating engagement with the web into tangible outcomes.
- Teacher support in digital media use. A manipulation check index based on a battery of item measuring student’s perceived support from teachers in digital media use.
All the outcomes are measured either by means of validated measures found in the literature or with newly constructed measures based on existing theory and evidence.

Scaling procedures
Dimensionality and local independence of the Digital knowledge and skills outcome are tested through structural equation modeling techniques, while its calibration and validation are conducted through Item Response Theory (IRT).
All the other outcomes are obtained from batteries of items following two alternative scaling methods. Outcomes derived from previously developed scales are validated on the respondents’ sample through the Confirmatory Factor Analysis technique (CFA), while the newly developed measures are validated following a two-step procedure, comprising Exploratory Factor Analysis (EFA) and, subsequently, CFA. Moreover, we evaluate internal reliability and measurement invariance of all the construct across groups of treatment.
Secondary Outcomes
Secondary Outcomes (end points)
Measured two times, at the beginning and the end of the first school year (before and after the intervention):
- General well-being

Measured only at the end of the first school year:
- Students’ achievement (administrative data on students’ average grades)
- Students’ performance in national standardized assessment (administrative data from INVALSI)

Secondary Outcomes (explanation)
Outcomes description
- General well-being is measured through a global index of satisfaction with life and indicators of happiness toward specific life domains.
- Students’ achievement is measured through average grades in Italian language and math coming from administrative data collected at the national level.
- The INVALSI standardized test consists in a written test that aims at evaluating students’ achievement in Italian language and math of all the Italian students at different school grades.

Scaling procedures
General well-being measures derives from batteries of item and are validated through confirmatory factor analysis techniques, while students’ average grades and standardized test scores are managed following the guidelines offered by data providers.
Experimental Design
Experimental Design
All high schools located in three different districts of the provinces of Milano and Monza-Brianza (school districts 23, 27 and 28) were invited to participate in the project. At the end of the recruitment stage, 18 schools out of 42 accepted to be involved in the initiative. All together, these schools count a total of 171 10th grade classes, belonging to 5 main courses of study (COS):
- lyceum (scientific area);
- lyceum (other areas);
- technical institute (economic area);
- technical institute (technological area);
- vocational institutes.
Some of the schools only have one COS, while others offer two or more COS. To guarantee a direct involvement of all the selected schools in the first year of intervention and, at the same time, ensure an equal distribution of the 10th grade classes from each of the above mentioned COS between treated and control groups, we opted for a cluster randomization approach based on blocks of classes defined by COS within each of the participant schools.
We then defined 31 intra-school blocks (COS within each school) and, in addition, we created other 2 inter-school blocks to cover also COS that counts only 1 class within a specific school.
Classes have been randomly assigned to the treated and control groups within each of the 33 blocks using the following rules:
- draw 1 treated class if the block counts 6 classes or less;
- draw 2 treated classes if the block counts 7 classes or more.
After randomization, schools were asked to organize separate meetings for teachers of each of the treated classes, during which they had the opportunity to collegially select the two candidates taking part to the training since the beginning of the 2017-2018 school year. On the other side, teachers of the control classes will simply have the opportunity to receive our intervention one year later, in the 2018-2019 school year.
Experimental Design Details
Randomization Method
Randomization done in office with a computer
Randomization Unit
Classes within intra-school and inter-school blocks defined by COS
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
171 classes within 33 blocks (31 intra-school + 2 inter-school)
Sample size: planned number of observations
Around 3420 10th grade students
Sample size (or number of clusters) by treatment arms
41 treated classes (around 820 students)
130 control classes (around 2600 students)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Conservative estimates for students’ primary outcomes around 0,106 expressed in terms of effect size
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Ethical Committee - University of Milano-Bicocca
IRB Approval Date
2018-05-28
IRB Approval Number
N/A (certificate of compliance with ethical standards)
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS