Digital Literacy and Well-Being: A Randomized Controlled Trial on Adolescents’ Use of Social Media and Algorithms

Intervention (Public)

The intervention is a school-based training program for adolescents aged 12–17 in Spain. Students participate in 3–4 classroom workshops (60 minutes each) on responsible social media use, algorithmic awareness, privacy management, emotional regulation, and digital well-being. Sessions combine interactive activities, real-life vignettes, and bioethical reflection to strengthen resilience and build critical digital literacy. Materials are adapted from the European DigComp 2.0 framework and bioethics principles.

The intervention will be implemented during the fall term of 2025 in participating secondary schools. Approximately 700 students are randomized either at the classroom level (block randomization in 1º–3º ESO, groups A–D) or within-classroom (1E, 2E, 3E, 4º ESO, 1º Bachillerato, 2º Bachillerato). Training is delivered by facilitators supported by student assistants from URJC.

The program consists of four modules:

Algorithmic awareness: how platforms like TikTok, Instagram, and Snapchat use recommendation systems to maximize engagement, covering echo chambers, attention manipulation, and privacy concerns.

Digital well-being and emotional regulation: strategies for screen-time management, sleep hygiene, coping with FOMO, and preventing compulsive use; reflection on mental health risks (depression, anxiety, self-esteem).

Privacy and online safety: understanding digital footprints, preventing cyberbullying, managing personal data, and addressing addictive design features.

Bioethics and responsible digital behavior: applying principles of autonomy, beneficence, non-maleficence, and justice to digital contexts, using case studies and classroom debates.

Evaluation is conducted through a randomized controlled trial (RCT) with pre- and post-surveys (Nov 2025 baseline, Jan 2026 follow-up). Validated instruments include ERA-RSI, PHQ-9, Rosenberg Self-Esteem, Ryff Scales, and Pittsburgh Sleep Quality Index. Outcomes cover mental health, digital literacy, risk perception, and ethical awareness

2025-10-13

2026-01-11

1. Mental health
o Social Media addiction (ERA-RSI)
o Sleep quality (Pittsburgh Index)
o Self-esteem (Rosenberg)
o Depression symptoms and psychological well-being (EIDAN)
2. Digital literacy
o Understanding of algorithmic recommendations
o Ability to identify risky online behaviors
o Privacy awareness and digital time management (DigComp 2.0 scales)

Secondary Outcomes
• Risk perception: cyberbullying awareness, addictive patterns, perceptions of algorithmic manipulation
• Behavioral outcomes: self-reported time online, number of hours on TikTok/Instagram, self-monitoring logs
• Ethical awareness: responses to vignettes on beneficence, autonomy and justice in digital contexts

The evaluation follows a Randomized Controlled Trial (RCT).
• Sample: Approximately 700 students aged 12–17, spanning 1º–4º ESO and 1º–2º Bachillerato.
• Intervention: Classroom-based training modules on responsible social media use, algorithmic awareness, emotional regulation, privacy, and AI ethics. Materials are adapted from DigComp 2.0 and bioethics frameworks (Beauchamp & Childress, 2013).
• Duration: 3–4 workshops of 60 minutes, delivered during the fall term 2025.
Empirical Method
Main specification:


Y_ig = α + β * Treatment_ig + γ * X_ig0 + λ_g + ε_ig

where:

Y_ig = outcome of student i in group g
Treatment_ig = indicator for assignment to treatment
X_ig0 = baseline covariates
λ_g = grade (or strata) fixed effects
ε_ig = error term
The main ITT specification will strictly use the full randomization strata fixed effects (λs) which fully capture the classroom-by-grade stratification, instead of just the grade fixed effects (λg) presented for simplicity in Equation (62).
• Balance Checks and Adjustment: Baseline balance will be checked on all covariates (Xig0) across treatment and control groups. We will flag any variable with a standardized difference greater than 0.2 or a p<0.10 from an F-test. If a significant imbalance is detected, the full model will include the unbalanced covariate as a control variable (Xig0) to absorb the pre-treatment difference.
• Missing Covariate Data: For missing data in the baseline covariates (Xig0), we will use mean imputation and include a dummy variable indicating whether the covariate was imputed. Robustness checks will employ listwise deletion or multiple imputation if the rate of missingness exceeds 5%.
• Multiple Hypothesis Correction: The pre-specified families for Holm-Bonferroni control are:
o Family 1 (Mental Health): Depression symptoms (PHQ-9), Self-esteem (Rosenberg), Psychological well-being (Ryff Scales; BSPWB-A), and Sleep quality (Pittsburgh Index).
o Family 2 (Digital Literacy): Understanding of algorithmic recommendations, Ability to identify risky online behaviors, Privacy awareness, and Digital time management (DigComp 2.0 scales).
• Estimation: ITT (intention-to-treat). The choice of an MDE of 0.20 SD is based on meta-analyses of similar digital literacy and mental health interventions in field settings, which typically report effect sizes ranging from 0.10 to 0.35 SD. An effect of 0.20 SD is considered the minimum effect size that is substantively and practically meaningful for educational policy and behavioral change in adolescents
• Robustness: Per-protocol (≥80% participation); alternative specifications with randomization strata fixed effects.
• Multiple hypotheses: Family-wise error control using Holm-Bonferroni within outcome families (mental health, digital literacy, risk perception).
• Heterogeneity: Interaction terms for gender, grade, and baseline digital habits.
• Exploratory analysis: Machine learning classifiers to detect latent subgroups with differential response to treatment.

Randomization strategy:
• 1º, 2º, 3º ESO (groups A–D): Block randomization by class, with two classes per grade in treatment and two in control.
• 1E, 2E, 3E, 4º ESO, 1º Bachillerato, 2º Bachillerato: Within-class randomization, where students are individually assigned to treatment or control inside their own classroom.
This dual design allows us to compare both between-classroom spillovers and within-classroom peer effects.

Randomization Unit
The unit of randomization varies by grade:

For 1º–3º ESO (groups A–D), randomization is at the classroom level (clustered).

For 1E, 2E, 3E, 4º ESO, 1º Bachillerato, and 2º Bachillerato, randomization is at the individual level within classrooms.

This dual design allows us to study both classroom-level spillovers and within-class peer effects.

Yes

12 classrooms (cluster-randomized arm).
Additionally, we will run individual-level randomization within ~16 classrooms (these are not treatment clusters).

700 pupils (secondary schools in Spain, ages 12–17).

Cluster-randomized arm: 6 classrooms treatment (~150 pupils) and 6 classrooms control (~150 pupils).

Individually randomized arm (within classrooms): ~200 pupils treatment and ~200 pupils control across ~16 classrooms.

Total: ~350 treatment, ~350 control (≈50/50 split).

Assumptions: two-sided α=0.05, power=80%, 50/50 assignment, average class size m≈25, ICC ρ=0.05–0.10 for the clustered arm, baseline adjustment R²≈0.50, strata fixed effects and classroom-clustered SEs. Pooled (both arms combined): MDE ≈ 0.12–0.13 SD (≈0.17–0.18 SD without baseline controls). Individually randomized arm (~400 pupils): MDE ≈ 0.14 SD (≈0.20 SD without controls). Cluster-randomized arm (~300 pupils): MDE ≈ 0.24–0.30 SD (≈0.34–0.42 SD without controls), with m≈25 and ρ=0.05–0.10.