The effects of a web application supporting young adults neither in education, employment, nor training

In this project we test the effects of the web application RØST. RØST is targeting young people not in employment, education or training (NEET), and is designed to help them handle stress, strengthen their belief in their capacities to learn, and to work towards goals in delibarate and strategic ways. We developed RØST building on recent studies which demonstrate that time-limited web-based psychological interventions can strengthen high school students' beliefs in their capacity to learn (Yeager et al., 2016; Bettinger et al., 2017), promote goal-setting strategies to fight procrastination (Duckworth et al., 2011; Kizilcec & Cohen, 2017), and promote strategies for coping with stressful events (Yeager et al., 2020). We have used design thinking to adapt the protocols from these interventions to target NEET youths.
RØST has five sessions, each lasting about 15-25 minutes. After a brief introductory session, the second session prompts the user to set a goal related to education or work. Thereafter it asks the user weekly to set a subgoal (small step) towards this goal, which will be followed up the next week. Importantly, to support goal achievement, each session in RØST has content helping the users overcome three possible obstacles: procrastination, limited beliefs in capacity to learn, and limited stress coping strategies.
The app for the control group looks similar to the RØST, with five sessions prompting the user to set a goal and subgoals related to education or work. However, the control app does not support the users to overcome obstacles related to goal achievement. Instead, participants learn about various web resources available for job and education applicants. This information is available from the Norwegian public employment office (NAV) web page or other web pages providing information about educational opportunities.
Our main interest is to explore the effect of RØST on participation in education and the labor market (registry data), on depressive symptoms, stress and well-being (survey data), and how possible effects operate through effects on growth mindset, stress mindset and procrastination (survey data).
Furthermore, we will investigate differential treatment effects of RØST across gender, socioeconomic status, academic achievement in compulsory schooling, and pre-intervention survey measures of growth mindset, stress mindset and procrastination.

2024-10-21

2025-12-31

(1) Register data measures on labor market participation or education enrollment: employed, earnings, enrollment in education, participation in labor market programs initiated by NAV.
(2) Survey measures on current life situation: 1) Perceived stress (PSS-10; Cohen and Williamson, 1988); 2) Warwick-Edinburgh mental well-being scale (Stewart-Brown et al., 2009; Tennant et al., 2007); 3) Symptoms checklist (SCL-10; Strand et al., 2003).

Mechanisms
(1) Self efficacy (Gaumer Erickson et al., 2016)
(2) Stress mindset (Yeager et al., 2021)
(3) Procrastination (IPS; Steel, 2002 & 2010; translation to Norwegian by Svartdal, 2015)

Treatment Fidelity
(1) Fixed mindset scale (Yeager, 2016)
(2) Stress manipulation test (Yeager et al., 2021)

Other outcomes
(1) Locus of control (Pearlin and Schooler, 1978)
(2) Knows future career (Resnjanskij et al., 2021).

All youths registered at the Norwegian public Follow-up Service will be invited to participate. The Follow-up Service in Norway supports young people up to the age of 24 who are entitled to upper secondary education but are not currently in school, training, or regular employment. They reach out to those who have not applied for education, have left education or training, or have lost their entitlement to it, helping them find suitable educational, training, or job opportunities.
Upon providing consent and logging into the web application, participants will complete a baseline survey. Following this, they will receive weekly content through the web platform. The participants will receive an SMS each week for five consecutive weeks, with a note that a new session in the app is available. There will also be reminders to complete the content in each session.

After consenting to participation, logging on to the web application and completing the baseline survey, subjects will be automatically randomized to the two treatment arms, involving different material in the web application. Subjects are blind to treatment.
Participants will earn points each time they have reached a milestone. These points can be exchanged for gift cards (3 giftcards with a total amount of NOK 1000). The incentives are independent of whether they are in the control or treatment group.
During a five-week period, a new chapter with material and goal setting prompts will be released weekly for both the treatment and control groups. The participants will receive an SMS each week for five consecutive weeks, with a note that a new session in the app is available (there will also be reminders to complete the content in each session). Participants are asked to complete a survey before starting the four sessions with intervention material, immediately after the four sessions are complete, and in a four week follow up. They will also be invited to complete a survey in a one year follow up.

Randomization happens individually and is automated in the app at the individual level.

We randomize at the individual level with equal probability of each treatment.

No

2000 individuals

We invite all youths aged 16-25 who are registered with the Norwegian public Follow-up Service and are not currently in a program involving full-time education, employment, or job training. This will give us a sample size of about 10.500 youth. We will have a first-serve policy up until at least 2000 youths have consented to participate.

We will have 1000 in treatment and 1000 in control.

We calculate the MDES using Optimal Design Software (Spybrooke et al., 2001). We have individual-level randomization and a sample of 2,000 youths who have signed up to participate. This sample is equally split between treatment and control. Assuming 80 percent power and a two-sided 95-percent confidence interval, we have a Minimum Detectable Effect Size (MDES) of 0.13 standard deviations, not including any control variables. If we assume that control variables explain 20% of the variation in the outcome variables (R^2=0.2), the MDES reduces to 0.11 standard deviations. If we assume an R^2 as high as 0.8 (as we observe baseline values of most of our outcome variables), it would go down to 0.06 standard deviations. Adding data from the first field implementation may give us even more precision.