Supporting young adults neither in education, emplyment or training

This intervention is divided into two sub-projects.

Subproject 1 (Effect of RØST)
In Subproject 1 we test the effects of the web application RØST. RØST is targeting young people not in employment, education or training (NEET), and is designed to help them handle stress, strengthen their belief in their capacities to learn, and to work towards goals in delibarate and strategic ways. We developed RØST building on recent studies which demonstrate that time-limited web-based psychological interventions can strengthen high school students' beliefs in their capacity to learn (Yeager et al., 2016; Bettinger et al., 2017), promote goal-setting strategies to fight procrastination (Duckworth et al., 2011; Kizilcec & Cohen, 2017), and promote strategies for coping with stressful events (Yeager et al., 2020). We have used design thinking to adapt the protocols from these interventions to target NEET youths.

RØST has five sessions, each lasting about 15-25 minutes. After a brief introductory session, the second session prompts the user to set a goal related to education or work. Thereafter it asks the user weekly to set a subgoal (small step) towards this goal, which will be followed up the next week. Importantly, to support goal achievement, each session in RØST has content helping the users overcome three possible obstacles: procrastination, limited beliefs in capacity to learn, and limited stress coping strategies.

The app for the control group looks similar to the RØST, with five sessions prompting the user to set a goal and subgoals related to education or work. However, the control app does not support the users to overcome obstacles related to goal achievement. Instead, participants learn about various web resources available for job and education applicants. This information is available from the Norwegian public employment office (NAV) web page or by talking to NAV advisors. The five sessions in the control app systematize and briefly summarize this information.

Our main interest is to explore the effect of RØST on participation in education and the labor market (registry data), on depressive symptoms, stress and well-being (survey data), and how possible effects operate through effects on growth mindset, stress mindset and procrastination (survey data).

Furthermore, we will investigate differential treatment effects of RØST across gender, socioeconomic status, academic achievement in compulsory schooling, and pre-intervention survey measures of growth mindset, stress mindset and procrastination.


Subproject 2 (Effect of Incentives on Research Participation)
In this project we investigate how incentives affect participation and compliance. We develop two versions of the web application experiment described in Subproject 1: Experiment A in which participants will earn points each time they have reached a milestone, which can be exchanged for gift cards (independent of whether they are in the control or treatment group); and Experiment B with no such incentives. The two experiments have identical online invitations with the exception of one important detail: In A the invitation includes information about participation payment through gift cards, whereas in B the invitation does not have any information about participation payment. After consenting for participation, logging on and completing baseline survey, subjects will be automatically randomized to the control or to the treatment app (as described in Subproject 1).

In this subprojects we will pool the data from the treatment and control group and test differences between experiment A and B.Our main interest is to investigate how incentives affect participation and compliance. Additionally, we will investigate if people who consent to participate with incentives (experiment A) are different from those who consent without incentives (experiment B). Specifically, we will investigate if they are less forward-looking, less conscientious and have less self-efficacy. Moreover, we will investigate if they experience more depressive symptoms, stress and lower well-being.

When data is available, we will also investigate if, due to the selection effect, the average treatment effect is higher for those who enroll in RØST B than for those who enroll in RØST A.

2023-02-20

2023-12-31

Sub project 1:
(1) Register data measures on labor market participation or education enrollment: employed, enrollment in education, participation in labor market programs initiated by NAV, and social security benefits.

(2) Survey measures on current life situation: 1) Perceived stress (PSS-10; Cohen and Williamson, 1988); 2) Warwick-Edinburgh mental well-being scale (Stewart-Brown et al., 2009; Tennant et al., 2007); 3) Symptoms checklist (SCL-10; Strand et al., 2003).

In follow up studies we will also look at educational attainment and income in registry data.

Sub project 2:
(1) Indicator for consent, indicator for program completion, time spent on program and surveys
(2) Survey measures on personality: 1) Time preferences (Falk et al., 2022); 2) Conscientiousness (BFI-2 XS, Soto and John, 2017); 3) Self efficacy (Gaumer Erickson et al., 2016)
(3) Survey measures on current life situation: 1) Perceived stress (PSS-10; Cohen and Williamson, 1988); 2) Warwick-Edinburgh mental well-being scale (Stewart-Brown et al., 2009; Tennant et al., 2007); 3) Psychological distress symptoms checklist (SCL-10; Strand et al., 2003).

Sub project 1:
Mechanisms
(1) Self efficacy (Gaumer Erickson et al., 2016)
(2) Stress mindset (Yeager et al., 2021)
(3) Procrastination (IPS; Steel, 2002 & 2010; translation to Norwegian by Svartdal, 2015)

Treatment Fidelity
(1) Fixed mindset scale (Yeager, 2016)
(2) Stress manipulation test (Yeager et al., 2021)

Other outcomes
(1) Locus of control (Pearlin and Schooler, 1978)
(2) Number of job applications (Bjorvatn et al., 2021)
(3) Knows future career (Resnjanskij et al., 2021).

All youths registered at the Norwegian public employment office (NAV) will be invited to participate. After consenting to participation, logging on to the web application and completing the baseline survey, subjects will be automatically randomized to different treatment arms, involving different material in the web application. The participants will receive an SMS each week for five consecutive weeks, with a note that a new session in the app is available. There will also be reminders to complete the content in each session. Subjects are blind to treatment.

Our first field implementation of RØST will allow us to test how incentives affect participation and compliance (Subproject 2), and at the same time gather some of the data for investigating effects of RØST (Subproject 1). In this implementation all youths registered at the Norwegian public employment office (NAV) in Rogaland will be invited to participate. They will be randomized into two different groups, which will receive different online invitations: Invitation A and B. Invitation A includes information about participation payment through gift cards, and links to Experiment A which includes payments. Invitation B does not have information about participation payment, and links to Experiment B.

After consenting to participation, logging on and completing the baseline survey, subjects will be automatically randomized to the control or to the treatment app (as described in Subproject 1). In Experiment A, participants will earn points each time they have reached a milestone. These points can be exchanged for gift cards (3 giftcards with a total amount of NOK 1000). The incentives in Experiment A are independent of whether they are in the control or treatment group. Experiment B has no incentives.

With the exception of the presence of incentives in Experiment A, both experiments are identical and both provide a control and a treatment version of the app as described above. During a five week period, a new chapter with material and goal setting prompts will be released weekly in both treatment and control. The participants will receive an SMS each week for five consecutive weeks, with a note that a new session in the app is available (there will also be reminders to complete the content in each session).
For both experiment A and B, participants are asked to complete a survey before starting the four sessions with intervention material, immediately after the four sessions are complete, and in a four week follow up.

Second field implementation
Our second field implementation will gather data for investigating effects of RØST (Subproject 1). All youths registered at the Norwegian public employment office (NAV) all across Norway, except Rogaland (which was included in the first implementation), will be invited to participate. The procedure is identical to the procedure described above for the first field implementation. Our second field implementation will be with (experiment A) or without incentives (experiment B), depending on how incentives affected participation in the first implementation.

Providing that the first field implementation has no technical difficulties, and no major changes are needed, we will include data from both the first and the second field implementation in Subproject 1. If implementation 1, however, suggests that many changes are needed, then we will only use data from the second field implementation for sub-project 1. Then we will add this change to the registration prior to the second field implementation.

Randomization of the invitation with and without incentives will be performed by a computer at NAV Statistical section at the individual level, with a witness present.
Randomization happens individually and is automated in the app at the individual level.

In both subprojects we randomize at the individual level with equal probability of each treatment.

No

1300 Individuals

First field implementation: We invite all youths registered as unemployed during the last two months at NAV Rogaland aged 18-29 years, for whom NAV has not registered any special difficulties making regular employment highly unlikely. This will give us a sample size of about 1300. Based on previous studies, we expect 383 participants to sign up. Second field implementation: We will invite all youths aged 18-29 years registered at NAV in Norway except from Rogaland, for whom NAV has not registered any special difficulties. We will have a first-serve policy up until at least 1000 youths have consented to participate.

First field implementation:
The 1300 youths are randomized to receive an invitation to experiment A or B with equal probability, leading to 650 participants in each group. We expect a 35 % uptake in A and a uptake of 24 % in B, giving a sample of 227 in A and 156 in B .

Second field implementation:
We will have 500 in treatment and 500 in control.

First field implementation: We calculate the MDES using Optimal Design Software (Spybrooke et al., 2001). For investigating effects on participation, we have 1300 observations. With this sample size, we are able to detect an effect of 0.16 standard deviations assuming 80 percent power and a two-sided 95-percent confidence interval. Assuming as above a 35% response rate, this translates into a difference of 7 percentage points. For investigating effects on selection, our sample is reduced to those who agreed to participate in the project. We expect our sample to be 383 individuals with 227 in experiment A and 156 in experiment B. This reduces the MDES to 0.30 standard deviations when comparing differences between participants in A and B. Second field implementation: We calculate the MDES using Optimal Design Software (Spybrooke et al., 2001). We have individual-level randomization and a sample of 1000 youths who have signed-up to participate. This sample is equally split between treatment and control. Assuming 80 percent power and a two-sided 95-percent confidence interval, we have a MDES of 0.18 standard deviations, not including any control variables. If we assume that control variables explain 20% of the variation in the outcome variables (R^2=0.2) the MDES reduces to 0.16 standard deviations. If we assume an R^2 of as high as 0.8 (as we observe baseline values of most of our outcome variables), it would even go down to 0.08 standard deviations. Adding data from the first field implementation may give us even more precision.