Evaluating the effectiveness of goal-setting strategies for NEETs

This study is a part of the evaluation of the effectiveness of an app developed for NEET-youth in Norway. The current study focuses on the effectiveness of the first module on goal-setting.

We will investigate the success of using mental contrasting and implementation intention (MCII) to improve goal achievement. The app, called RØST, aims to improve their ability to overcome procrastination and begin to take steps towards their goal. We will measure the effect of the MCII module on self-reported goal achievement, as well as on their information-seeking behavior and their reported mood. We hypothesize that the MCII module will increase the likelihood that the youth begin to take practical steps towards reaching their goals.

The youth are randomized to treatment and control upon entering the app. Both groups go through a module about goal-setting. They are presented with example goals from other youth, and then asked to set an achievable goal for themself. They then describe a first step towards this goal, and set a time and place to do this. If they decide on a time, they will get a reminder at that time. The treatment group, however, is additionally treated with the MCII-approach to goal-setting. They are first given an introduction about the challenge of procrastination and that it is possible to overcome it through the methods they will soon learn. It never explicitly mentions the MCII approach, but rather talks about how obstacles can create a gap between our goals and our actions. After setting their goal, they are then asked to write down two positive scenarios that will happen if they achieve their goal and imagine the details and how it will feel. This is part of what psychologists call “mental contrasting” where they envision the desired future, as well as some obstacles they may encounter. Finally, they are asked to write down what they will do to overcome obstacles. This is the “implementation intentions” part of the treatment. As described earlier, the control group is also asked to set a goal and to break it down into smaller goals, but not asked to envision the desired future or obstacles (mental contrasting) nor to make a plan to handle obstacles that may arise (implementation intentions). The difference between the treatment and the control group is thus the MCII approach to goal-setting.

Summary of intervention content for treatment and control:

CONTROL
- Set goal
- Break down to next step
- Decide time and place to do the next step
- Get reminder on sms

TREATMENT
- Introduction about how obstacles can make us procrastinate (MCII)
- Set goal
- Think about the positive outcome (MCII)
- Break down to next step
- Decide time and place to do the next step
- Get reminder on sms
- Think about and imagine obstacles (MCII)
- Make plan to overcome obstacle (MCII)

2024-10-21

2025-03-31

Goal achievement

See details in experimental design

The participants in the study are NEET-youth who receive welfare benefits. 1300 youth will get an invitation to participate in the project.

The youth get the opportunity to participate in testing the app through the “Norwegian Labour and Welfare Administration”. Based on the participation rate in previous studies we expect 383 participants to sign up and consent to be part of the study. In total, the app includes one module on mental contrasting and implementation intention (MCII), one module on growth mindset and one module about stress mindset. The current study focuses on the effectiveness of the first module on improving the goal-directed behavior of the young adults. The intervention is further described under that section.

At the end of the module the youth are provided with some links that can be useful for getting into a job or education. Next week, when the youth return to the app to go through a new module, they are immediately asked about their mood that day and whether they finished their goal. The self-reported goal attainment will be the primary outcome of this study. Their mood and the amount of links they click will be included as secondary outcomes. The goals the youth wrote down will be subject to text analysis.

Primary outcomes:
-Goal achievement: Whether the respondent answered yes or no to a question about whether they did their first step.

Secondary outcomes:
- Whether the participant clicked ANY of the links provided at the end of the session (binary)
-How many of the links were clicked at least once (numeric)
-Reported mood one week after. Participants rank their mood from 1-5, based on smiley faces and words that describe each mood.

Sub-sample analysis will be conducted on the following groups:

Age:
The age of the participants span between 18 and 30 years. It might be that the older youth are less open to changing their behavior in light of the intervention content than the younger. It is important for social services and policy makers to know whether there are substantial differences according to age. We plan to split the sample in half based on age, comparing one younger and one older group.

Procrastination tendency:
The participants answer a survey at baseline where they self-report on their tendency to procrastinate. We hypothesize that youth with higher tendency to procrastinate will benefit more from the intervention than those with lower tendencies to procrastinate.
The procrastination tendency is measured by 9 questions where the participants respond on a scale of 1-5 how well statements like for example "I procrastinate" describes them. These questions are from Steel (2002; 2010); and translated to Norwegian by Svartdal (2015).

Growth mindset/Self-efficacy
We hypothesize that youth with higher growth mindset at baseline have a higher belief that they may succeed in reaching their goals if they put in the effort towards taking the first step. They will likely be more open to the content of the intervention.
We use the self-efficacy questionnaire from Gaumer Erickson et al. (2016) which includes 11 statements that the respondent rates on a scale of 1-6 how well it describes them.

Upon consenting to participate in the study, the youth will be randomized to control or treatment by the app at the individual level.

Individual

No

Not clustered

1500

191

The minimum detectable effect size (MDES) for the sample is calculated through using Optimal Design Software (Spybrooke et al., 2001). With 383 participants and individual randomization and without adding controls, the MDES is approximately 0.29 standard deviations for 80 % power and 95% confidence interval. If we assume that control variables explain 20% of the variation in the outcome variable (R^2 = 0,2 for control variables), this is reduced to 0.26.