Secondary Outcomes (explanation)
The secondary outcome for students’ mental health is captured in a student survey. Rather than solely relying on the short-term effect to appear through graduation rate, we believe the intervention potentially can impact the mental well-being of students as students should feel more supported and motivated with their studies and experience that they belong and are able to add value to their new profession.
Mental well-being will be measured based upon seven questions from the Warwick-Edinburg Mental Well-being Scale (Clarke et al 2011, Koushede et al 2018, Powell et al 2013). Five of the questions are taking from the Short Warwick-Edinburg Mental Wellbeing Scale while the last two have been chosen because we believe that they measure what the intervention is trying to achieve. We will construct one mental wellbeing score by taking the sum of the seven questions.
The other secondary outcomes are explorative and originates from the administrative records. First, they capture the students’ potential of a better education match, the students’ achievement, measured as the students’ grades, and the students’ probability of signing an apprenticeship. Second, they capture input factors measured as student and teacher attendance, which may change because of the intervention.
The intervention is meant to give students a better chance to graduate from the GF2 program. However, graduation from the GF2 program is not the final outcome for the vocational students’ education because after the GF2 program comes the vocational main program of around 3 years after which the student receives a certification for the labor market. We therefore include a secondary outcome measure 8 months after initial enrollment to see if the students are enrolled in any education or have graduated from the GF2 program. This measure at 8 months after initial enrollment will capture both the students who graduated the GF2 program and it will also capture students who have started a different education.
The intervention is meant to give students a better idea from the beginning of the program about whether the chosen education is the right one. This means that one consequence of the intervention could be that some students will drop out earlier when they realize that the specific education is not a good match. Some of these students who drop out because of a bad match may have improved skills due to the intervention that can help them choose another education than the one they drop out from (such as the ability to network, a better understanding of the industry, and understanding drop-out as learning rather than failure). These students should be better prepared to start a different education and stay with it. By measuring students’ graduation and any other educational enrollment 8 months after initial enrollment, we capture both the restarters and the students who graduated from the main program.
The students receive grades in some of the subjects that are not practical (e.g. math). The practical exam is a pass/fail. If the intervention increases the probability of graduating from the GF2, we could expect the grade distribution to improve. The average grade of the graduates will depend on the composition of the graduates and it is therefore not clear if the average grade, conditional on passing the exam, will increase or decrease. However, as a fraction of all the enrolled students, we should expect the grades to improve. To measure whether the intervention has an impact on the high end of the grade distribution, we will measure the effect of the intervention on a variable that takes value one if the student achieves a high grade and takes value zero if the student passes but achieves a non-high grade, the student fails, or if the student drops out, which capture all the enrolled students.
We hope to include a measure of the probability of obtaining an apprenticeship at a firm and the date at which the agreement with the firm is made. If the intervention is successful at connecting the students more to the industry and that the network groups helps, we expect that the fraction of students finding an apprenticeship with a firm increases just as the fraction of students having an apprenticeship at the school should decrease . We will use the False Discovery Rate (Benjamin and Hochberg 1995, Andersen 2008) to obtain q-values for the hypotheses tests corresponding to the following two outcomes: probability of a high grade and probability of obtaining an apprenticeship.
Finally, we expect the students’ attendance to increase if the students can see an added benefit to coming to class from the intervention. The teachers’ attendance could also be affected if students’ higher motivation leads teachers to be more motivated and less likely to suffer from stress or other sicknesses as a result of the intervention. We will use the False Discovery Rate (Benjamin and Hochberg 1995, Andersen 2008) to obtain q-values for the hypotheses tests corresponding to the following two outcomes: students’ attendance and teachers’ attendance.
Outcomes that will be used to explain the mechanisms at play
Through the student survey, which the student answers at week 3 and 13 of the GF2, we also ask questions covering topics such as: (1) self-efficacy, (1) feeling of “succeeding as a team”, (3) sense of belonging to the profession, (4) perception of the curriculum, (5) feeling of having value, (6) Number of contacts with workplaces. These six concepts are tightly related to the underlying theory of change of the intervention.
(1) Self-efficacy will be measured through an index which includes the answers to 2 questions which the research team has formulated. (2) Feeling of “succeeding as a team” will be measured using an index which includes the answers to 3 questions. (3) Sense of belonging is measured through an index including 2 questions uncovering if the student is uncertain about their belonging in the profession. (4) perception of curriculum is measured through 1 question uncovering if the student finds the acquired skills and experiences in GF2 to have value later on. (5) Feeling of having value is related to if the student feels like they can contribute to the workplace and vocation of their choice. (6) number of contacts with workplaces is measured as one question asking about this. We will use the False Discovery Rate (Benjamin and Hochberg 1995, Andersen 2008) to obtain q-values for the hypotheses tests corresponding to the following six outcomes: self-efficacy index, succeeding as a team index, sense of belonging on the profession index, perception of curriculum, perception of having value, and number of contacts with workplaces.