Helping High-School Students in Choosing their Career: Experimental Evidence from a Large Scale School Policy

Last registered on January 03, 2020

Pre-Trial

Trial Information

General Information

Title
Helping High-School Students in Choosing their Career: Experimental Evidence from a Large Scale School Policy
RCT ID
AEARCTR-0004587
Initial registration date
December 23, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 03, 2020, 4:45 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Institute for Employment Research

Other Primary Investigator(s)

PI Affiliation
Institute for Employment Research
PI Affiliation
Institute for Employment Research
PI Affiliation
Institute for Employment Research
PI Affiliation
Institute for Employment Research
PI Affiliation
Institute for Employment Research
PI Affiliation
Institute for Employment Research

Additional Trial Information

Status
On going
Start date
2019-09-16
End date
2022-06-30
Secondary IDs
Abstract
There is a long-standing interest in the determinants of education and occupation choice. At the same time, there is the question how targeted interventions can help improve decision-making in this context. We provide experimental evidence for the role of professional career counselors and online self-assessment tools by evaluating a German-wide policy for high school students.
The policy is part of an initiative by the German federal employment agency implemented in 2019 to improve individuals’ career choice throughout their life. At the high school level, the initiative focuses on the decision-making with respect to post-secondary education and occupation. In addition, there is the interest to equalize educational opportunities for students and improve knowledge about the whole spectrum of career opportunities. To achieve these goals, the federal employment agency is revising and expanding their services offered to high school students.

Treatments
For our analysis, we focus on the two central parts of the policy and consider them in two separate treatments: direct interaction of professionally trained career counselors with students (treatment 1), and a self-assessment online tool that analyzes the fit of different fields of study and occupations (treatment 2).
Specifically, treatment 1 considers the expansion of face-to-face contact with students. Prior to the implementation of the policy, counselors already offered one-on-one talks with students, although primarily at the vicinities of the employment agency, and gave presentations informing about post-secondary education at schools. With the policy, first, there is a focus on offering one-on-one meetings at the school premises to allow easy access and closer interactions and second, counselors offer more group activities such as presentations to high schools. For treatment 1, students in treated schools get access to the intensified counselling, whereas those at control schools have access only to the pre-policy program. We therefore expect the treatment to lead to differences in the contact between students and counselors as a first stage.
In contrast, treatment 2 evaluates the effects of a self-assessment online tool developed by the employment agency. In this tool, students fill out skill tests and express their agreement to a broad set of statements. Based on the answers, the tool provides a ranking and additional information for suitable fields of study and vocational trainings. The employment agency considers the tool an effective way to help high school students in their decision-making and intends to encourage more students to complete the according tests. In the treatment, we provide information about the tests and incentives to complete them. As a result, we expect a difference in the probability of completion by treatment status and analyze the effects.

Experimental Design
To implement our treatment 1, we cooperate with employment agencies (Agenturen für Arbeit) in 42 districts in eight federal states (Bundesländer). We contacted all high schools (n=940) in these employment agency districts and invited them to participate in a large-scale experiment on the effects of career counselors on education and occupation choice. We randomly allocated the participating schools in a treatment and control group using pairwise randomization stratified by local employment agency districts and date of declaring participation. To facilitate the implementation of the treatment for the employment agencies we randomized at three points in time. After each of the three randomizations, we communicated the results to the employment agency in the districts where the schools are located to allow them to prepare the treatment. Participating schools did not receive any information about their treatment status and treatment started only after the baseline data collection took place.
For the implementation of treatment 2 we build on the recruitment process of schools for treatment 1. We randomize students in both treatment and control group of treatment 1 into a treatment and control group for treatment 2. After this second randomization, there are four groups (no treatment, only treatment 1, only treatment 2, treatment 1 and treatment 2).

Data Collection
217 schools (Treatment 1: 109 TG and 108 CG) agreed to be part of the baseline sample and to take part in our trial. Within these schools, we focus on the high school students of the graduating classes of 2020 and 2021 to assess the effects of our treatments. We invited the students to fill out surveys in context of our study without mentioning the exact treatments of the experiment. The participation was voluntary and in seven of the eight federal states, the consent of the respective parents was required.
The baseline survey (Q_0) took place as a paper assisted personal interview (PAPI) during class time. In the participating schools, a professional data collection team (Infas/IEA) conducted Q_0 between September and November 2019. Overall, we expect around 9,000 students to participate in Q_0.
We collect data through four additional questionnaires. These questionnaires will take place outside the school context as computer assisted web or telephone interview (CAWI/CATI). We plan to interview the participating students in February to March 2020 (Q_1), October to December 2020 (Q_2), February to March 2021 (Q_3) and October to December 2021 (Q_4).
The questionnaires include information about occupational expectations, believes, personality measures, school performance, take-up of our treatments and further information to capture the primary and secondary outcomes described below.
We also collect data through surveys from the parents and school principals of participating students. For the parents, we plan two questionnaires in CAWI/CATI format of which the first will take place directly after Q_0. For the school principals, we collected data through a paper-based questionnaire at the same time of Q_0.
Furthermore, we enrich our data collection with administrative data of students and parents who gave consent to data linkage. The administrative data include information about students’ meetings with the career counselors of the employment agencies and labor market outcomes such as employment, wages, and occupation.
In addition to the data sources described above, we plan to evaluate treatment 2 by collecting data about the utilization of the online self-assessment tool.

Outcomes
To assess the success of the policy we select outcome variables that are informative about the educational and occupational decisions of high school students.
As primary outcome, we consider the choice of post-secondary education after high school completion. In the case of Germany, most high school graduates either start a college degree in a particular field of study or choose vocational training for a specific occupation. Depending on their decision, students then select either their field of study or occupation, respectively. They can also decide to defer further formal education and instead start temporary internships, voluntary work, etc. We therefore analyze the impact of the treatment along this dimension as well.
These outcome variables provide a short-term assessment of the policy and an analysis close to implementation of the policy (Q_0–Q_2). For medium-run effects, we continue surveying students with a focus on the persistence in education and occupation choice (Q_3-Q_4). On the one hand, students may drop out of university or vocational training all together or they may switch between fields of study or occupation.
The average effect allows for a general assessment of the policy, however, it may mask important heterogeneity. We consider socioeconomic status and gender as two important subgroup categories that can speak to the policy goal of educational equality. We further provide an analysis by students’ post-secondary education intention at baseline to examine the malleability of intentions and the associated direction of adjustments.
To further pin down the importance of different channels, we use secondary outcome variables. The survey questionnaire permits the investigation of various explanations including knowledge about post-secondary education, beliefs about future outcomes, and secondary education outcomes such as grades and school satisfaction.
--
The information given in the registration reflect the current state of the trial. At this point, we collected the baseline data and implemented treatment 1 in coordination with the employment agencies. The external data collection company currently processes the data and subsequently will provide the results of the baseline survey.
External Link(s)

Registration Citation

Citation
Anger, Silke et al. 2020. "Helping High-School Students in Choosing their Career: Experimental Evidence from a Large Scale School Policy." AEA RCT Registry. January 03. https://doi.org/10.1257/rct.4587-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Treatment 1 gives students in treated schools access to intensified face-to-face job counselling

Treatment 2 increases information about and gives incentives to complete a self-assessment online tool which provides the students a ranking and additional information for suitable fields of study and vocational trainings.

For details see abstract.
Intervention Start Date
2019-10-14
Intervention End Date
2021-06-30

Primary Outcomes

Primary Outcomes (end points)
SHORT-TERM:
College vs. vocational training choice
Field of study choice or vocational occupation
Deferring further education


LONG-TERM:
Persistence in institutional choice (college, vocational training)
Persistence in subject choice (field of study, vocational occupation)
Drop out
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Knowledge about post-secondary education,
Beliefs about future outcomes,
secondary education outcomes such as grades and school satisfaction.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
See abstract.
Experimental Design Details
Randomization Method
Clustered randomization stratified by employment agency district and date of enrollment
Randomization Unit
School level
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
217 schools
Sample size: planned number of observations
9,000 pupils
Sample size (or number of clusters) by treatment arms
109 TG, 108 CG
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethic Commission of the University Bamberg
IRB Approval Date
2019-08-22
IRB Approval Number
2019-07/20

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials