Experimental Design
The proposed experiment takes place within the online career counseling platform for preparatory vocational education (vmbo in Dutch) of Qompas (www.qompas.nl). Qompas is a private company that, among other things, provides career guidance services to secondary and post-secondary education students in the Netherlands. The platform provides preparatory vocational education students with computer assignments and tests to do in class. The assignments and tests help the students get to know what they like, what they're good at and what occupations might fit them. All the information generated throughout the various steps of the Qompas assignments are stored in an individual `student file'. The students are supposed to periodically review this file. One of the tests is the so-called `occupation test'. During this test, students answer a number of questions about themselves and potential occupations. Based on the answers given, Qompas calculates a score for each of the 353 occupations in their system. This score represents how well the occupation fits the student's preferences and abilities. Our experiment starts right after this test.
After the test, students are first asked what study profile they intend to choose. We refer to this as the prior intended profile choice.
After this, the students are shown the twenty occupations that fit them best according to the test. They are subsequently asked to select the five occupations they are most interested in pursuing out of these twenty occupations. After selecting the five occupations, the students are shown information about what the work in the different occupations entails. After the students are provided with this information, they are asked to rank the five occupations on the basis of how much they want to pursue that occupation. We refer to this as the prior ranking of occupations. When making the ranking, the study profiles associated with the different occupations are shown.
After the ranking, the students are asked about their beliefs about the labor market prospects of the different occupations. They are asked to estimate both the opportunities of finding a job in six years and the current hourly wage of intermediate vocational education graduates for the selected occupations. The opportunities of finding a job can be either very bad, bad, reasonable, good or very good. The options for the hourly wage range between €10,- and €26,- with €1,- intervals. We refer to these beliefs as prior beliefs.
Once the beliefs are elicited, students are shown a number of screens, depending on the treatment group they belong to. Randomization of students into control and treatment groups is done at the school level. There is one control group and there are four treatment groups. For the exact randomization strategy, see the corresponding section. The control group is shown no information about the labor market prospects of different occupations and is therefore shown no sender either. The treatment groups are provided with information about either just the job opportunities of their selected occupations or both the job opportunities and hourly wage levels of their selected occupations. This information is provided by Maastricht University's Research Center for Education and the Labor Market (www.roa.nl). As part of one of its research programs, this research center has developed labor market forecasts (http://roa.sbe.maastrichtuniversity.nl/?portfolio=poa-project-onderwijs-arbeidsmarkt-2; For more information about the forecasts, please contact d.fouarge[at]maastrichtuniversity.nl). These forecasts predict the job opportunities for 113 different occupational groups for the coming six years. The Qompas occupations are matched to these occupational groups. The information about the gross hourly wage is taken from administrative data. It is the mean hourly wage for individuals who graduated from an intermediate vocational education program and work in the occupational group associated with the Qompas occupation. The reason for not taking the seemingly more relevant median hourly wage is that preparatory vocational education students probably know the definition of the mean, but may not know the definition of the median.
Treatment 1 and 2 are given information about just the job opportunities. Treatments 3 and 4 get information on both the job opportunities and the hourly wage levels for different occupations.
In treatments 1 and 3, the information is presented by a reseacher from the Research Center for Education and the Labor Market. These `information senders' are divided into four groups: low-status males, high-status males, low-status females and high-status females. In this context, status is defined by the seniority of the information sender. A researcher that does not have a Ph.D. (yet) is considered low-status, whereas a researcher with a Ph.D. is considered high-status. To ensure understanding, the designation presented to students is either 'beginning researcher' or 'experienced researcher' (In Dutch: `beginnend onderzoek(st)er' and `ervaren onderzoek(st)er'). The reason for not presenting the different statuses as `junior' and `senior', respectively, is that we are again worried about a lack of understanding. `Beginning' and `experienced' are more commonly used in the scenario described above in Dutch than in English. For each sender, the name and status are shown. Gender is not explicitly mentioned, but the names of all senders are indicative of their gender and the Dutch word for `researcher' is different for men and women. No pictures of the sender are shown, so as to avoid bias caused by appearance unrelated to status or gender.
In treatments 2 and 4, no human information sender is specified. Instead, students are told the Research Center for Education and the Labor Market provides them with the information.
To decide on the way in which to display the information, we held a short survey among students of a high school that does not work with Qompas, but is otherwise very similar. Approximately 72% (44 out of 61) of students indicated that the current way of displaying information was the most clear to them.
We are primarily interested in the effect of the information about job opportunities. Therefore, we did not include a treatment group that only gets information about hourly wages, so as to preserve power. The main reason for this is that the information about job opportunities is a forecast of the situation in six years, which is more relevant for the students than the current hourly wage levels.
Next, after the first ranking (control group) or information provision (treatment groups), all groups are shown a video (https://www.youtube.com/watch?v=YJ78VDQrO3c) about work in general. We do this to allow both treated and non-treated students some additional reflection time. The video does not mention any particular occupations or the importance of job opportunities and wages.
After the video, the students are asked to make a second ranking. They are shown their first ranking and asked whether they want to change anything. We refer to this as the posterior ranking at second elicitation. After this second ranking, beliefs are again elicited. We refer to these beliefs as the posterior beliefs.
It is possible that students select occupations with very bad labor market prospects only. Based on a sample of historical data, approximately 20% of students is expected to select only such occupations. In this case, providing information is not very useful. To deal with this, students who selected only occupations that have very bad, bad or reasonable job opportunities are shown a number of alternative occupations. The occupations suggested are those with the best labor market prospects out of the twenty occupations they selected, ranked first on job opportunities and then on hourly wages. The difference between the control group and treatment groups lies in the information provided. The control group gets no information about why these occupations are suggested. The treatment groups are told these occupations have better prospects. Treatments 1 and 2 then get information about just the job opportunities and treatments 3 and 4 get information about both the job opportunities and the hourly wages.
After the alternative occupations are suggested, the students can decide to learn more about these occupations or not. We refer to this as the decision to learn more about the alternative occupations or not. If the students decide to learn more about these occupations, they are thereafter asked whether they want to change their ranking one last time. In this ranking, the students are allowed to include the alternative occupations. Initially, the new occupations are placed at the bottom of the ranking in a random order. We refer to this ranking as the posterior ranking at third elicitation. The final top five occupations are subsequently included in the student file.
A number of months after the experiment, the second-year students have to choose a study profile. Students are supposed to indicate what study profile they chose in the `definitive profile choice survey'. We refer to this as the actual profile choice. We also include a question about the intended profile choice at the end of the experiment. In the analyses, we refer to this as the posterior intended profile choice. The fourth-year students have to choose a study program a number of months after the experiment. This choice is captured in one of the other Qompas assignments. Naturally, the study program choices of the current third- and second-year students are captured a year and two years later, respectively.