x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
Labor Market Information and Education Decisions
Last registered on February 09, 2019

Pre-Trial

Trial Information
General Information
Title
Labor Market Information and Education Decisions
RCT ID
AEARCTR-0003220
Initial registration date
August 31, 2018
Last updated
February 09, 2019 8:18 AM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
Maastricht University
Other Primary Investigator(s)
PI Affiliation
Erasmus University Rotterdam
PI Affiliation
Maastricht University
Additional Trial Information
Status
In development
Start date
2018-09-03
End date
2020-09-01
Secondary IDs
Abstract
Education is known to affect the labor market prospects of individuals. The literature shows that individuals' beliefs about how education influences labor market prospects plays a role in their decisions on whether and in what program to enroll. While the number of field-experimental studies on the role of labor market information in educational choices is growing, a number of key issues have not been dealt with yet. First, while information is often presented by an individual, the way in which the identity of the information presenter influences the way in which the receiver uses that information is unclear. Second, none of the existing studies have looked at the potentially detrimental effect of information overload. With this paper, we hope to provide more evidence on the effect of labor market information on education decisions and fill in some of the above mentioned gaps in the literature.
Registration Citation
Citation
de Koning, Bart, Robert Dur and Didier Fouarge. 2019. "Labor Market Information and Education Decisions." AEA RCT Registry. February 09. https://doi.org/10.1257/rct.3220-5.0
Former Citation
de Koning, Bart, Robert Dur and Didier Fouarge. 2019. "Labor Market Information and Education Decisions." AEA RCT Registry. February 09. https://www.socialscienceregistry.org/trials/3220/history/41247
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
See experimental design.
Intervention Start Date
2018-09-03
Intervention End Date
2019-07-20
Primary Outcomes
Primary Outcomes (end points)
For the analysis, we use measurements of the first, second and third ranking of, and the first and second elicitations of beliefs about, the selected occupations; the third ranking is only measured for individuals that decided to learn more about the suggested alternative occupations, which is also measured and analyzed. Moreover, we use measures of the profile choice for second-year students and the study program choice for fourth-year students. As soon as data about the study program choice of third- and second-year students becomes available, these are included in the analyses.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The proposed experiment takes place within the online career counseling platform for preparatory vocational education (vmbo in Dutch) of Qompas (www.qompas.nl). Qompas is a private company that, among other things, provides career guidance services to secondary and post-secondary education students in the Netherlands. The platform provides preparatory vocational education students with computer assignments and tests to do in class. The assignments and tests help the students get to know what they like, what they're good at and what occupations might fit them. All the information generated throughout the various steps of the Qompas assignments are stored in an individual `student file'. The students are supposed to periodically review this file. One of the tests is the so-called `occupation test'. During this test, students answer a number of questions about themselves and potential occupations. Based on the answers given, Qompas calculates a score for each of the 353 occupations in their system. This score represents how well the occupation fits the student's preferences and abilities. Our experiment starts right after this test.

After the test, students are first asked what study profile they intend to choose. We refer to this as the prior intended profile choice.

After this, the students are shown the twenty occupations that fit them best according to the test. They are subsequently asked to select the five occupations they are most interested in pursuing out of these twenty occupations. After selecting the five occupations, the students are shown information about what the work in the different occupations entails. After the students are provided with this information, they are asked to rank the five occupations on the basis of how much they want to pursue that occupation. We refer to this as the prior ranking of occupations. When making the ranking, the study profiles associated with the different occupations are shown.

After the ranking, the students are asked about their beliefs about the labor market prospects of the different occupations. They are asked to estimate both the opportunities of finding a job in six years and the current hourly wage of intermediate vocational education graduates for the selected occupations. The opportunities of finding a job can be either very bad, bad, reasonable, good or very good. The options for the hourly wage range between €10,- and €26,- with €1,- intervals. We refer to these beliefs as prior beliefs.

Once the beliefs are elicited, students are shown a number of screens, depending on the treatment group they belong to. Randomization of students into control and treatment groups is done at the school level. There is one control group and there are four treatment groups. For the exact randomization strategy, see the corresponding section. The control group is shown no information about the labor market prospects of different occupations and is therefore shown no sender either. The treatment groups are provided with information about either just the job opportunities of their selected occupations or both the job opportunities and hourly wage levels of their selected occupations. This information is provided by Maastricht University's Research Center for Education and the Labor Market (www.roa.nl). As part of one of its research programs, this research center has developed labor market forecasts (http://roa.sbe.maastrichtuniversity.nl/?portfolio=poa-project-onderwijs-arbeidsmarkt-2; For more information about the forecasts, please contact d.fouarge[at]maastrichtuniversity.nl). These forecasts predict the job opportunities for 113 different occupational groups for the coming six years. The Qompas occupations are matched to these occupational groups. The information about the gross hourly wage is taken from administrative data. It is the mean hourly wage for individuals who graduated from an intermediate vocational education program and work in the occupational group associated with the Qompas occupation. The reason for not taking the seemingly more relevant median hourly wage is that preparatory vocational education students probably know the definition of the mean, but may not know the definition of the median.

Treatment 1 and 2 are given information about just the job opportunities. Treatments 3 and 4 get information on both the job opportunities and the hourly wage levels for different occupations.

In treatments 1 and 3, the information is presented by a reseacher from the Research Center for Education and the Labor Market. These `information senders' are divided into four groups: low-status males, high-status males, low-status females and high-status females. In this context, status is defined by the seniority of the information sender. A researcher that does not have a Ph.D. (yet) is considered low-status, whereas a researcher with a Ph.D. is considered high-status. To ensure understanding, the designation presented to students is either 'beginning researcher' or 'experienced researcher' (In Dutch: `beginnend onderzoek(st)er' and `ervaren onderzoek(st)er'). The reason for not presenting the different statuses as `junior' and `senior', respectively, is that we are again worried about a lack of understanding. `Beginning' and `experienced' are more commonly used in the scenario described above in Dutch than in English. For each sender, the name and status are shown. Gender is not explicitly mentioned, but the names of all senders are indicative of their gender and the Dutch word for `researcher' is different for men and women. No pictures of the sender are shown, so as to avoid bias caused by appearance unrelated to status or gender.

In treatments 2 and 4, no human information sender is specified. Instead, students are told the Research Center for Education and the Labor Market provides them with the information.

To decide on the way in which to display the information, we held a short survey among students of a high school that does not work with Qompas, but is otherwise very similar. Approximately 72% (44 out of 61) of students indicated that the current way of displaying information was the most clear to them.

We are primarily interested in the effect of the information about job opportunities. Therefore, we did not include a treatment group that only gets information about hourly wages, so as to preserve power. The main reason for this is that the information about job opportunities is a forecast of the situation in six years, which is more relevant for the students than the current hourly wage levels.

Next, after the first ranking (control group) or information provision (treatment groups), all groups are shown a video (https://www.youtube.com/watch?v=YJ78VDQrO3c) about work in general. We do this to allow both treated and non-treated students some additional reflection time. The video does not mention any particular occupations or the importance of job opportunities and wages.

After the video, the students are asked to make a second ranking. They are shown their first ranking and asked whether they want to change anything. We refer to this as the posterior ranking at second elicitation. After this second ranking, beliefs are again elicited. We refer to these beliefs as the posterior beliefs.

It is possible that students select occupations with very bad labor market prospects only. Based on a sample of historical data, approximately 20% of students is expected to select only such occupations. In this case, providing information is not very useful. To deal with this, students who selected only occupations that have very bad, bad or reasonable job opportunities are shown a number of alternative occupations. The occupations suggested are those with the best labor market prospects out of the twenty occupations they selected, ranked first on job opportunities and then on hourly wages. The difference between the control group and treatment groups lies in the information provided. The control group gets no information about why these occupations are suggested. The treatment groups are told these occupations have better prospects. Treatments 1 and 2 then get information about just the job opportunities and treatments 3 and 4 get information about both the job opportunities and the hourly wages.

After the alternative occupations are suggested, the students can decide to learn more about these occupations or not. We refer to this as the decision to learn more about the alternative occupations or not. If the students decide to learn more about these occupations, they are thereafter asked whether they want to change their ranking one last time. In this ranking, the students are allowed to include the alternative occupations. Initially, the new occupations are placed at the bottom of the ranking in a random order. We refer to this ranking as the posterior ranking at third elicitation. The final top five occupations are subsequently included in the student file.

A number of months after the experiment, the second-year students have to choose a study profile. Students are supposed to indicate what study profile they chose in the `definitive profile choice survey'. We refer to this as the actual profile choice. We also include a question about the intended profile choice at the end of the experiment. In the analyses, we refer to this as the posterior intended profile choice. The fourth-year students have to choose a study program a number of months after the experiment. This choice is captured in one of the other Qompas assignments. Naturally, the study program choices of the current third- and second-year students are captured a year and two years later, respectively.
Experimental Design Details
Not available
Randomization Method
We employ a stratified randomization procedure at the school level. The reason for randomizing at the school level is twofold. First, it reduces the chance of there being spillover effects between the different treatments. Second, we expected schools to be less willing to participate if some of their students were to be provided with information, whereas others weren't. As stated, 286 schools participate in the experiment. Last year, the occupation test was completed by approximately 32,750 students. A third of the schools is assigned to the control group. The remaining two thirds are divided equally over the treatment groups.

Schools are stratified on the basis of three characteristics: the available study profile choices in the school, the number of students that completed the occupation test last year, and the quality of life indicator of neighborhoods the students come from. For the available profile choices, we rely on data from Qompas. This data includes information on what study profiles schools do not provide. Unfortunately, no data can either mean that the school provides all profiles or that it did not register the possibilities in the Qompas system. The number of students that completed the occupation test last year is also registered by Qompas. However, data is again not available for all schools. If no data is available in this case, however, the number is predicted using the number of newly registered students in the Qompas system and the total number of students in the school itself (Data from Dutch education executive agency; https://duo.nl/open_onderwijsdata/databestanden/vo/leerlingen/leerlingen-vo-2.jsp; Retrieved: 22-06-2018). If data on one of the two is not available, the number is predicted using just the available measure. For the quality of life in neighborhoods students come from, we rely on the quality of life indicator developed by the Ministry of the Interior and Kingdom Relations (https://data.overheid.nl/data/dataset/leefbaarometer-2-0---meting-2016; Retrieved: 22-06-2018). All neighborhoods in the Netherlands have a score, ranging from 1 (very low quality of life) to 9 (very high quality of life). For every school, we calculate the weighted average quality of life indicator score of the neighborhoods the school's student body comes from. If no data on the residential location of students is available, we predict the average quality of life indicator score using the score of the school's neighborhood.

We use a block design to randomize. Because the profile choice is one of our outcome variables and largely determines the variety of occupations the students are likely to be interested in, we first seek balance on this dimension. We divide the schools into three groups: predetermined choice (only one profile type available), limited choice (between one and three profile types available) and unknown. In many cases, schools in the unknown category will offer all profile types. Within these groups, we subsequently rank schools based on the number of students that completed the occupation test last year. Groups are split in three based on this dimension. Because schools vary a lot in size, this improves the balance in terms of sample size. Lastly, within each of the now nine groups, schools are ranked on the basis of the weighted average quality of life indicator score. These groups are then further split in two. Increased balance on this dimension is important as we intend to estimate heterogeneous effects based on the indicator. In the end, we are left with eighteen strata.

To assign schools to a treatment group, we use randtreat, a user-written STATA command (Alvaro Carril, 2015. "RANDTREAT: Stata module to randomly assign treatments uneven treatments and deal with misfits," Statistical Software Components S458106, Boston College Department of Economics, revised 13 Apr 2017.) Within each stratum, schools are randomly assigned to the different treatment types according to the specified division. As not every stratum contains a perfect multitude of six schools, this will result in some schools not being assigned. We deal with these unassigned schools by recreating strata as mentioned above, omitting the division in two based on the weighted average quality of life indicator score. Within each of the now nine strata, schools are again randomly assigned. For unassigned schools arising from this procedure, we repeat the procedure once more, now stratifying only based on the freedom of profile choice. The last few remaining unassigned schools are sorted based on the freedom of profile choice. They are then assigned based on a randomly ordered list of the control and treatment groups.

As stated, within the sender treatment (treatment groups 1 and 3), the actual sender shown is randomized at the individual level. This means that each student part of a school allocated to treatment 1 or treatment 3 is shown a random sender.
Randomization Unit
School
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
286 schools
Sample size: planned number of observations
Approximately 35,000 students
Sample size (or number of clusters) by treatment arms
96 schools control, 47 schools treatment 1, 47 schools treatment 2, 48 schools treatment 3, 48 schools treatment 4
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number