Experimental Design
This experiment will take place online, a website has been designed for this purpose. It concerns job seekers over the age of majority, registered at Pôle emploi, accepting email communications with Pôle emploi, who agreed to respond to surveys emanating from Pôle emploi, and having a valid Reasonable Employment Offer (contracted research with their advisor, Offre Raisonnable d’Emploi [ORE]).
(1) email invitation :
All job seekers concerned by this experiment will receive by email (using a functional email address) an invitation to participate in the experiment, containing a secured link to an ad hoc and secured website designed for the experiment. This email will also contain a short description of the experiment. If the job seeker clicks on the link, he’ll access to the website landing page.
The only exception are job seekers from the pure control group (see below): they will not receive any invitation to participate in the experiment and only information already collected by Pôle emploi in the usual processes will be used.
(2) Access to the online interface/surveys :
The website landing page contains: (1) a short description of the purpose of the study, (2) what use of their data will be done [they will be fully anonymized before being studied for statistics and research purposes only], (3) a request of informed consent clearly stated and (4) a button to notify consent to participate in the experiment and to access the questionnaire for collecting preferences.
(a) Individual preferences data collection :
The job seekers who notify their consent to participate will access to the survey designed to collect individual preferences. This survey is designed in order to be intuitive. Job seekers have to allocate 15 weights to the five criteria (occupation targeted, distance from home, salary, type of contract and working hours) according to their importance. The whole 15 weights must be allocated, the more important the criterion, the more weight the job seeker allocates to it. If a criterion does not matter, the job seeker can allocate zero weight to it, and if only one criterion is important, he can allocate the whole weights to this single criterion. The button to validate is only accessible when the whole weights are attributed.
(b) Internal validity (PILOT STUDY ONLY) :
One way to ensure that job seekers have understood the first survey, is to ask them to confirm what the algorithm suggests using their individual preferences. Using the individual preferences collected through the questionnaire described above as input of the recommendation algorithm, we select 8 of the personalized vacancies and show them reshuffled to the user. He will be asked to sort them from the one he prefers to the one he likes least, giving him the possibility to put aside 2 offers maximum. We then will be able to validate (or not) the way to collect preferences in the first place by comparing their order to this we found using personalized weighted algorithm.
Job seekers who are not concerned by the pilot study will go directly to the next step (c), without going through this step (b).
(c) personalized job offers and link to the Pôle emploi portal :
Then, simplified job offers will be displayed to the user; according to his treatment group, he will have personalized recommendation using the current algorithm based on his ORE’s criteria (Partial treatment group and Control group, see below) or fully personalized recommendation using his ORE’s criteria and his associated personalized weights (Full treatment group, see below). Clicking on simplified job offers redirects the user to the offer published online on the Pôle Emploi website, and exits the interface.
We distinguish three groups of “treated” job seekers (see figure below):
(1) The individualized preferences will be used on a random subsample of fully treated job seekers, to whom job offers recommendations will be made thanks to the matching system using the weighting corresponding to their own preferences. (2) Even if they have filled in the survey to collect their preferences, a partially treated group of job seekers will get recommendations from the initial and fixed weighting. (3) A control group of remaining job seekers will get recommendations from the initial and fixed weighting, without preferences data collection, (4) and finally, we will also consider another subsample of job seekers we will not directly solicit: the pure control group.
- a first pilot study: 10 000 job seekers will be randomly selected for this stage.
This pilot study aims at: (1) testing the job seekers’ understanding of the questionnaire developed to collect their preferences, (2) testing internal validity; and it also aims at (3) identifying if there are some frequent sets of preferences across job seekers, which could help to identify some job seekers' average profiles and also highlight the limits of the current fixed weighting used in the matching system. Depending on the understanding/internal validity of the questionnaire, the researchers may adapt the preference data collection survey.
- the main study: 250 000 job seekers will be selected for this stage (and will be different from the 10 000 of the pilot study).