Experimental Design Details
Employers post job ads in 51 industries associated with 59 job occupations. These ads must follow a standard template to describe important information about the job and any necessary employee characteristics. A job ad is finished with an “apply” button at the end of the job ad webpage. Clicking this button means a job seeker has successfully applied for a job and has sent his/her pre-generated resume to the employer. However, if the job seeker has not logged into the job board’s account when clicking the “apply” button, it will open a login page requesting logging into the job board’s user account.
To apply for jobs, job seekers need to register a user account and fill in required individual information to construct a resume. Job seekers can search for jobs that they are interested in and apply to jobs. (Job seekers can search and read the job ads without registering or logging into an account but they must log into the job board’s account before they can apply a job. Recently, more and more users are switching to mobile device such as smart phone to use this website at the time of the experiment.) Alternatively, employers can search for potential employees and invite them to apply. Job seekers can choose whether to make their resumes (without personal identification information such as name and contact information) publicly available. But in any case, when a job seeker sends a resume to apply for a job, the corresponding employer can always receive the complete resume. After receiving qualified resumes, most employers choose to contact the applicants directly by offline measures such as telephone or email for further recruiting procedures, although the job board also provides a communication system for this purpose. This implies that it is rarely possible for the job board to observe the final result of the applications.
To facilitate dissemination of job ads, the job board provides an email pushing service to employers, which helps send emails containing job ad information to potentially suitable candidates based on matching criteria. A job seeker receives job recommendation emails depending on the information provided in his/her resumes and/or the job searching behavior on the job board. An active job seeker receives 1-2 emails per day during the time when he/she searches for jobs intensively.
Experimental setting – employer, positions/occupations and job seekers
We collaborate with a start-up company in the information technology (IT) industry that had real recruitment demand for several positions and intended to explore flexible work conditions. We chose a company in IT because this industry had the most jobs in the online job board and commonly to allows flexible working conditions. We chose a start-up company to ensure that the company was not well-known, and so that our large-scale intervention would not affect the market.
The experiment involves the company recruiting for six occupations/positions. The six occupations are feasible for setting up independent remote tasks that can be performed with an internet connection from different locations at different times. These occupations are also among the most wanted/popular jobs on the job board.
We define eligible job seekers based on the following criteria: (1) current residence is Beijing; (2) college education or above; (3) active job searchers – that is, they need to have logged into their account in the job board within the past month prior to the starting day of our experiment; (4) any of the “intended occupations” match our chosen occupations.
We use a 2 × 2 factorial design in which we vary the flexibility conditions. To provide more detail, in the NoFlex treatment, the job does not contain any timing or place flexibility; that is, the employees need to come to the office of the company to work for 8 hours at regular working hours between 9 am and 6 pm with a one-hour lunch break, from Monday to Friday. In the TimingFlex treatment, on Monday the condition is the same as in the NoFlex treatment; from Tuesday to Friday, employees are allowed to work in office for 8 hours starting between 7 am and 10 pm and ending between 4 pm and 7 pm. In the PlaceFlex treatment, on Monday, the condition is still the same as in the NoFlex treatment; from Tuesday to Friday, employees are allowed to work wherever they like between 9 am and 6 pm by logging into the company’s online working system. In the FullFlex treatment, both timing and place flexibility conditions are applied: on Monday, the condition is still the same as in the NoFlex treatment; from Tuesday to Friday, employees are allowed to work wherever they like via logging into the company’s online working system for 8 hours starting between 7 am and 10 pm, and ending between 4 pm and 7 pm.
We use a two-stage procedure. In the first stage, following the conventional recruiting procedure used by the employers on the job board, we first published our job ads, which followed the standard job ad template of the job board, on the employer’s webpage. Right after doing this, we asked the job board to push the job ads to our sampled eligible job seekers, via email, and gave them six days to apply.
For the email recipients, applying to the job takes two steps. First, they needed to click the “apply” button at the end of the job ad email. Clicking this button directs the job seekers away from their emails to the corresponding job ad on employer’s webpage on the job board. Second, they needed to click the “apply” button on the webpage to actually apply to this job and send their pre-generated resume to the employer. The transformation from the email system to the webpage was standard for all email recipients contacted by this email pushing service, and all the actions on the job board could be recorded.
In the second stage, for those who had applied, we then sent them an email including the link to a questionnaire, which they were requested to fill out within a week.
To avoid missing the job ad due to not checking emails regularly, we sent a text message informing/reminding them of the questionnaire one day after the questionnaire was distributed. Communication with subjects was only via emails and text messages, and every interaction was carefully scripted.
We will conduct the four treatments sequentially one at a time for each of the six occupations, because all the active job ads must be shown on the employer’s webpage on the job board. It does not make sense to post different work flexibility conditions for the same job occupation at the same time. The experiment lasts for four weeks, one week for each treatment. We simultaneously operate the experiment for the different occupations. To control for the potential temporal confounds due to the appearance order of the treatments, we randomly alter the sending order of the job ads of different treatments for the six occupations to control for the “seasonal”/ “order” effect of treatments across occupations.
We publish the job ads for designated treatment on Thursday morning Beijing time each week, and state explicitly in the ads the application deadline to be 9 am the following Wednesday, beyond which the job ads on the job board will become invalid and the applications will not be considered. For each occupation, we post ads for each treatment within a short enough time frame (one treatment a week, in total one month) to minimize any possible temporal-based influence.