Job Preference Signaling and Matching Efficiency: Experiment in a Large Online Job Market

Our intervention is a job preference signaling mechanism that allows job applicants to express their interest to few employers. It is essentially identical to the one implemented in the Job Openings for Economists by the American Economic Association. Below are specifics about our intervention:
- Endowment of signal: Each jobseeker is endowed with one signal every week. If a signal is not used during the given week, the signal is lost, not carried over to next weeks.
- Timing: Jobseekers can send signals when and after they apply for jobs. Signals can be sent to any active jobs, regardless of whether jobs are posted in the same week or previous weeks. Signals will be attached to job application and delivered to employers immediately.
- Once signals are submitted, they cannot be cancelled.
- No more than one signal can be sent to any job.

The experiment is conducted in the entirety of a large online job portal. Every jobseeker is endowed with a signal every week, which can be sent to any active job ad.

2022-03-28

2022-11-11

[Job application-level outcomes]
- Whether an application is viewed
- Whether an applicant is shortlisted

[Jobseeker-level outcomes]
- The number of applications that are viewed by employers
- The number of applications that are shortlisted
- Being employed
- Job match quality: (i) monthly salary; (ii) how well worker’s skills fit his/her job; (iii) how interesting a current job is; (iv) overall job satisfaction; (v) job tenure

[Job/employer-level outcomes]
- The number of applications viewed
- The number of applications shortlisted
- Whether a position is filled; the number of hires
- Job match quality: (i) employer’s subjective assessment on hired workers’ skills; (ii) subjective assessment on workers’ motivation; (iii) employer’s overall satisfaction on hired workers; (iv) worker retention
- Employer’s satisfaction on recruitment experience in the job portal

We will create an index of job match quality.

[Job application-level outcomes]
- Whether an applicant is hired

[Jobseeker-level outcomes]
- Aspired wage in the future
- Life satisfaction
- Whether looking for a job in last 4 weeks
- Self-perceived job search prospects: the percent chance of finding a full-time job within 6 months

The experiment takes place in the entire job portal. During a 4-month experiment period, every jobseeker can send one signal per week. They can send signals to any active jobs when and after they apply for jobs.

Employers can view whether applicants send signals to their jobs. More specifically, on the employer account page, employers can first view the list of all applicants. This list shows summary information about each applicant including whether s/he sent a signal. Then, if employers click an applicant from the list, employers can open his/her application package and view his/her detailed information.

The data are the following:
1. Administrative data of the job portal: The data include the information about jobs, employers, jobseekers, applications, and signals available in the job portal.
2. Baseline online survey data: We conduct baseline online surveys of jobseekers and employers. The survey population is all active jobseekers and employers registered in the job portal.
3. Endline online survey data: We conduct online surveys to collect jobseekers and employers approximately 2 months after the end of the experiment period. The survey population is all active jobseekers and employers registered in the job portal.
4. Endline phone/video interview survey data: We conduct phone- or video-call interviews of employers twice approximately 2 and 9 months after the end of the experiment period. Note that the endline surveys are subject to funding availability.

[Randomization]
Our randomization is only across employers. There is no randomization across jobseekers. Every jobseeker is allowed to send one signal per week. While signals can be sent to any employers, they are disclosed only to randomly selected employers, namely, treatment employers. Signals that are sent to the other employers (control employers) are not disclosed to them. Control employers are not notified of the signaling mechanism at all, so they are presumably not aware of the signaling mechanism. Jobseekers are not informed that employers are randomized or that signals are disclosed only to treatment employers.

[Source of Empirical Identification]
The effects on job/employer-level outcomes will be estimated based on comparison between treatment and control employers. Those on jobseeker-level outcomes will be estimated by comparing jobseekers who send signals to treatment employers and those who send to control employers, because the de facto number of signals is ex post greater for the former jobseekers than the latter. The effects on application-level outcomes will be estimated based on comparison between applications with and without signals to treatment employers’ jobs, controlling for the differences between applications with and without signals to control employers’ jobs.

[Analysis]
a) Employer level: We will regress outcome variables on employer’s treatment status.
b) Jobseeker level: We will regress outcome variables on the number of signals sent to treatment employers and the total number of signals sent. Essentially, among those who used the same number of signals, we will estimate whether the outcome variables differ depending on how many signals are ex post effective (i.e., sent to treatment employers) and how many are ex post wasted (i.e., sent to control employers.
c) Application level: We will regress job application outcomes (e.g., whether an applicant is viewed, shortlisted, and hired) on the interaction between the dummy indicating whether the applicant sent a signal to the job and the dummy for employer’s treatment status. The control variables include the dummy whether the applicant sent a signal to the job and the dummy for employer’s treatment status.

Randomization done in the job portal

Firms

No

8000 firms

[Administrative Data] 8,000 firms; 20,000 job ads; 500,000 jobseekers; 10 million applications [Online Survey] 500 firms; 50,000 jobseekers [Phone/Video Interview Survey] 2,000 firms

4000 firms