Job Preference Signaling and Matching Efficiency: Experiment in a Large Online Job Market

Last registered on October 27, 2022

Pre-Trial

Trial Information

General Information

Title
Job Preference Signaling and Matching Efficiency: Experiment in a Large Online Job Market
RCT ID
AEARCTR-0009327
Initial registration date
April 28, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 29, 2022, 10:39 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
October 27, 2022, 11:31 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Florida International University

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2022-02-01
End date
2023-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Asymmetric information about job applicants' preference—i.e., how interested they are in the jobs they apply for—may cause inefficiency in job matching. The inefficiency can be theoretically reduced by credible signaling of job preference. To empirically examine the effects of job preference signaling, we conduct a large experiment in Bangladesh’s largest online job portal. By applying a novel and ethical randomization design, our experiment estimates the effects of signaling on application-, jobseeker-, and employer-level outcomes. The signaling mechanism in the experiment allows jobseekers to express their interest to few employers. It is essentially identical to what has been implemented in the economist job market by American Economic Association.
External Link(s)

Registration Citation

Citation
Matsuda, Norihiko. 2022. "Job Preference Signaling and Matching Efficiency: Experiment in a Large Online Job Market." AEA RCT Registry. October 27. https://doi.org/10.1257/rct.9327-2.1
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Our intervention is a job preference signaling mechanism that allows job applicants to express their interest to few employers. It is essentially identical to the one implemented in the Job Openings for Economists by the American Economic Association. Below are specifics about our intervention:
- Endowment of signal: Each jobseeker is endowed with one signal every week. If a signal is not used during the given week, the signal is lost, not carried over to next weeks.
- Timing: Jobseekers can send signals when and after they apply for jobs. Signals can be sent to any active jobs, regardless of whether jobs are posted in the same week or previous weeks. Signals will be attached to job application and delivered to employers immediately.
- Once signals are submitted, they cannot be cancelled.
- No more than one signal can be sent to any job.

The experiment is conducted in the entirety of a large online job portal. Every jobseeker is endowed with a signal every week, which can be sent to any active job ad.
Intervention (Hidden)
The online job platform is Bdjobs.com.
Intervention Start Date
2022-03-28
Intervention End Date
2022-11-11

Primary Outcomes

Primary Outcomes (end points)
[Job application-level outcomes]
- Whether an application is viewed
- Whether an applicant is shortlisted

[Jobseeker-level outcomes]
- The number of applications that are viewed by employers
- The number of applications that are shortlisted
- Being employed
- Job match quality: (i) monthly salary; (ii) how well worker’s skills fit his/her job; (iii) how interesting a current job is; (iv) overall job satisfaction; (v) job tenure

[Job/employer-level outcomes]
- The number of applications viewed
- The number of applications shortlisted
- Whether a position is filled; the number of hires
- Job match quality: (i) employer’s subjective assessment on hired workers’ skills; (ii) subjective assessment on workers’ motivation; (iii) employer’s overall satisfaction on hired workers; (iv) worker retention
- Employer’s satisfaction on recruitment experience in the job portal
Primary Outcomes (explanation)
We will create an index of job match quality.

Secondary Outcomes

Secondary Outcomes (end points)
[Job application-level outcomes]
- Whether an applicant is hired

[Jobseeker-level outcomes]
- Aspired wage in the future
- Life satisfaction
- Whether looking for a job in last 4 weeks
- Self-perceived job search prospects: the percent chance of finding a full-time job within 6 months
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experiment takes place in the entire job portal. During a 4-month experiment period, every jobseeker can send one signal per week. They can send signals to any active jobs when and after they apply for jobs.

Employers can view whether applicants send signals to their jobs. More specifically, on the employer account page, employers can first view the list of all applicants. This list shows summary information about each applicant including whether s/he sent a signal. Then, if employers click an applicant from the list, employers can open his/her application package and view his/her detailed information.

The data are the following:
1. Administrative data of the job portal: The data include the information about jobs, employers, jobseekers, applications, and signals available in the job portal.
2. Baseline online survey data: We conduct baseline online surveys of jobseekers and employers. The survey population is all active jobseekers and employers registered in the job portal.
3. Endline online survey data: We conduct online surveys to collect jobseekers and employers approximately 2 months after the end of the experiment period. The survey population is all active jobseekers and employers registered in the job portal.
4. Endline phone/video interview survey data: We conduct phone- or video-call interviews of employers twice approximately 2 and 9 months after the end of the experiment period. Note that the endline surveys are subject to funding availability.
Experimental Design Details
[Randomization]
Our randomization is only across employers. There is no randomization across jobseekers. Every jobseeker is allowed to send one signal per week. While signals can be sent to any employers, they are disclosed only to randomly selected employers, namely, treatment employers. Signals that are sent to the other employers (control employers) are not disclosed to them. Control employers are not notified of the signaling mechanism at all, so they are presumably not aware of the signaling mechanism. Jobseekers are not informed that employers are randomized or that signals are disclosed only to treatment employers.

[Source of Empirical Identification]
The effects on job/employer-level outcomes will be estimated based on comparison between treatment and control employers. Those on jobseeker-level outcomes will be estimated by comparing jobseekers who send signals to treatment employers and those who send to control employers, because the de facto number of signals is ex post greater for the former jobseekers than the latter. The effects on application-level outcomes will be estimated based on comparison between applications with and without signals to treatment employers’ jobs, controlling for the differences between applications with and without signals to control employers’ jobs.

[Analysis]
a) Employer level: We will regress outcome variables on employer’s treatment status.
b) Jobseeker level: We will regress outcome variables on the number of signals sent to treatment employers and the total number of signals sent. Essentially, among those who used the same number of signals, we will estimate whether the outcome variables differ depending on how many signals are ex post effective (i.e., sent to treatment employers) and how many are ex post wasted (i.e., sent to control employers.
c) Application level: We will regress job application outcomes (e.g., whether an applicant is viewed, shortlisted, and hired) on the interaction between the dummy indicating whether the applicant sent a signal to the job and the dummy for employer’s treatment status. The control variables include the dummy whether the applicant sent a signal to the job and the dummy for employer’s treatment status.
Randomization Method
Randomization done in the job portal
Randomization Unit
Firms
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
8000 firms
Sample size: planned number of observations
[Administrative Data] 8,000 firms; 20,000 job ads; 500,000 jobseekers; 10 million applications [Online Survey] 500 firms; 50,000 jobseekers [Phone/Video Interview Survey] 2,000 firms
Sample size (or number of clusters) by treatment arms
4000 firms
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
The Social and Behavioral Institutional Review Board of Florida International University
IRB Approval Date
2022-03-07
IRB Approval Number
IRB-22-0070
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials