Highlighting Benefits in Job Ads

Potential job candidates in the work-from-home treatment group see the job ads in a version, which highlights the opportunity to work from home (if the job offers this employee benefit). Potential job candidates in the flex time treatment group see the job ads in a version, which highlights the opportunity to have flexible working hours (if the job offers this employee benefit). In the control group, no job characteristic is especially highlighted.

2023-12-06

2024-06-06

The primary outcome variables are the number of clicks on the ads, the number of applications, the composition of the applicant pool with regards to demographics (gender and age) as well as applicant fit (as measured by different indicators: i) a score the recruiting service provider computes based on the applicants’ answers to personality questions as well as clients’ job requirement profiles, ii) the stage the applicant reaches in the application process iii) interest by the applicant and the client, i.e. whether there was an email correspondence, whether the applicant sent further documents/added information, the applicant’s star rating.
Additionally, we will investigate whether targeted assignment of the highlighting of benefits depending on the job characteristics and the job requirements can increase the number of suitable applicants.

Composition of the applicant pool with regards to other applicant characteristics

Our cooperation company provides recruiting services to a broad range of other companies (“clients”). One of these services is active sourcing, i.e. they actively search for new employees by posting job ads on social media platforms. The social media platforms then display the ads to their user group. To use this service, clients provide a job description as well as additional information, e.g. the employee benefits offered, to our cooperation company.

During the experimental period, our cooperation company posts job ads in three different versions. Each user of the social media platforms can only see one version of the job ads. A user can either see a version highlighting that a job offers work-from-home, a version highlighting that a job offers flexible working hours or a version not highlighting any job characteristic. If a job only offers one of these employee benefits, e.g., it only offers work-from-home, the company does only post two versions, the version without any highlighted job characteristics and the one with the employee benefit offered, i.e. work-from-home. If a job does not offer any of the two employee benefits, it is not included in the experiment

The three versions are shown for up to two months. If the job ad is online for longer than two months, only the version that has generated the lowest costs per lead during the first two months will be displayed after that for budget reasons.

Update: Due to technical difficulties, who can see which version of the job ad is re-randomized after one month. This happens before I have had a chance to look at the data.

Not available

The randomization is done using the Meta Developer API, randomizing on the individual user level.

Individual subject

No

The number of clusters is the same as the number of observations (please see below).

The RCT starts at the beginning of December 2023 and our cooperation company confirmed that we can run it for six months (until the beginning of June 2024). During this period, all job ads that are posted via Meta and offer either work-from-home, flexible working hours or both, will be part of our experiment. The only limitation is that we can only randomize 100 job ads at the same time. Thus, if there were potentially more than 100 ads, which are posted at the same time, we have to prioritize. The company decided to prioritize in this case regarding clients’ booked options. As the number of job ads depends on the number of clients that want to post a job ad during this period, we are not able to predict the exact number of job ads that will be posted. We are also not able to predict the number of applications. Therefore, we provide here information about the numbers in previous months. The number of job ads posted varied between 20 and 120 per month with 0-200 applications per job ad. Not all of these offering either work-from-home, flexible working hours or both. Also, the number of clicks varied between 0 and more than 3000.

The sample size by treatment arm depends on the number of job ads posted that offer the respective employee benefits. Same as for the number of observations, we are not able to predict the number of jobs ads that will fulfill this requirement during our experimental period.