The Supply Effects of Additional Screening in Recruitment

In the project we study whether job applicants respond to different screening methods during the recruitment process for a real job. To study this we experimentally vary the mode of screening applicants.

We consider this as the supply side experiment. A future demand side experiment will be pre-registered prior to collecting the demand side data.

2024-04-22

2024-05-30

We collect the following primary outcomes on the supply side:
• Initial Dropout
• Screening Dropout

Initial Dropout – upon receiving an email to take part in a screening task (or in the control continued interest in the job), applicants must click a link/button that confirms their interest. This variable is equal to one if the applicant does not click the button/link and zero if they click the link/button.

Screening Dropout: This is a dummy variable equal to one if the candidate does not complete the screening. A candidate is said to have completed the assessment if they answer all interview questions.

We will analyze these outcomes overall and by job.

To understand possible mechanisms, we elicit the following secondary outcome variables:

Evaluation scores: This is the score of the assessment calculated by the algorithm/human.

Time taken: This is calculated as the time the candidate spends completing the whole screening.

Type of language used: We will use an algorithm to assess the type of language used in applicants’ responses.

Fairness/bias perceptions : We will elicit applicants’ perceptions of whether the interview process they experienced was fair or biased

Interview anxiety: We will elicit applicants’ interview anxiety through the index of McCarthy and Goffin (2004).

Measure of perceived meritocracy of the interview process: We will elicit applicants’ perceptions of meritocracy for different types of interview methodologies.

Recommendation intention: We will elicit the applicants’ willingness to recommend the employer as an indicator of perception about the employer through a modified Reichheld (2003) measure.

See above

Our design aims to measure the impact various types of additional screening stages in the recruitment process on the behavior of job applicants.

Not available

Randomization will be carried out by a computer.

The randomization unit will be the individual for all treatments.

No

Treatment is assigned at the level of the individual so clustering will take place at the individual level.

The clusters will be equal to the number of legitimate candidates. Based on previous experience we expect to have at least 2400 legitimate job applicants across 3 jobs, but this will likely vary depending on a number of factors. However, we plan to keep the jobs open until we have at least 2400 applicants. A legitimate candidate is someone who resides in the United States and completes the initial application form. We plan to assign an equal proportion of the sample to each treatment except for the zoom treatment, which will have a smaller assignment depending on the available evaluating resources. Further, we will stratify by gender such that there is the same proportion of men/women in each treatment.

We plan to have at least 2400 applicants. We will split the applicants equally across the treatments except for the zoom treatment, which will be based on available resources.

Assuming a sample of 400 in each treatment, power of 80% and assuming the proportion of the study population that would have a value of 1 for the binary outcome in the absence of the treatment is 0.50, we expect a MDE of 0.10. The MDE for the zoom treatment will be larger than the other treatments, which implies a potential underpowering in this treatment. This situation arises due to the practical challenges associated with conducting interviews for a substantial number of candidates.