Experimental Design Details
The experiment randomizes individuals who directly visit the Company’s career website. We identify individuals by their IP address and randomly sort them into one of two groups: (1) a Control group where individuals see the original version of the job posting, or (2) a Treatment group, where individuals see a version of the job posting which delete optional credentials, delete adjectives describing skills, and reframe vague credentials.
The Treatment varies depending on the job posting. In some job postings, there are lots of optional qualifications (e.g., “PhD preferred), adjectives describing skills (e.g., “Excellent” before “coding skills”), and vague language/criteria (e.g., “think like your enemy”). We document the treatment for every job posting included in our experiment (i.e., optional qualifications deleted, adjectives deleted, vague language/criteria reframed).
Upon seeing the job posting on the Company’s career website, individuals decide whether or not to apply for the position. For those who decide to apply, the first step is to click on the “Apply Now” button, after which the individual is given the option of applying by signing in (if they already have an account on the Company’s career website), applying using their LinkedIn profile (where data from the individual’s LinkedIn profile is transmitted to the Company), submitting their resume, or applying “manually” by filling out responses to a series of questions.
We are able to track the treatment status of the applicants in our experiment throughout the hiring process (application, initial phone screen, technical phone screen, on-site interview, and offer). If an individual applies to more than one job posting at the Company over the duration of the experiment, we are able to examine the portfolio of their application choices.