Experimental Design Details
Using relevant resumes from the skill requirements in job advertisements, we construct a large number of application profiles. The applications will be divided into six branches. The applicants are first randomly divided into two groups — those with previous tech-industry experience and those without. Then, within each group, we randomly assign applicants to either have or not have a tech certificate. In addition, we randomly assign applicants to either be young (~22 yo) or older (~35 yo). Finally, we drop individuals who are older and also have previous tech experience, since these individuals will be more experienced tech workers who do not compete in the labor market of interest for this study.
The hypothetical job applicants' resumes have three key components — educational background, previous employment, and skills and qualifications. We design our resumes such that all hypothetical applicants have obtained their bachelor's degree from non-selective public four-year institutions. Their degrees will be in fields that are not directly related to the tech industry, but related to industries that are most affected by skill-biased technological changes (e.g. mechanical engineering). Furthermore, the year of graduation will be listed as between 2012 and 2015, so that the applicants will be perceived to have five to eight years of work experience and to be between 25 to 30 years of age. This allows the applications to reflect backgrounds for individuals seeking a career transition, and also avoids the applicants from being subject to the potential age discrimination in the tech industry.
When designing the previous employment experience of the applicants, we rely on job descriptions from real-world hiring advertisement posted on the job-search platform. We scrape the job descriptions of relevant job advertisements, and rephrase and repackage them into descriptions for the applicants' previous employment. Job descriptions for applicants with relevant tech-industry experience will be sourced from job advertisements in our targeted tech industry verticals. The descriptions of experience for applicants without relevant tech-industry experience will come from job advertisements in industries most-affected by skill-biased technological changes (e.g. manufacturing).
In the skills and qualifications section of the resumes, we will include the certification assigned by randomization and also include certain computer/tech skills that are necessary in applying for tech-industry jobs. For example, basic coding knowledge in Java and working knowledge in Microsoft Excel will be included in all resumes.
We search for appropriate job openings through the online search platform we partner with. In conducting the search, we will use key words such as “technology”, “information technology”, or “IT manager”, and also restrict search to jobs requiring zero to three years of tech-related experiences.
We will create a large amount of email accounts to be associated to the job applications. Our study team has established partnership with a US enterprise telecommunication service company that will provide us with a large number of phone numbers with voicemail services. When sending out the job applications, members of the study team will be instructed to not provide any information that are not designed into the resumes. This step is to ensure that the randomization design stays intact and also avoids the risk of raising concerns from the hiring firms.
The study team will record call-backs by regularly monitoring the email accounts and by recording the voicemails in the phone accounts. To avoid the risk of disclosing additional information and exposing the artificial nature of the study, the members of the study team will be instructed to not have direct phone conversations with the employers. In a previous audit study, Zhu (2020) documents that more than 95% of employers leave voicemails with sufficient information to document call backs when they call audit study job applicants. If an employer calls the job applicant for more than three times, the study team will reach out to the hiring office of the employer indicating that she is no longer looking for a job. No interview opportunity or jobs will be accepted by the study team.
In the resume review part of the study, we send a random subset of the artificial job applications to professional resume reviewers. The subset will be representative of the universe of applicant-job advertisement pairs from our audit study, while preserving the four basic groups of job applicants. We also provide a de-identified summary of the job opening associated with each application. Then, we ask the reviewers two questions. First, based on the application and the job description, will you be willing to provide this candidate an interview opportunity? Second, based on the application and the type of job the individual is applying to, what expected salary they would offer? The reviewers are potentially aware of that they are part of an experiment. This is because in most real-life situations of resume rating, reviewer will also have access to the name of the employer and many other identifying details, which we will have to omit from our IRR study because of confidentiality concerns. Reviewers will be hired at market rates.