Nonbinary hiring discrimination and the politicization of pronouns: A resume audit study

A resume audit study design will be leveraged to estimate hiring discrimination against nonbinary applicants (signaled on resumes via “they/them” pronouns listed below the name) and cisgender applicants who disclose pronouns (signaled on resumes via binary pronouns congruent with name-implied sex—for example, “she/her” for Madeline and “he/him” for William). Outcomes among these two treatment groups will be compared to a control applicant who does not disclose pronouns on their resume.

2023-05-18

2023-10-20

Positive employer response to job applications is the outcome of interest. Emails and phone lines will be set up and carefully tracked to identify applicants for whom employers respond.

Employer response will be viewed as "positive" if they contact the applicant and either offer an interview or request the applicant contact them.

A team of research assistants will search major job posting websites (Indeed, Monster, Craigslist, etc.) for occupation vacancies in a selection of United States CBSAs. When an appropriate job posting is found, a pair of fictitious, randomized, matched, formatted resumes will be generated and sent in in response. To reduce cost, job postings will only be applied to if the application process involves uploading a resume PDF and answering simple, standardized questions that can be easily determined from the randomized resume (e.g., how many years of relevant experience do you have?) or that can have a standard general response (e.g., can you reliably commute to work at this location?—yes). When applying, information on job posting, employer, and resume characteristics will be recorded in an encrypted database.

Resume characteristics (name, pronouns, objective, work experience, education, skills, certifications) are randomly generated using a program from Lahey and Beasley (2009). During randomization, some characteristics will be matched same (i.e., if the first resume is randomly assigned characteristic A, then the matched pair will also be given characteristic A) and other characteristics matched different (i.e., if the first resume is randomly assigned characteristic A, then the matched pair will be randomly assigned a characteristic aside from A). The characteristic of interest is pronouns, where the research objective is to determine whether similar applicants with different pronoun disclosures receive different positive employer response rates. Once resume content is generated for the matched pair, each will be randomly given one of two possible resume formats.

Employer response will be carefully tracked via phone and email. For each CBSA, two phone lines will be set up using an area code local to the area. To limit fraud detection by email providers and job boards, there are in total two female names and two male names used in each state (e.g., all matched resume pairs in Colorado where the name-implied sex is female will use the same two names). Emails are specific to names, and each name will always use the same phone number when applying in a given city. As a result, with 12 names, 12 emails will be set up in total. Phone voicemails and emails will be monitored on an ongoing basis to identify applications which receive a positive employer response.

Randomization is done in office by a computer. In particular, the computer program by Lahey and Beasley (2009) is used to construct resume randomization logic. A detailed breakdown is included in the pre-analysis plan.

Randomization occurs at the firm (job posting) level.

Matched pairs of resumes are generated simultaneously, where they are matched same in some characteristics (e.g., years of relevant work experience) and matched different in other characteristics (e.g., name). The set of inputs over which randomization occurs is occupation-specific for some characteristics (e.g., work experience) and independent of occupation for other characteristics (e.g., name). These matched pairs are both sent in response to one job posting by a particular firm (location and employer) for a particular occupation.

Yes

3,240 pairs of matched resumes sent to the same number of job postings

6,480 total resumes

Approximately 2160 resumes with "they/them" pronouns, 1080 resumes with binary pronouns congruent with sex implied by name, 3,240 control resumes.

Of these, approximately half will have male-sounding names and half female-sounding names. Approximately half will be in Republican geographies and half in Democratic geographies. Approximately one third will be in female-dominated, male-dominated, and non-dominated occupations. Approximately one third will be in high, medium, and low customer interaction occupations.

For simplicity, I denote applicants who send resumes with nonbinary pronouns “T1” matched to control “C1,” and resumes with binary pronouns congruent with name-implied sex “T2” matched to control resumes with no pronouns “C2.” All of the following is for 80% power, 5% statistical significance. Minimum Detectable Effect (MDE) for all of the following is in terms of percentage point differences between treatment and control positive employer response. All MDE reported are associated with a one-sided proportion test. When comparing T1 to C1 or T2 to C2 and assuming the control group has a 10% callback rate: Full sample (2,160 pairs): -2.20% Half sample (1,080 pairs): -3.05% Third sample (720 pairs): -3.70% Assuming the control group has a 15% callback rate: Full sample (2,160 pairs): -3.08% Half sample (1,080 pairs): -3.67% Third sample (720 pairs): -4.57% When comparing T1 to T2 and assuming the T2 has a 10% callback rate: Full sample (3,240 total): -2.65% Half sample (1,620 total): -3.71% Third sample (1,090 total): -4.47% Assuming the T2 has a 15% callback rate: Full sample (3,240 total): -3.21% Half sample (1,620 total): -4.47% Third sample (1,080 total): -5.38% When comparing T1 in group A to T1 in group B and assuming the T1(A) has a 10% callback rate: Half sample (1,620 total): -3.05% Third sample (1,090 total): -3.70% Assuming the T1(A) has a 15% callback rate: Half sample (1,620 total): -3.67% Third sample (1,080 total): -4.57%

Pre-Analysis Plan

Populated Pre-Analysis Plan