Experimental Design Details
Our experiment, focusing on entry-level jobs for college graduates, will be conducted between September 2024 and July 2026. We will target entry-level job positions that require either a 3-year associate’s degree or a 4-year bachelor’s degree and no more than three years of work experience. Fictitious resumes will be submitted to online job advertisements posted on two major platforms in China.
Stage 1: Determining the sample size
To calculate the required sample size, we conduct a power analysis. For a two-group t-test, power analysis requires three key pieces of information: the desired significance level, power level, and effect size. Typically, the significance level, or Type I error, is set at 0.05, while the power level—representing the probability of correctly rejecting a false null hypothesis—is set at 0.8 . Given that our study involves multiple comparisons, we also adopt the Bonferroni correction to adjust the significance level and reduce the risk of false positives.
Stage 2: Choosing Target Firms and Job Posts
We consolidate the 134 original categories that were recruited by at least 50 companies, representing approximately 73% of all job advertisements in 2023 into 23 experience-based occupations. We then focus on the top 16 experience-based occupations, which represent over 97% of all job postings, to further refine our selection of target firms.
We begin by targeting job vacancies in publicly listed companies in China, enabling us to assess the distribution of gender discrimination within some of the country’s most promising and economically influential firms. Since we also aim to investigate gender discrimination at firm level, we focus on selecting firms that can provide sufficient sample sizes to examine gender discrimination and occupation-specific hiring patterns within firms. We identify that 188 firms meet the effective sample size of 1,048. We select these 188 firms as our core firm groups, while the remaining firms are kept in reserve. These companies demonstrate sufficient geographic variation, with job postings in more than three provinces in 2023, and they posted over 11 occupations in 2023 (See details in "Research design in detail.docx").
Stage 3: Resume Construction
Unlike a typical US audit study, which applies for jobs by emailing or uploading pre-generated resumes, Chinese job seekers submit their resume information through filling out a standard application form. This template consists of six sections: personal information, educational history, student leadership experiences and honors, work experience, skills and certificates, a self-statement and career objectives. Specifically, we randomly assign applicants to one of four types (Male/Female×(non) Elite Universities) within each vacancy.
Group 1: Male and Elite University
Group 2: Male and non-Elite University
Group 3: Female and Elite University
Group 4: Female and non-Elite University.
The resumes will be randomly generated by a Python program and self-designed inputs. Gender and education assignments will be stratified. Other resume characteristics will be unconditionally randomly assigned.
Dates of birth and the start of work experience are required fields in the standard application template provided by Chinese online job boards. We calculate these dates to match the educational background (starting school at age 6, followed by 9 years of compulsory education, 3 years of high school, and 4 years of undergraduate study) and the work experience required for the job. Our fictitious applicants are between 22 (the senior students actively seeking employment) and 25 years old (those who have graduated with a bachelor's degree within the past three years).
Working Experience and Internship: Work experience and internship details are also scraped from real resumes and classified according to major. If no specific work experience is required, the applicant's work history includes two internships, each lasting between 3 and 6 months, reflecting the most common patterns observed in real resumes. We ensure that the internship experiences are as relevant as possible to the target job vacancy. For positions requiring work experience, we assigned one full-time job that matches the number of years specified in the job advertisement, supplemented by an additional internship. We avoid any employment gaps, and all applicants are presented as currently employed by their most recent employer. Furthermore, all work histories are in the same city as the job posting.
Contacts: We will register about 128 email addresses at 163.com, one of China's largest email service providers. Each email address will be created by combining the assigned first and last names with random integers. Additionally, we will purchase about 128 local mobile phone numbers for the corresponding residential cities. Each number will be assigned to a specific name.
Stage 4: Submitting Resume to Target Firms
Match Job Vacancies and Resumes: We will submit gender-differentiated resumes to job vacancies that meet our criteria: those posted by our target firms and belonging to our screened 16 specific occupations. These resumes will be constructed and classified by majors, ensuring they are nearly identical in characteristics other than gender. For each qualified job vacancy, we will send male and female resumes on two consecutive days, in random order. We developed a code to scrape daily job vacancies posted on online job boards for each firm. The collected information includes the firm name, job requirements, and expected salary. We will retain job ads that meet specific criteria: full-time positions requiring a bachelor's degree with three or fewer years of work experience. Additional filters will be applied: jobs offering salaries below the minimum threshold (defined as the 5th percentile for all jobs requiring a bachelor's degree) will be excluded, as well as jobs requiring photographs and those posted by intermediary companies.
Submit Resumes: Our 17 research assistants (RAs) will be divided into six groups, each assigned to a specific job sector. Each RA will manage approximately 8 email accounts paired with 8 phone numbers. Female RAs will handle the fictitious resumes for female applicants to ensure that any phone notifications from employers are appropriately managed, although most notifications are expected to be sent via email. For each job vacancy, two pairs of resumes will be submitted within one week. For instance, the first pair (female + Elite university and male + Elite university) will be submitted within the first three days of the week, with at least a four-hour gap between the two submissions to minimize the risk of employers recognizing them as linked. The second pair (female + non-Elite university and male + non-Elite university) will be submitted starting one day after the first pair but within seven days of the initial submission. The order of submissions—whether male or female, or the more or less selective pair based on their institution—is randomized.
Stage 5: Data Collection for Analysis
Successfully Receiving a response: We will carefully track employers’ responses to each application through phone calls, text messages, and emails within 60 days.
Stage 6: Quality checks
Building on insights from a pilot study conducted in August 2024, we developed a detailed training manual and will provide on-site training on campus in September 2024. We will also establish strict quality-control procedures during the process of the audit study.