Health conditions and discrimination in the labor market

We set up a field experiment in which fictitious job applications signaling candidates with different health conditions (treatment) are sent to employers with advertised jobs (subjects). Field experiments have the advantage that they are carried out in a real environment and that employers (subjects) are not aware of the experiment. The field experiment will measure the causal relationship between the specific health conditions (treatment) and the chance of getting a first interview for a job opening.

2023-04-01

2024-08-31

Our primary end-point is the callback rate defined as the share of applications having received an invitation for a follow-up interview.

We send fictious CVs to job vacancies. We send 4 applications to job openings, composed of a CV and a motivation letter. We signal three types of health conditions and one control (the undisclosed health condition). Health conditions are signaled in both the CV and the motivation letter.

Applications for a job opening are composed of a CV and a motivation letter. We have four different application types: three treatments, which include two physical health conditions (wheelchair users, and obese individuals), and a neurodevelopmental condition (Asperger syndrome) and one control (the undisclosed health condition). Health conditions are signaled in both the CV and the motivation letter.

Treatment 1 refers to the obese individual. Obesity is signaled with a portrait photo showing an obese individual in the CV. Additionally, there is reference to caritative work experience for associations fighting against obesity. The photos used result of the manipulation of normal-weight photographs into appearing obese. We have selected 26 gender-balanced photos on the basis of the results of a web survey aiming an evaluation regarding age, attractiveness, nationality and weight.

Treatment 2 refers to the wheelchair user, signaled with work experience for association related to physical handicap and mention in the reference letter of use of a wheelchair due to a due to a congenital injury in the back. Treatment 3 refers to Asperger syndrome which is mentioned in the motivation letter as well as CV mentioning caritative work for an association related to autism.

The control is an application which does not disclosure any health condition. The CV also mentions to caritative work in Unicef.

1.Subjects
Subjects are firms posting job offers for specific occupations in the main Spanish and French job search platforms. We include occupations that give a representative picture of the labor market. Some of the occupations are high skill others low skill, some include high contact with clients, others not. The occupations we apply are Auditor, IT-programmer, receptionist, shop sales assistant, tele-sales assistant, administrative assistant, client advisor, community manager, mechanical engineer, production engineers, production operators, sales manager.

Newly added exploratory analyses, as of July 2024
About two-thirds into the data collection (in June 2024) it was clear that firms do not discriminate job applicants with a disability. To explore one mechanisms behind this result, we decided to add an ethnic minority applicant for the reminder of the experiment. This allows us to further explore whether firms discriminate against other minority groups (job applicants with a North-African origin have been found to be discriminated in other studies), even if not discriminating against those with a disability. We start adding a third applicant with ethnic minority background as of July 2024 (N = ~300). Ethnic minority background is signaled through common North-African-sounding names (i.e., Fatima Saidi and Mohamed Saidi for female and male applicants, respectively). Beyond the names, the application material for these applicants is the same as for the other candidates, including for example education obtained in France, thus signaling a second-generation immigrant background. Since these analyses are exploratory, no power calculations are conducted, and the number of observations is inevitably smaller than in the main analysis. The results from these exploratory analyses must therefore be interpreted with caution and should be considered as suggestive, not conclusive.

Randomization
We send fictious CVs to job vacancies within the occupations considered. To each job offer, we send the three treatments and control. For each occupation, we create 4 versions of a CV and motivation letter corresponding to the three treatments and the control used. For the obese profile we have 26 different photo-applications, for which we randomly pick one. The randomization ensures that in the aggregate health conditions and gender of the applicants are fully balanced.
To avoid order effects associated with the treatments (the health conditions) we create a predefined sequence of applications to be sent to job vacancies. This sequence establishes that the order of applications of treatments and control sent varies across job vacancies (subjects). We send four applications to each job vacancy, sequentially in different days, but making sure all four applications are sent within 4-7 days. The randomization ensures that in the aggregate health signals and gender of the applicants are fully balanced.
3. Experimental design
The field experiment is to be conducted between June 2023-December 2024. During this period, job advertisements in selected occupations found on a few different employment search websites will be collected. A clear majority of employers posting vacant jobs at these sites want applications to be sent in by e-mail. This facilitates our experimental design of attaching photos to the job application, since it can be done electronically. Callbacks for a job interview will be received via telephone or e-mail. To minimize inconvenience to the employer, invitations will be promptly declined.
We have done a pilot of the experiment in a pilot of the experiment over April 2022-August 2022. Simple averages show difference in treatment across health conditions being women in wheelchair the least successful applicants. The objective is to increase precision of the estimates with more applications sent.

4. Deception
The correspondence testing methodology relies on the use of deception, a practice deemed acceptable by ethical review boards when the societal value of the research outweighs the potential costs associated with deception. The pursuit of equal opportunities for all citizens in the hiring process holds significant social value. Notably, the EU Code of Ethics for Socio-Economic Research supports the use of deception in correspondence testing studies on hiring discrimination (Dench, Iphofen and Huws, 2004, Zschirnt, 2019). However, these guidelines also emphasize the importance of minimizing the impact on employers undergoing the study. Adhering to these strict protocols, we have taken measures to reduce the time and effort required by employers to evaluate the additional job application. The deliberate use of minimal deception ensures that research subjects are not unduly harmed.
We must consider the consequences of sending fictious applications and subsequently declining an invitation in our specific experiment. We observe employers recruiting in their normal hiring situation, and we do not lure employers into a new and unfamiliar situation in which they are enticed to deviate from their normal behavior. We limit the burden on the employers’ time by considering an employer only once. Any invitations to interviews are quickly and politely declined in order to keep the extra effort to a minimum. By following these rules the loss of time should be considered minimal. Finally, data is anonymized and the results presented in aggregate form.
We believe that our intervention plays a minor role in the broader job market dynamics, and that deception does not harm subjects beyond what is usual in reality, or in a typical economic experiment without deception (see Appendix II for a note on deception).
5. Soundness of the randomization procedure
We send the applications as job vacancies are posted following the sequences pre-determined. Each recruiter faces the three treatments and the control 1. Therefore randomization is guaranteed by design.

companies posting job offers.

Yes

approximately 1759 companies posting job offers.

7036 cvs sent to 1759 companies.

1759, each firm is treated as a unit of experimentation exposed to the three treatments and the control.

In a standard design where there is a hypothesis which means that e.g. comparing proportions in two independent groups, statistical power can be calculated using an analytical formula with assumptions about the size of the treatment effect, the standard deviation of the outcome variable, and the sample size. To perform the power calculation, we need to make assumptions about the expected treatment effects. In previous work in Spain we had baseline callback rates of 17% (see our previous contribution Goulão et al., 2023). Take for a benchmark a callback rate of around 17%. If you aim at detecting that specific health conditions must apply to 25% more job offers this results in a callback rate of 13.6% (0.136=0.17/1.25). To detect such significant differences in callbacks we need to respond to N=1,759 jobs per country.