Increasing diversity: Experimental evidence

RCT within the tech firm:

In the RCT, we exogenously vary text blocks at the top of job ads.

These text blocks are part of a so called ‘teaser text’. The ‘teaser text’ is located at the top of the job ads and provides a high-level and general description of the advertised job and the firm itself.

We use a within-job-ad-randomization to identify treatment effects. In the control group, we keep the ‘teaser text’ as it is. In the first treatment (‘flexibility treatment’) we add a text block highlighting the opportunity for flexibility in the advertised position. The second treatment (‘growth treatment’) adds an additional text block (to the control treatment job ad) highlighting the opportunity for future wage growth.

Survey experiment :

The survey experiment is conducted in cooperation with several laboratories for economic research of Universities across Germany.

2022-10-27

2024-04-30

RCT within the tech firm:

number of applications, number and share of female applicants, number and share of male applicants, number and share of non-native applicants

Survey experiment:

measures for applicants’ expectations regarding the working conditions, measures for the participants interest in the job

RCT within the tech firm:

Our first main outcome is the total number of applications.

Our second main outcome variable is the number as well as the share of female, male applicants and non-native applicants.

Survey experiment:

Our first main outcome variables are measures for applicants’ expectations regarding the working conditions, in particular regarding workplace flexibility and compensation. This includes among others the expectations regarding the opportunity for home-office, support in childcare, the opportunity to work abroad, the expected wage, the expected wage growth, job search intensity.

Our second main outcome variables are measures for the participants interest in the job. First, we ask in the survey whether participants would like to apply for the job; note that we will focus in our analyses for the variable mainly on people who live in commuting distance to the respective plants. Second, we measure a signal of the participants that communicated an increased interest in the job to the firm (email to the firm or state their email address to be contacted by an employee of the firm). Third, participants have the opportunity to state why / why they do not apply for the job. We intend to classify these reasons in various categories. Fourth, we measure whether participants indeed apply to the position and how successful they are in the following hiring process in the firm (e.g. whether they get an actual offer for the job).

RCT within the tech firm:

fit of the application to the advertised position, quality of the applicant / application, commuting time to the current employer and our study firm’s plant, residence of the applicants, success of the applicants in the hiring process, probability that at least one woman applies, interaction of the various outcome variables, heterogeneous treatment effects with respect to the required work experience for the job

Survey experiment:

interaction of the various outcome variables

RCT within the tech firm:

Our third outcome variable is the fit of the application to the advertised position (in particular, does the educational background and previous job experience fit to the advertised position).

Our fourth outcome variable is the quality of the applicant / application. Here, we use various measures. First, the classification of the quality of the application by the firm’s recruiters. Second, whether all required documents are submitted, whether there are typos in the application, etc.. Third, the grades of the applicant at university, and whether the university is one of the top universities in the respective field in Germany. Fourth, the quality of job market experience of the applicant (e.g. prestige of the current employer, prestige of the internships). To measure the quality of the application, we will potentially also run expert interviews among HR managers in which HR manager are asked to rate the quality of the applications.

Our fifth type of outcome variables are the commuting time to the current employer and our study firm’s plant as well as the cities of residence of the applicants.

Our sixth type of outcome variables are measures regarding the success of the applicants in the hiring process. First, we want to check how far applicants get in the hiring process (e.g. invited for an interview). Second, we want to analyse which candidates eventually get the job.

Our seventh type of outcome variable is a binary variable being equal to one if at least one women applies for the job.

Note that we are also planning to interact the different outcome variables in order to explore heterogenous treatment effects, e.g. to study the interaction of gender and commuting time, with respect to the required work experience for the job (e.g. entry- or senior-level positions).

Survey experiment:

We are also interested on interaction of the various outcome variables, e.g. whether peoples’ expectation about the job and their interests to apply interact. With respect to heterogeneous treatment effects, we will also study how people’s interest to apply depends on gender, their preferences for certain job characteristics, ability beliefs, their risk preferences, competitiveness preferences, patience and trust/reciprocity.

RCT within the tech firm:

In the RCT, we exogenously vary text blocks at the top of job ads.

These text blocks are part of a so called ‘teaser text’. The ‘teaser text’ is located at the top of the job ads and provides a high-level and general description of the advertised job and the firm itself.

We use a within-job-ad-randomization to identify treatment effects. In the control group, we keep the ‘teaser text’ as it is. In the first treatment (‘flexibility treatment’) we add a text block highlighting the opportunity for flexibility in the advertised position. The second treatment (‘growth treatment’) adds an additional text block (to the control treatment job ad) highlighting the opportunity for future wage growth.

Survey experiment:

In the survey, we show participants job ads which are part of the RCT described above. Thus, the job ads are ads for “real” jobs in a “real” company which invites applications for the position. The design of job ads shown to each respondent in the survey will vary (across randomization).

More specifically, we will randomly provide survey participants with one of three different job ads: the control treatment, the flexibility treatment, or the growth treatment.

In the survey we ask among others for demographic characteristics, educational background, preferences, ability beliefs and employment status. Moreover, we elicit participants’ expectations about the working conditions in the firm.

Specifically, we ask for the participants’ expectations regarding the working condition several times. The first time, we do not show them the location for which the job is advertised and ask them to assume that the job is located in a distance with a reasonable commuting time. Then we ask the expectation question again for in total two different cities in Germany (among which is also the true location for which the job is advertised). This allows us to check to which extent expectations are influenced via the location of the job.

Finally, we reveal the true location of the job and tell participants that the company invites applications for the position. Subsequently, we ask respondents whether they want to apply for the job and ask them to reason why or why they do not want to apply to the job. If they answer yes and they have an increased interest for the job, we offer them the opportunity to follow-up with an employee of the firm.

RCT within the tech firm:

We use a within-randomization to identify treatment effects.

The firm receives almost all the applications for jobs within the first 30 days of the job ad being posted online. In our study, we thus focus mainly on observations of the first 30 days. We randomly assign either the control treatment (no text block), the flexibility treatment or the growth treatment to the first ten days in which the job ad is posted online. Among the set of remaining treatments (excluding the one which was assigned for the ten days), we randomly assign one to the following ten-day period. The remaining treatment, which was not assigned to the first two ten-day blocks is assigned to the last ten-days.

The job ad is online until the company agrees that they have received sufficiently many high-quality applications or the position is filled (which is usually within 30 days). If the job ad is online for more than 30 days, the treatment assigned to the last ten-days will be online until the job ad is put offline by the company.

Survey experiment:

We randomize across participants. More specifically, we will randomly provide survey participants with one of the three different job ads: the control treatment, the flexibility treatment, and the growth treatment.

RCT within the tech firm:

We randomize within job ads.

Survey experiment:

We randomize job ads across participants.

No

n.a.

RCT within the tech firm: We start the RCT at the end of October 2022. The company confirmed that we can run the RCT until the end of October 2023; we aim to extent the RCT at least until the end of April 2024. We are not able to predict the exact number of observations in the RCT. Because of that, we summarize here the process how the data will be collected and our expected number of observations. Each year, the tech company is posting on average between 80 and 120 of job ads for engineering, business administration, IT and science (e.g., physics, biology) positions in the plant that cooperates with us. All of these job ads will be part of the RCT. For each job ad, the company receives approximately between 2 and 60 applications. In our RCT, one observation is one application. Standard errors will be clustered at the job ad level. We receive detailed data from the company about each application (e.g. gender, employment history, educational background) and data from the hiring process (e.g. classification of the quality of the applicant from the firm’s recruiters, which applicant is invited to an interview). Survey experiment: We seek to recruit survey participants whose educational background matches with the requirements listed in the job ad. In the laboratory pools, we select participants based on their field of study. All subjects whose field of study fits to the advertised position are invited to participate in the survey. Yet, we do not know how many suitable candidates are available in the laboratory pools, as we do not know which jobs will be advertised by the company during next year (and thus not the requirements). For 15-30 job ads (if possible) we will run the survey in the lab (the exact number depends on the total number of vacancies and the size of the subject pools / response rates in the labs). As there is uncertainty with regard to the required educational background needed for the positions and the positions are partly highly specialized, we do not know yet, whether we will be able to find sufficiently many suitable potential survey participants for the lab. Our aim is to collect for each job ad in total at least 45 observations (at least 15 survey participants for each treatment group), but we aim at 90 observations for each job ad (30 survey participants for each treatment group). Note that it is possible that this number will vary because of varying response rates in the different labs and because of the uncertainty with respect to the fit of the subjects’ background (job experience, education) to the job ads. We made some adjustments compared to our previous version of the preregistration (see the former version from November 18, 2022). Our initial plan was to run surveys for 20 job ads in cooperation with the market research agency Bilendi. However, as we started to pilot our surveys with them, we experienced a very low response quality. Simple quality checks (like reading the job ad for at least 20 seconds) led to a dropout of roughly 75% of the sample. Because of the low answer quality as well as potential selection bias resulting from it, we decided to stop working with Bilendi before analyzing the data; we conduct our whole survey via various econ labs across German speaking countries. Moreover, for statistical reasons, we decided to reduce the targeted number of job ads used in the survey for the sake of an increased number of participants in each survey wave. As the subject pool in the labs are mainly students with no or limited job experience, we are also planning to analyze in our RCT heterogeneous effects with respect to job experience.

n.a.