|
Field
Abstract
|
Before
In this project, we aim to study the ability for organizations to increase diversity through job advertisements that highlight certain job characteristics, namely job flexibility and wage growth. We further investigate how such changes to job advertisements affect the perceived attractiveness of a position and applicants’ expectations in terms of working conditions, working environment, and wages. More generally, we study how the highlighting of certain job characteristics in job ads changes the composition of the applicant pool.
We conduct an RCT and a survey experiment. In this project, we cooperate with a large tech firm. In an RCT, we vary the design of job advertisements to investigate the effects of highlighting different job characteristics on the composition of the applicant pool. The survey experiment is conducted by a market research agency company as well as several laboratories for economic research of universities across Germany.
|
After
In this project, we aim to study the ability for organizations to increase diversity through job advertisements that highlight certain job characteristics, namely job flexibility and wage growth. We further investigate how such changes to job advertisements affect the perceived attractiveness of a position and applicants’ expectations in terms of working conditions, working environment, and wages. More generally, we study how the highlighting of certain job characteristics in job ads changes the composition of the applicant pool.
We conduct an RCT and a survey experiment. In this project, we cooperate with a large tech firm. In an RCT, we vary the design of job advertisements to investigate the effects of highlighting different job characteristics on the composition of the applicant pool. The survey experiment is conducted in cooperation with laboratories for economic research of universities across Germany.
|
|
Field
Last Published
|
Before
November 18, 2022 12:31 PM
|
After
May 25, 2023 02:02 AM
|
|
Field
Intervention (Public)
|
Before
RCT within the tech firm:
In the RCT, we exogenously vary text blocks at the top of job ads.
These text blocks are part of a so called ‘teaser text’. The ‘teaser text’ is located at the top of the job ads and provides a high-level and general description of the advertised job and the firm itself.
We use a within-job-ad-randomization to identify treatment effects. In the control group, we keep the ‘teaser text’ as it is. In the first treatment (‘flexibility treatment’) we add a text block highlighting the opportunity for flexibility in the advertised position. The second treatment (‘growth treatment’) adds an additional text block (to the control treatment job ad) highlighting the opportunity for future wage growth.
Survey experiment :
The survey experiment is conducted in cooperation with a market research agency (Bilendi) as well as several laboratories for economic research of Universities across Germany.
Bilendi is a market research company maintaining a large representative panel of the German population.
|
After
RCT within the tech firm:
In the RCT, we exogenously vary text blocks at the top of job ads.
These text blocks are part of a so called ‘teaser text’. The ‘teaser text’ is located at the top of the job ads and provides a high-level and general description of the advertised job and the firm itself.
We use a within-job-ad-randomization to identify treatment effects. In the control group, we keep the ‘teaser text’ as it is. In the first treatment (‘flexibility treatment’) we add a text block highlighting the opportunity for flexibility in the advertised position. The second treatment (‘growth treatment’) adds an additional text block (to the control treatment job ad) highlighting the opportunity for future wage growth.
Survey experiment :
The survey experiment is conducted in cooperation with several laboratories for economic research of Universities across Germany.
|
|
Field
Experimental Design (Public)
|
Before
RCT within the tech firm:
In the RCT, we exogenously vary text blocks at the top of job ads.
These text blocks are part of a so called ‘teaser text’. The ‘teaser text’ is located at the top of the job ads and provides a high-level and general description of the advertised job and the firm itself.
We use a within-job-ad-randomization to identify treatment effects. In the control group, we keep the ‘teaser text’ as it is. In the first treatment (‘flexibility treatment’) we add a text block highlighting the opportunity for flexibility in the advertised position. The second treatment (‘growth treatment’) adds an additional text block (to the control treatment job ad) highlighting the opportunity for future wage growth.
Survey experiment:
In the survey, we show participants (either recruited via Bilendi or via the laboratory pools) job ads which are part of the RCT described above. Thus, the job ads are ads for “real” jobs in a “real” company which invites applications for the position. The design of job ads shown to each respondent in the survey will vary (across randomization).
More specifically, we will randomly provide survey participants with one of three different job ads: the control treatment, the flexibility treatment, or the growth treatment.
In the survey we ask among others for demographic characteristics, educational background, preferences, ability beliefs and employment status. Moreover, we elicit participants’ expectations about the working conditions in the firm.
Specifically, we ask for the participants’ expectations regarding the working condition several times. The first time, we do not show them the location for which the job is advertised and ask them to assume that the job is located in a distance with a reasonable commuting time. Then we ask the expectation question again for in total two different cities in Germany (among which is also the true location for which the job is advertised). This allows us to check to which extent expectations are influenced via the location of the job.
Finally, we reveal the true location of the job and tell participants that the company invites applications for the position. Subsequently, we ask respondents whether they want to apply for the job and ask them to reason why or why they do not want to apply to the job. If they answer yes and they have an increased interest for the job, we offer them the opportunity to follow-up with an employee of the firm.
|
After
RCT within the tech firm:
In the RCT, we exogenously vary text blocks at the top of job ads.
These text blocks are part of a so called ‘teaser text’. The ‘teaser text’ is located at the top of the job ads and provides a high-level and general description of the advertised job and the firm itself.
We use a within-job-ad-randomization to identify treatment effects. In the control group, we keep the ‘teaser text’ as it is. In the first treatment (‘flexibility treatment’) we add a text block highlighting the opportunity for flexibility in the advertised position. The second treatment (‘growth treatment’) adds an additional text block (to the control treatment job ad) highlighting the opportunity for future wage growth.
Survey experiment:
In the survey, we show participants job ads which are part of the RCT described above. Thus, the job ads are ads for “real” jobs in a “real” company which invites applications for the position. The design of job ads shown to each respondent in the survey will vary (across randomization).
More specifically, we will randomly provide survey participants with one of three different job ads: the control treatment, the flexibility treatment, or the growth treatment.
In the survey we ask among others for demographic characteristics, educational background, preferences, ability beliefs and employment status. Moreover, we elicit participants’ expectations about the working conditions in the firm.
Specifically, we ask for the participants’ expectations regarding the working condition several times. The first time, we do not show them the location for which the job is advertised and ask them to assume that the job is located in a distance with a reasonable commuting time. Then we ask the expectation question again for in total two different cities in Germany (among which is also the true location for which the job is advertised). This allows us to check to which extent expectations are influenced via the location of the job.
Finally, we reveal the true location of the job and tell participants that the company invites applications for the position. Subsequently, we ask respondents whether they want to apply for the job and ask them to reason why or why they do not want to apply to the job. If they answer yes and they have an increased interest for the job, we offer them the opportunity to follow-up with an employee of the firm.
|
|
Field
Planned Number of Observations
|
Before
RCT within the tech firm:
We start the RCT at the end of October 2022. The company confirmed that we can run the RCT until the end of October 2023; we aim to extent the RCT at least until the end of April 2024.
We are not able to predict the exact number of observations in the RCT. Because of that, we summarize here the process how the data will be collected and our expected number of observations.
Each year, the tech company is posting on average between 80 and 120 of job ads for engineering, business administration, IT and science (e.g., physics, biology) positions in the plant that cooperates with us. All of these job ads will be part of the RCT.
For each job ad, the company receives approximately between 2 and 60 applications. In our RCT, one observation is one application. Standard errors will be clustered at the job ad level.
We receive detailed data from the company about each application (e.g. gender, employment history, educational background) and data from the hiring process (e.g. classification of the quality of the applicant from the firm’s recruiters, which applicant is invited to an interview).
Survey experiment:
We seek to recruit survey participants whose educational background matches with the requirements listed in the job ad. Bilendi takes care of recruiting participants based on whether they have a STEM background or not. In the laboratory pools, we select participants based on their field of study. All subjects whose field of study fits to the advertised position are invited to participate in the survey.
Yet, we do not know how many suitable candidates are available in the laboratory pools, as we do not know which jobs will be advertised by the company during next year (and thus not the requirements).
Bilendi will run the survey for a total of 20 job ads. For 40 additional job ads (if possible) we will run the survey in the lab (the exact number depends on the total number of vacancies and the size of the subject pools / response rates in the labs). As there is uncertainty with regard to the required educational background needed for the positions and the positions are partly highly specialised, we do not know yet, whether we will be able to find sufficiently many suitable potential survey participants for the lab.
Our aim is to collect for each job ad in total at least 45 observations (at least 15 survey participants for each treatment group). Note that it is possible that this number will vary because of varying response rates in the different labs and because of the uncertainty with respect to the fit of the subjects’ background (job experience, education) to the job ads.
|
After
RCT within the tech firm:
We start the RCT at the end of October 2022. The company confirmed that we can run the RCT until the end of October 2023; we aim to extent the RCT at least until the end of April 2024.
We are not able to predict the exact number of observations in the RCT. Because of that, we summarize here the process how the data will be collected and our expected number of observations.
Each year, the tech company is posting on average between 80 and 120 of job ads for engineering, business administration, IT and science (e.g., physics, biology) positions in the plant that cooperates with us. All of these job ads will be part of the RCT.
For each job ad, the company receives approximately between 2 and 60 applications. In our RCT, one observation is one application. Standard errors will be clustered at the job ad level.
We receive detailed data from the company about each application (e.g. gender, employment history, educational background) and data from the hiring process (e.g. classification of the quality of the applicant from the firm’s recruiters, which applicant is invited to an interview).
Survey experiment:
We seek to recruit survey participants whose educational background matches with the requirements listed in the job ad. In the laboratory pools, we select participants based on their field of study. All subjects whose field of study fits to the advertised position are invited to participate in the survey. Yet, we do not know how many suitable candidates are available in the laboratory pools, as we do not know which jobs will be advertised by the company during next year (and thus not the requirements).
For 15-30 job ads (if possible) we will run the survey in the lab (the exact number depends on the total number of vacancies and the size of the subject pools / response rates in the labs). As there is uncertainty with regard to the required educational background needed for the positions and the positions are partly highly specialized, we do not know yet, whether we will be able to find sufficiently many suitable potential survey participants for the lab. Our aim is to collect for each job ad in total at least 45 observations (at least 15 survey participants for each treatment group), but we aim at 90 observations for each job ad (30 survey participants for each treatment group). Note that it is possible that this number will vary because of varying response rates in the different labs and because of the uncertainty with respect to the fit of the subjects’ background (job experience, education) to the job ads.
We made some adjustments compared to our previous version of the preregistration (see the former version from November 18, 2022). Our initial plan was to run surveys for 20 job ads in cooperation with the market research agency Bilendi. However, as we started to pilot our surveys with them, we experienced a very low response quality. Simple quality checks (like reading the job ad for at least 20 seconds) led to a dropout of roughly 75% of the sample. Because of the low answer quality as well as potential selection bias resulting from it, we decided to stop working with Bilendi before analyzing the data; we conduct our whole survey via various econ labs across German speaking countries. Moreover, for statistical reasons, we decided to reduce the targeted number of job ads used in the survey for the sake of an increased number of participants in each survey wave. As the subject pool in the labs are mainly students with no or limited job experience, we are also planning to analyze in our RCT heterogeneous effects with respect to job experience.
|
|
Field
Secondary Outcomes (End Points)
|
Before
RCT within the tech firm:
fit of the application to the advertised position, quality of the applicant / application, commuting time to the current employer and our study firm’s plant, residence of the applicants, success of the applicants in the hiring process, interaction of the various outcome variables
Survey experiment:
interaction of the various outcome variables
|
After
RCT within the tech firm:
fit of the application to the advertised position, quality of the applicant / application, commuting time to the current employer and our study firm’s plant, residence of the applicants, success of the applicants in the hiring process, probability that at least one woman applies, interaction of the various outcome variables, heterogeneous treatment effects with respect to the required work experience for the job
Survey experiment:
interaction of the various outcome variables
|
|
Field
Secondary Outcomes (Explanation)
|
Before
RCT within the tech firm:
Our third outcome variable is the fit of the application to the advertised position (in particular, does the educational background and previous job experience fit to the advertised position).
Our fourth outcome variable is the quality of the applicant / application. Here, we use various measures. First, the classification of the quality of the application by the firm’s recruiters. Second, whether all required documents are submitted, whether there are typos in the application, etc.. Third, the grades of the applicant at university, and whether the university is one of the top universities in the respective field in Germany. Fourth, the quality of job market experience of the applicant (e.g. prestige of the current employer, prestige of the internships). To measure the quality of the application, we will potentially also run expert interviews among HR managers in which HR manager are asked to rate the quality of the applications.
Our fifth type of outcome variables are the commuting time to the current employer and our study firm’s plant as well as the cities of residence of the applicants.
Our sixth type of outcome variables are measures regarding the success of the applicants in the hiring process. First, we want to check how far applicants get in the hiring process (e.g. invited for an interview). Second, we want to analyse which candidates eventually get the job.
Note that we are also planning to interact the different outcome variables in order to explore heterogenous treatment effects, e.g. to study the interaction of gender and commuting time.
Survey experiment:
We are also interested on interaction of the various outcome variables, e.g. whether peoples’ expectation about the job and their interests to apply interact. With respect to heterogeneous treatment effects, we will also study how people’s interest to apply depends on gender, their preferences for certain job characteristics, ability beliefs, their risk preferences, competitiveness preferences, patience and trust/reciprocity.
|
After
RCT within the tech firm:
Our third outcome variable is the fit of the application to the advertised position (in particular, does the educational background and previous job experience fit to the advertised position).
Our fourth outcome variable is the quality of the applicant / application. Here, we use various measures. First, the classification of the quality of the application by the firm’s recruiters. Second, whether all required documents are submitted, whether there are typos in the application, etc.. Third, the grades of the applicant at university, and whether the university is one of the top universities in the respective field in Germany. Fourth, the quality of job market experience of the applicant (e.g. prestige of the current employer, prestige of the internships). To measure the quality of the application, we will potentially also run expert interviews among HR managers in which HR manager are asked to rate the quality of the applications.
Our fifth type of outcome variables are the commuting time to the current employer and our study firm’s plant as well as the cities of residence of the applicants.
Our sixth type of outcome variables are measures regarding the success of the applicants in the hiring process. First, we want to check how far applicants get in the hiring process (e.g. invited for an interview). Second, we want to analyse which candidates eventually get the job.
Our seventh type of outcome variable is a binary variable being equal to one if at least one women applies for the job.
Note that we are also planning to interact the different outcome variables in order to explore heterogenous treatment effects, e.g. to study the interaction of gender and commuting time, with respect to the required work experience for the job (e.g. entry- or senior-level positions).
Survey experiment:
We are also interested on interaction of the various outcome variables, e.g. whether peoples’ expectation about the job and their interests to apply interact. With respect to heterogeneous treatment effects, we will also study how people’s interest to apply depends on gender, their preferences for certain job characteristics, ability beliefs, their risk preferences, competitiveness preferences, patience and trust/reciprocity.
|