Understanding the Implications of Job Location Arrangements on the Job Application Process

Last registered on May 03, 2023

Pre-Trial

Trial Information

General Information

Title
Understanding the Implications of Job Location Arrangements on the Job Application Process
RCT ID
AEARCTR-0011354
Initial registration date
May 03, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 03, 2023, 4:40 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Monash University

Other Primary Investigator(s)

PI Affiliation
Monash University
PI Affiliation
Claremont McKenna College

Additional Trial Information

Status
In development
Start date
2023-07-01
End date
2024-07-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In this project, we study whether candidates for a real job behave differently when they are informed that the position will be remote.
External Link(s)

Registration Citation

Citation
Avery, Mallory, Jeffrey Flory and Andreas Leibbrandt. 2023. "Understanding the Implications of Job Location Arrangements on the Job Application Process." AEA RCT Registry. May 03. https://doi.org/10.1257/rct.11354-1.0
Experimental Details

Interventions

Intervention(s)
In this project, we study whether candidates for a real job behave differently when they are informed that the position will be remote.
Intervention (Hidden)
To understand the potential impacts of in-office vs. work from home and hybrid work arrangements on application behavior by different groups, we design a novel experiment. In the experiment, we collect applicant data by posting a job advertisement for a general set of jobs. All candidates must answer a series of interview style questions and provide their CV. We then provide them with additional information about a specific position. In this, we will randomly tell them that the job is remote either before or after they have completed the application. In other words, we have two treatments:
i) In-Office:
Applicants will not be informed of the location of the work when applying.

ii) Work from Home
Applicants will be informed that they will be invited to apply for a remote or work from home position.

All applicants will then be invited to complete an application for that position.
Intervention Start Date
2023-07-01
Intervention End Date
2024-07-01

Primary Outcomes

Primary Outcomes (end points)
We collect the following primary outcomes:
- Proportion who starts the application: This is defined as the number of candidates who start the application/ The number who are invited to apply. A candidate is considered to have been invited to apply if they are sent an email with details on the specific position and a link to the application.
-Proportion who complete: This is defined as the number who complete the application/ The number who are invited to apply. A candidate is considered to have been invited to apply if they are sent an email with details on the specific position and a link to the application. A candidate is said to have completed if they answer all questions and clicked submit.
Primary Outcomes (explanation)
Below we outline how we will use our primary outcomes and the key hypothesis.
To understand whether work location arrangements the behaviour of candidates we study the proportion that complete and the proportion who start the assessment when informed the work location is in-office vs. work from home. We generate the following hypothesis:
Hypothesis 1: Application/Completion rates are higher in the treatment where candidates are informed they will be working from home compared to not being informed.
Hypothesis 2: Females/ male ratio in application rates are higher in the work from home treatment compared to the in-office treatment.

Secondary Outcomes

Secondary Outcomes (end points)
To understand possible mechanisms we elicit the following secondary outcome variables:
Time to complete the questions: the candidates completion time is recorded.
Diversity Analysis:
We will also use a number of variables to understand the influence of work location arrangements on candidate diversity. Our main focus will be on gender but as a secondary analysis conditional on obtaining sufficient observations we also study ethnicity. To focus on diversity, we will interact the treatment variables by the gender and possibly ethnicity (separately) of the candidate.
Heterogeneity Analysis:
We will also consider the impact of work location arrangements on the quality of candidates. As such, we will measure how observable characteristics such as education and work experience of completed applications varies with the treatment and gender/ethnicity of the applicants.
We also conduct a number of robustness tests
-As a robustness to completion and application rates, we also track the proportion of people who open the email.

Secondary Outcomes (explanation)
See above

Experimental Design

Experimental Design
Our design aims to measure the impact of work location arrangements (i.e. in-office vs. working from home) on behavior of job candidates.
Experimental Design Details
The design consists of two stages.
In stage 1, we will post a job ad for a real position across the United States at several job portals (e.g., joinhandshake.com, indeed.com). To indicate interest, job applicants must send their CV and fill out a short survey (e.g. years of experience education, demographics). Applicants must reside in the United States.
In stage 2, after the interest period closes, we will invite all applicants to apply to a job. This will be a standard job application. Prior to taking part in the assessment, candidates are randomly assigned into one of our treatments (described above).
Randomization Method
Randomization will be carried out by a computer.
Randomization Unit
The randomization unit will be the individual for all treatments.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
The clusters will be equal to the number of legitimate candidates. Based on previous experience we expect to have 1200 legitimate job applicants, but this will likely vary depending on a number of factors. A legitimate candidate is someone who resides in the United States and completes the initial application form. We plan to assign 50% of the sample to the in-office treatment and 50% to the work from home treatment. Further, we will stratify by gender such that there is the same proportion of men/women in each treatment.
Sample size: planned number of observations
The number of observations is the same as the number of clusters. We hope to have up to 1200 observations.
Sample size (or number of clusters) by treatment arms
See above. We plan to assign 50% of the sample to the in-office treatment and 50% to the work from home treatment. Further, we will stratify by gender such that there is the same proportion of men/women in each treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The MDE for the following outcomes are: Proportion who complete and start: Using a significance of 0.05, power of 0.8 and a value of 0.6 as the proportion who complete/start for in-person positions we can detect a minimum effect size of 0.079.
IRB

Institutional Review Boards (IRBs)

IRB Name
Claremont McKenna College
IRB Approval Date
2023-04-25
IRB Approval Number
2023-03-024

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials