Back to History Current Version

Whether to Apply: Fall 2021 Follow-up

Last registered on February 15, 2022

Pre-Trial

Trial Information

General Information

Title
Whether to Apply: Fall 2021 Follow-up
RCT ID
AEARCTR-0008223
Initial registration date
November 24, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 28, 2021, 6:32 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
February 15, 2022, 10:49 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Harvard Business School

Other Primary Investigator(s)

PI Affiliation
University of Toronto

Additional Trial Information

Status
In development
Start date
2021-11-29
End date
2022-02-28
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In this project, we study the decisions of candidates about whether to apply for different opportunities. We aim to tackle the question of whether there are gender differences in how job-seekers perceive their own qualifications for different opportunities, and how this impacts their decision about whether or not to apply. In addition, we ask whether whether reducing ambiguity around required qualifications for a given opportunity reduces the gender gap in willingness to apply among qualified candidates.

The first experiment we ran for this project was a field experiment on the online labor market platform, UpWork. Serving as a potential employer, we created job opportunities to which participants could apply. In our baseline condition, we found that qualified women were significantly less likely to apply to our more demanding and more lucrative job opportunity than equally qualified men. Our treatment conditions tested a simple policy intervention: we provided more clarity on what “the bar” was in terms of desired qualifications. We found that the gender gap in in application rates among qualified candidates was reduced when the desired qualifications for the opportunity were less ambiguous.

The study we are pre-registering is a follow-up to that study. In particular, we are looking to replicate the results from the UpWork experiment, using a very similar paradigm, but in a new population (Prolific Workers). In addition to replicating our main results, we are adding additional questions aimed at uncovering the mechanisms underlying our results.
External Link(s)

Registration Citation

Citation
Coffman, Katherine and Manuela Collis. 2022. "Whether to Apply: Fall 2021 Follow-up." AEA RCT Registry. February 15. https://doi.org/10.1257/rct.8223-2.2
Experimental Details

Interventions

Intervention(s)
We will have participants build a "resume" for a job opportunity by completing a screening task. Then, we will present them with the opportunity to apply to either an expert-level job or a more basic job.
Across subject, we vary the language used to describe the required qualifications for the expert-level job. In particular, the control treatment contains a large degree of ambiguity about what "the bar" is for being good enough to be qualified for the expert-level job. Our two treatments reduce this ambiguity by providing clearer, more specific, objective information on what the required qualification is in terms of the screening task.
Intervention Start Date
2021-11-29
Intervention End Date
2022-01-31

Primary Outcomes

Primary Outcomes (end points)
Decision to apply to the expert-level job
Primary Outcomes (explanation)
Our main outcome variable will be the decision to apply to the expert-level job, which takes either 1 (applied to expert-level job) or 0 (did not apply to expert-level job). We will exclude from analysis any participant who did not answer the attention check question correctly. Furthermore, we will exclude any worker who chose to apply to neither job (as indicated in their application decision), and indicated this was because of a reason other than perceived qualifications (as indicated in the follow-up question that asks why they did not apply). Workers who did not apply to either job and indicated in the follow-up question that they did not apply specifically because they did not feel qualified for either job will be included in the analysis and coded as not having applied to the expert-level job.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will have participants build a "resume" for a job opportunity by completing a screening task. Then, we will present them with the opportunity to apply to either an expert-level job or a more basic job.
Across subject, we vary the language used to describe the required qualifications for the expert-level job. In particular, the control treatment contains a large degree of ambiguity about what "the bar" is for being good enough to be qualified for the expert-level job. Our two treatments reduce this ambiguity by providing clearer, more specific, objective information on what the required qualification is in terms of the screening task.
Experimental Design Details
Randomization Method
Individuals will be randomized into one of three treatments (Control, Positive, Normative) by the experimental program (Qualtrics), at equal rates.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
2400 individuals
Sample size: planned number of observations
2400 individuals
Sample size (or number of clusters) by treatment arms
2400 individuals, 800 in each treatment, split evenly across men and women
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

Documents

Document Name
Explanation of Purpose of Experiment
Document Type
other
Document Description
File
Explanation of Purpose of Experiment

MD5: 7115b0a8bb4dde1bcf4e8d2f90f9970c

SHA1: d2f5954f9bb8ed13543c0c5fba41dc30f4db6939

Uploaded At: November 24, 2021

Document Name
Whether to Apply _ September 2021
Document Type
other
Document Description
Draft of Whether to Apply working paper as of September 2021
File
Whether to Apply _ September 2021

MD5: 6b70e4559affe52d3fceddd5263dbb64

SHA1: 866a35572816e23fdb3f87fc2ea97944473a7df7

Uploaded At: September 10, 2021

IRB

Institutional Review Boards (IRBs)

IRB Name
Harvard Business School IRB
IRB Approval Date
2021-09-29
IRB Approval Number
IRB21-1282
IRB Name
University of Toronto Rotman
IRB Approval Date
2021-11-19
IRB Approval Number
00041720
Analysis Plan

Analysis Plan Documents

Analysis Plan for Fall 2021 Followup

MD5: 11a5ecd2c03513303e4901d6d32bd066

SHA1: d0bae2fb16f3253952f1bbfffcc9e16d34e2afd9

Uploaded At: December 03, 2021

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
December 15, 2021, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
December 15, 2021, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
2,243 Prolific participants
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
2400 Prolific participants
Final Sample Size (or Number of Clusters) by Treatment Arms
Our final sample size was 2,243, with no fewer than 142 participants in any given cell (3 treatment versions × gender × qualified)
Data Publication

Data Publication

Is public data available?
Yes

Program Files

Program Files
Yes
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials

Description
journal article
Citation
Coffman, Katherine B., Manuela R. Collis, and Leena Kulkarni. "Whether to Apply." Management Science (2023).