x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Labor Market Returns to Upskilling - A Combination of Audit Study and Resume Review
Last registered on April 21, 2020

Pre-Trial

Trial Information
General Information
Title
Labor Market Returns to Upskilling - A Combination of Audit Study and Resume Review
RCT ID
AEARCTR-0005718
Initial registration date
April 20, 2020
Last updated
April 21, 2020 11:27 AM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
Yale University
Other Primary Investigator(s)
PI Affiliation
Yale University
Additional Trial Information
Status
In development
Start date
2021-06-01
End date
2021-09-01
Secondary IDs
Abstract
The US labor market has experienced significant shifts over the last few decades due to skill-biased technological changes (Autor et al., 2010). This polarization of employment opportunity has exacerbated income inequality across the country, marginalizing a large fraction of the workforce, who do not have the background and resources to pursue high-skilled careers through higher education. One potential solution is to retrain displaced workers and disadvantaged population into technological industries with considerable growth potential. However, there is a lack of evidence on what type of retraining programs are effective in helping disadvantaged groups gain foothold in high-skilled industries. This research project aims to estimate the effectiveness of entry-level online tech certificates in helping individuals move from other sectors into the tech industry. In addition, this project asks whether online tech certificates are equally valuable, more valuable or less valuable for workers without relevant tech-experience and education. Finally, this project also asks whether age interacts with the returns to tech certificates.

Credibly estimating the labor market returns to online certificates poses significant challenges. Many retraining programs (eg. Year Up) have strict admission guidelines for participants, and often work closely with corporate partners to place participants at the end of the program. Therefore, simply comparing income of individuals with and without certificates would lead to biased estimates, because workers who choose to attain certificates are inherently different from those who do not. To address these challenges, this project uses a combination of audit study and Incentivized Resume Rating (IRR) methods. The core of the research design is an audit study that elicits employers’ true preferences for IT certificates. In this step, we first carefully design artificial job applications and randomly assign part of the application profiles to have selected online tech certificates. Then, the resumes are sent out to job openings posted on a large online job search platform in the US, and call-back responses are analyzed. A subset of these artificial applications are further subjected to incentivized resume rating, where third-party resume evaluators decide whether they would call-back these applicants and what salary they would be offered. By analyzing the results on call-back rates from the IRR and audit studies, we first establish a correspondence between the weights that resume evaluators and real employers assign to various observable worker characteristics. Then, this correspondence is applied to resume reviewers' behavior in the IRR salary study to find the expected labor market returns to skill certificates in a hypothetical audit study on salary.

Results from this project would inform policy-makers on the viability of using certifications and related training programs in assisting displaced workers move into the tech industry. Given the increasing rate of technological advancement and changing nature of jobs, these results have potential to guide policies that can reduce income inequality and at the same time create a pool of high-skilled workforce. This project would generate rich datasets on worker profiles typical of young and entry-level job applicants in the tech industry, call-back rates from the audit and IRR studies, and expected offers from the IRR salary study. Summary statistics from this dataset will be of independent interest to organization that are invested in designing and administering online skill certification programs. Methodologically, this project provides the first direct examination of whether third-party resume reviews are a viable alternative to audit studies, and proposes a new method to utilize the relative advantages of both audit study design and resume review studies.
External Link(s)
Registration Citation
Citation
Sinha, Sourav and Zhengren Zhu. 2020. "Labor Market Returns to Upskilling - A Combination of Audit Study and Resume Review." AEA RCT Registry. April 21. https://doi.org/10.1257/rct.5718-1.0.
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2021-06-01
Intervention End Date
2021-09-01
Primary Outcomes
Primary Outcomes (end points)
Call back response for audit study.
Reviewer decision on call back, job offer, and expected wage.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
This project uses a combination of audit study and Incentivized Resume Rating (IRR) methods. The core of the research design is an audit study that elicits employers’ true preferences for IT certificates. In this step, we first carefully design artificial job applications and randomly assign part of the application profiles to have selected online tech certificates. In addition, we randomly assign the applicant to either have or not have previous tech-related experience, and also randomly assign the applicant to either be young (~22 yo) or older (~35 yo). Then, the resumes are sent out to job openings posted on a large online job search platform in the US, and call-back responses are analyzed. A subset of these artificial applications are further subjected to incentivized resume rating, where third-party resume evaluators decide whether they would call-back these applicants and what salary they would be offered. By analyzing the results on call-back rates from the IRR and audit studies, we first establish a correspondence between the weights that resume evaluators and real employers assign to various observable worker characteristics. Then, this correspondence is applied to resume reviewers' behavior in the IRR salary study to find the expected labor market returns to skill certificates in a hypothetical audit study on salary.
Experimental Design Details
Not available
Randomization Method
Randomization is implemented with random resume generator (Lahey and Beasley, 2007)
Randomization Unit
Individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
12000 individual applications
Sample size: planned number of observations
12000 individual applications
Sample size (or number of clusters) by treatment arms
6 types of resumes: no tech experience + no certificate + young, no tech experience + with certificate + young, with tech experience + no certificate + young, with tech experience + with certificate + young, no tech experience + no certificate + older, no tech experience + with certificate + older. 2000 individual applications in each arm. Total of 12000 individual applications.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Power: 0.81; 6 treatment arms lead to 15 pairwise comparison; group A proportion: 0.16; group B proportion: 0.13.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Yale University IRB
IRB Approval Date
2020-03-20
IRB Approval Number
2000027608