Do Firms Like STARs? Measuring Demand for Job Candidates Skilled Through Alternative Routes

Last registered on December 06, 2023


Trial Information

General Information

Do Firms Like STARs? Measuring Demand for Job Candidates Skilled Through Alternative Routes
Initial registration date
November 27, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 06, 2023, 7:59 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.



Primary Investigator

University of Toronto

Other Primary Investigator(s)

Additional Trial Information

In development
Start date
End date
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.

The research will investigate whether firms are actively engaging in skill-based hiring and their likelihood of employing individuals who have acquired skills through alternative routes (STARs).
External Link(s)

Registration Citation

Oseguera, Mariana . 2023. "Do Firms Like STARs? Measuring Demand for Job Candidates Skilled Through Alternative Routes." AEA RCT Registry. December 06.
Experimental Details


Intervention Start Date
Intervention End Date

Primary Outcomes

Primary Outcomes (end points)
Whether the applicant receives a call or email from the potential employer.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Will be made public when the trial is complete.
Experimental Design Details
This study explores the evolving landscape of skill-based hiring in the contemporary job market, addressing the disconnect between employers' perceptions of college degrees and the actual skills required for various roles. Recent discussions have highlighted a growing emphasis on skill-based hiring, focusing on the direct alignment of job applicants' skills with job requirements. This shift raises critical questions about the effectiveness and biases inherent in current hiring practices.

The research will investigate whether firms are actively engaging in skill-based hiring and their likelihood of employing individuals who have acquired skills through alternative routes (STARs), such as on-the-job experience, bootcamps, or micro-credentials, rather than traditional college education. The design is an audit study focusing on software engineering and digital marketing occupations. The study will be structured around several questions:

(1) It will examine whether the callback rate for job applicants varies based on the holding of a college degree, despite similar on-the-job experience, and will consider the field of experience/major and its relevance to the job.
(2) The study will evaluate the efficacy of STARs in signaling their skills through specific means, such as bootcamps, micro-credentials, and showcasing work on platforms like GitHub.
(3) It will explore whether firms are more inclined to hire STARs if they have previously been employed by high-status firms.
(4) The study will analyze the impact of "Tear the Paper Ceiling" initiatives on hiring practices, particularly focusing on whether adherence to these initiatives by firms or states influences the hiring of STARs.
(5) It will also consider how these effects vary with respect to applicants' race and gender, offering insights into potential biases in skill-based hiring.

Fictitious applicants will be of similar age. Those with college degree will have either around 2 years of experience or around 1 year + internship. Those without college degrees will have either 2 years of very relevant experience + 3 years of other work experience or 4 relevant experience +2 other working experience.

The research team will send job applications to open job positions that require 2-4 years of experience in the relevant field. The applications will be similar in all but the randomly assigned treatment dimensions:
(1) college degree or not
(1a) conditional on having college the relevance of the major
(1b) conditional on not having college listing a micro-credential, BootCamp, GitHub link
(2) Years of relevant and overall experience
(4) Reputation/status of last employer
(5) Race: Black and White
(6) Gender

Characteristics will be independently randomly assigned to a job applicant with equal (50%) probability. Race is indicated via the name of the applicant.

Standard templates for resumes contain the following information: name, contact information, education, work history, skills, and additional information such as credentials, relevant links, or projects. The constructed resumes follow the same structure while varying the treatments of the study.

Each resume lists a phone number and an email address in which callbacks will be received. An email address was set up for each name in each Combined Statistical Area (CSA). Two phone numbers were set for each CSA. A standard voicemail was set up in each of the phone lines prompting callers to leave a message for the applicant. Some job applications require to state an address. For those cases, four fictitious addresses in large apartment complexes in middle-income areas in each CSA were generated. These addresses were randomly assigned to resumes.

High schools and public colleges are selected using the Common Core Data. In particular, for high schools, 8 non-charter or magnet schools per CSA are selected for which the confidence interval of socioeconomic status contains the 75 percentile of socioeconomic status within the CSA.

A small team of research assistants (4) will help apply for jobs and record applications. Each research assistant will be assigned one CSA and instructed to search for daily eligible jobs using the ONET classifications as keyword searches. Research assistants will not apply to jobs from staffing companies, that are paid via commission, or that have been posted more than 30 days prior. Emails will be set up per name per CSA so that each research assistant can oversee their own applications.

Data on the job application will be collected: job text, degree requirement statements, date posted, application date, sent resume, employer characteristics, and whether the employer contacted the applicant.
Randomization Method
Randomization is performed via computer using the Lahey and Beasley resume randomizer. This tool helps randomize across different characteristics and construct multiple profiles. Each job opening will only receive a maximum of two applications: one from an applicant with a college degree and one from another without one.
Randomization Unit
Individual applicant level. Note that some individual applicants' applications will be sent to multiple businesses, as described above, so the number of unique applicants is lower than the number of applications.
Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters
Approx. 6,000 job applications for software engineering jobs
Approx. 6,000 job applications for marketing jobs
Sample size: planned number of observations
Approx. 12,000 job applications
Sample size (or number of clusters) by treatment arms
For each occupation:
Approx. 3,000 resumes without a college degree
Approx. 3,000 resumes with a college degree
All other characteristics will be randomized 50/50 percent across groups
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The final power calculations depend on the exact feasible sample size. Callback rates in previous audit studies are approximately 10-15 percent. However, most of these studies focus on entry-level jobs or take place in looser labor markets. To be conservative, I use a callback rate of 7 percent. Additionally, among health sector jobs, Deming et al, (2016) finds that the difference in callback rates between college degree holders and high school graduates is around 3 percent and the difference between for-profit college graduates and high school graduates is around 2 percent. Considering these differences, a sample of 6,000 would allow me to detect an effect of 2 percent for the overall treatment and key subgroups.

Institutional Review Boards (IRBs)

IRB Name
University of Toronto REB
IRB Approval Date
IRB Approval Number


Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information


Is the intervention completed?
Data Collection Complete
Data Publication

Data Publication

Is public data available?

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials