Master's for hire: Experimental evidence on employers' perceptions of master's degrees from for-profit institutions

Last registered on December 17, 2020

Pre-Trial

Trial Information

General Information

Title
Master's for hire: Experimental evidence on employers' perceptions of master's degrees from for-profit institutions
RCT ID
AEARCTR-0005725
Initial registration date
April 15, 2020
Last updated
December 17, 2020, 9:54 AM EST

Locations

Primary Investigator

Affiliation
Vanderbilt University

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2020-04-17
End date
2021-12-31
Secondary IDs
Abstract
Enrollment in graduate degree programs has expanded in recent decades, facilitated in part through for-profit institutions and other primarily online programs. Despite the prominence of such comparatively broad-access graduate degree programs, there is little evidence on the impacts of such graduate credentials on labor market opportunities. To help address this gap in the literature, this correspondence study compares employer callback rates for occupations in business and health care, two of the most common fields for master’s degrees. For each job opening, I submit fictitious résumés that vary systematically only by master’s degree attributes: each résumé will list a master’s from a for-profit institution, a master’s from another primarily online institution, a master’s from a primarily on-campus regional institution, or no master’s degree. The results will provide some of the first experimental evidence of employers’ preferences between job candidates who attended various types of broad-access master’s programs, and will also facilitate comparisons between candidates with and without master’s degrees. Additionally, subgroup analyses will explore how such employer preferences may vary by the applicant’s sex and race. Such findings will be relevant to prospective students, universities, and policymakers in their understanding of the potential economic returns to master’s degrees.
External Link(s)

Registration Citation

Citation
Bennett, Chris. 2020. "Master's for hire: Experimental evidence on employers' perceptions of master's degrees from for-profit institutions." AEA RCT Registry. December 17. https://doi.org/10.1257/rct.5725-1.1
Experimental Details

Interventions

Intervention(s)
One of the most conspicuous areas of growth in graduate education has been in the for-profit sector, where graduate enrollment rose more than ten-fold between 1996 and 2016 (U.S. Department of Education, 2018a). Nearly 90 percent of graduate students at for-profits are enrolled in master’s programs, almost all of which are offered primarily online (U.S. Department of Education, 2018b). As a result of their rapid growth and strong emphasis on master’s-level enrollment, for-profit institutions account for almost one-tenth of master’s students nationwide, including one-fourth of black master’s students (Baum & Steele, 2017). Despite the notable market share of for-profit institutions, there is scant evidence about the job search experiences of master’s degree recipients from these institutions. The extent to which individuals benefit from earning a master’s degree at various types of institutions is currently an unresolved question that is highly relevant for hundreds of thousands of master’s students annually. Especially given that for-profit institutions account for a disproportionate share of black students, students from low-income backgrounds, and older students (Baum & Steele, 2017), understanding the labor market value of master’s degrees by institutional attributes has significant equity implications for students who have been historically underrepresented in graduate education.

To address this gap in our understanding of the returns to master’s degrees, this correspondence experiment involves applying to job openings on behalf of fictitious job candidates who differ systematically only in terms of their graduate education background (or lack thereof). This work builds on earlier correspondence experiments involving for-profit credentials at the undergraduate level (Darolia et al., 2015; Deming et al., 2016; Deterding & Pedulla, 2016). I am applying to job openings in business and health, two of the most common fields for master’s degrees, for occupations that employ substantial numbers of master's recipients. The four treatment arms include listing a master’s from a for-profit institution, a master’s from another primarily online institution, a master’s from a regional institution that primarily awards master’s degrees through on-campus programs, or no master’s degree (i.e., bachelor’s only). These options represent comparatively broad-access graduate school options that are within the choice set for a significant number of prospective master’s students, including the choice to not pursue a graduate degree. To explore possible variation in benefits (or disadvantages) that accrue to master’s-degree holders, this study will block by sex and race for each job posting (i.e., all fictitious job candidates for an opening will have names suggesting they are the same sex and race as one another).


Baum, S., & Steele, P. (2017). Who goes to graduate school and who succeeds? Urban Institute. Retrieved from https://www.urban.org/research/publication/who-goes-graduate-school-and-who-succeeds.
Darolia, R., Koedel, C., Martorell, P., Wilson, K., & Perez‐Arce, F. (2015). Do employers prefer workers who attend for‐profit colleges? Evidence from a field experiment. Journal of Policy Analysis and Management, 34(4), 881-903.
Deming, D. J., Yuchtman, N., Abulafi, A., Goldin, C., & Katz, L. F. (2016). The value of postsecondary credentials in the labor market: An experimental study. American Economic Review, 106(3), 778-806.
Deterding, N. M., & Pedulla, D. S. (2016). Educational authority in the ‘‘open door’’ marketplace: Labor market consequences of for-profit, nonprofit, and fictional educational credentials. Sociology of Education, 89(3), 155-170.
U.S. Department of Education, National Center for Education Statistics. (2018a). Table 303.80. Total postbaccalaureate fall enrollment in degree-granting postsecondary institutions, by attendance status, sex of student, and control of institution: 1970 through 2027. In U.S. Department of Education, National Center for Education Statistics (Ed.), Digest of Education Statistics (2017 ed.). Retrieved from https://nces.ed.gov/programs/digest/d17/tables/dt17_303.80.asp.
U.S. Department of Education, National Center for Education Statistics. (2018b). Table 318.50. Degrees conferred by postsecondary institutions, by control of institution, level of degree, and field of study: 2015-16. In U.S. Department of Education, National Center for Education Statistics (Ed.), Digest of Education Statistics (2017 ed.). Retrieved from https://nces.ed.gov/programs/digest/d17/tables/dt17_318.50.asp.
Intervention Start Date
2020-04-17
Intervention End Date
2020-11-28

Primary Outcomes

Primary Outcomes (end points)
In this study, the primary outcome of interest is whether a job applicant receives a positive response from an employer via email or telephone.
Primary Outcomes (explanation)
A positive response from an employer could include an explicit invitation for an interview or some other non-perfunctory indication of employer interest (e.g., request to complete a skills assessment, question about salary requirements, request for names of references).

Secondary Outcomes

Secondary Outcomes (end points)
The secondary outcome of interest is whether a job applicant receives an explicit invitation for an interview. (This is a sub-component of the primary outcome, which also includes various other expressions of employer interest as positive responses.)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
1) Identifying job listings
For this study, I am searching two national websites for job openings that correspond to selected Standard Occupational Classification (SOC) codes in the fields of business or health. Eligible job openings include those that match the SOC codes of interest (based on title and duties), require a bachelor's degree or higher, and list an employer for which an application has not already been submitted in the metropolitan area. For each opening, I store the text of the job listing and record attributes such as the employer name (to prevent repeated applications), degree requirements, and salary range (if listed).

2) Developing fictitious, randomized resumes
I am using the Resume Randomizer from Lahey & Beasley (2009) to develop resumes. Having varied the home addresses, educational backgrounds, and work histories based on the metropolitan area of the job opening, I run the Resume Randomizer to generate 4 resumes for each job opening. The tool assigns each resume within a group of 4 to have different treatment arms, but blocks by race and sex (i.e., all applicant names for a given job opening suggest the applicants have the same race and sex).

3) Submitting applications
Using the resumes generated from the Resume Randomizer, I separately submit each application through the process specified on the website. In some cases, this involves submitting a resume directly from the job search website, while in others it involves submitting the application through the employer's website or a third-party website (e.g., HR software brands such as Taleo and Brassring). I record the date on which I submitted each application.

4) Recording positive employer responses
I check all the email accounts and phone numbers each weekday. If an employer reaches out to the candidate with a positive response, I record their response. I also reply within 1 business day to inform them that the applicant is no longer available for the position, enabling the employer to move on to a different (non-fictitious) job candidate.
Experimental Design Details
Randomization Method
Resume Randomizer from Lahey & Beasley (2009). See https://data.nber.org/resume-audit/README.txt.
Randomization Unit
Randomization occurs at the level of the job opening.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Up to 2,370 job openings
Sample size: planned number of observations
Up to 9,480 applications
Sample size (or number of clusters) by treatment arms
Up to 2,370 applications for each of 4 treatment arms
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Power calculations suggest that a sample size of up to 9,480 may be required to detect an effect of 10 percent of the sample mean with 80% power and alpha = .05.
IRB

Institutional Review Boards (IRBs)

IRB Name
Vanderbilt Human Research Protection Program
IRB Approval Date
2020-03-10
IRB Approval Number
192456
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials