Experimental Design Details
This study investigates the labor market effects of Ban the Box legislation, which prohibits employers from asking about criminal records on job applications. The design is a correspondence audit study focusing on entry-level employers in New Jersey, which adopted a Ban-the-Box law effective March 1, 2015. Fictitious online job applications will be sent to these entry-level employers both before the law goes into effect (the pre-period) and after the law goes into effect (the post-period).
We are investigating several questions:
(1) What is the effect of stating (on an online job application) that one has a criminal record on the probability that employers will give an applicant a positive response? This question will be principally tested using our pre-period data. (After Ban-the-Box goes into effect, most employers will presumably remove the criminal records question. Some employers do not have the question even in the pre-period.)
(2) Does this effect vary depending on the applicant's other characteristics? The primary interaction effect of interest is race x record. We will also test interactions with characteristics that may signify incarceration: GED (vs high school diploma) and employment gaps, and we will test whether race interacts with these other characteristics.
(3) What is the main effect of other applicant characteristics on positive response probability? Again, we will assess race (the effect of primary interest), education type, and employment gap effects, using both pre- and post-period data.
(4) Do employers in fact comply with Ban-the-Box by removing the criminal records question? (This is a purely observational question--it is not subject to experimental manipulation.)
(5) After Ban-the-Box goes into effect, do employers (and particularly those employers that asked about criminal records in the pre-period) become more likely to discriminate based on characteristics that (in the real world) are correlated with criminal records, in particular race, GED (vs. high school diploma), and employment gap?
(6) Before Ban-the-Box, is there cross-sectional variation in the effect of these characteristics (race, GED, work gap) between businesses that ask about criminal records on the application and those that do not?
Questions (5) and (6) speak to the same basic question: whether limited access to criminal record information makes employers more likely to statistically discriminate against groups that they perceive as likely to have records. In order to test that question, we will look both at the two-way interactions described in (5) and (6), and assuming we are able to build a sample that provides sufficient statistical power, at three-way interactions that exploit both cross-sectional and temporal variation (e.g., race*period*box).
Our focus is New Jersey, but other states may be added if they adopt Ban-the-Box while we are working on the study. We will send (fictitious) online job applications to businesses who have posted availability for jobs that are suitable for candidates with limited work experience, no post-secondary education, and no specialized skills. If there are open jobs in both the pre- and post-period, each business will receive four applications total, one pair in each period. The applications will be similar on all but our randomly assigned treatment dimensions. Those dimensions are:
(1) Has felony criminal conviction or not
(1a) Conditional on being convicted of a crime: drug conviction or theft conviction
(2) Race: Black or White
(3) Has 1-year employment gap or not
(4) GED or High School Diploma
Characteristics will be independently randomly assigned to a job applicant with equal (50%) probability. Race is indicated via the name of the applicant – with distinctly black and distinctly white names chosen based on a study of birth certificate records from New Jersey.
All other job applicant characteristics are assigned to be as similar as possible, but randomly varied slightly so as to disguise the similarity of the applications. These include:
(1) Work history: All job applicants have approximately 3.5 years of work experience - approximately 2 years as a crew member at a fast food chain or convenience store and approximately 1.5 years in a manual labor job such as home improvement, landscaping, or moving. All applicants are unemployed at the time of the job application, having ended their most recent job 2-3 months before the application is sent. Descriptions of job duties and reasons for leaving jobs were slightly varied.
(2) Address: Because it is likely that employers will be concerned about employees being able to get to work, we want applicants to live near the jobs to which they apply. To achieve this, we chose 40 cities/towns in NJ that are nearest to the most open jobs on snagajob.com using an optimization tool included in ArcGIS software. These 40 town/cities serve as “centers” where our applicants will have address and local phone numbers. Within these centers, 4 addresses are chosen from neighborhoods that are at least 10% black, at least 20% white, and do not have a median income above $100,000. Addresses are then slightly perturbed so as not to represent real addresses. These addresses are then randomly assigned to applicants.
(3) High school (if Diploma earner): High schools were chosen to be in cities that are at least 10% black, at least 20% white, have at least 25,000 people, and do not have median income above $100,000. In addition, the high schools do not have test scores above the 90th or below the 10th percentile. Then applicants in a center were randomly assigned from this set of high schools that are at least 30 miles from the center (but within New Jersey), to reduce the probability that the high school can send any unobservable signals to the employer. Applicants with GEDs were randomly assigned descriptions/names of GED training programs.
(4) References: Two fictitious references with phone numbers were created, representing the applicant's supervisors for each previous job.
(5) Phone number: Each applicant will be assigned a phone number (purchased from callfire.com) based on their center, race, criminal history, and time period (that is each, each center has 4 potential phone numbers during each time period); the result of this division is that no business will ever receive two applications using the same phone number. The wording and voice on the outgoing voicemail message was randomized.
(6) Email address: A unique email address will be created for each applicant, with the format randomly varied.
We then randomly create applicant profiles for each center via the Resume Randomizer program created by Lahey and Beasley (2009). An applicant profile consists of: a name, a phone number (assigned at the center x race x crime level), an address, an employment history, a unique email address, 2 references with phone numbers, information on high school diploma or GED receipt, a criminal history status and information about the criminal charge, and a formatted resume (for use if application requires resume only).
Finding Jobs:
We will use two strategies to identify job openings for which our applicants appear qualified:
(1) snagajob.com A large online job board dedicated to hourly employment
(2) Job applications sent directly to job openings listed on the websites of chain businesses with at least 30 locations and 300 employees in NJ, per the BusinessUSA database. Industries include restaurants, convenience stores, department stores, home centers, grocery and convenience stores, pharmacies, miscellaneous retail, service stations, and hotels/motels.
Applying for the job:
Each RA will be randomly assigned a "center" in which to search for jobs via the above-described methods. Once a job is identified, a profile is randomly chosen from that center to apply for the job. The RA will then use the information contained in the profile to fill out the job application.
For each job vacancy up to two job applications will be sent in each time period. In the case that two applications are sent, we use the resume randomizer program to ensure that the second application to the same business doesn’t use the same race (and thus name and phone number), address, or criminal conviction type. The time between first and second job applications will vary.
It is possible that multiple locations of the same chain business could use the same hiring managers. To avoid allowing these managers to view extremely similar applicants from the same towns and thus arousing potential suspicion, whenever an RA applies to the same company within a geographic center, he/she will always use the same pair of applicant profiles. This mirrors real-world applicant behavior (applying to multiple nearby locations at once is common). Applications to locations in different centers, in contrast, will differ. Applications from different centers are less similar to one another (different cities of residence, most recent employers, high schools, etc.), so even if a manager covers locations in multiple centers, this is less likely to trigger suspicion.
Application record-keeping:
While filing the job application, the RA will also fill out a spreadsheet that indicates, among other things, which profile was sent to the job, the date and time of the job application, the name of the company being applied to, the name of the position, address/location of the job, when the job was posted (if available), whether the application asked about criminal history.
Outcome data collection:
We will record whether the applicant receives a call or email from the employer. Responses will be further classified as "positive response" (any contact suggesting interest) or "interview" (specifically requests interview). The main outcome of interest will be "positive response." Data will be collected for 8 weeks from the application date. Note that we are registering this study after