Intervention (Hidden)
Setting and Sample
Our partner is Michigan Works! Southwest (MWS), which operates federally-funded American Job Centers in Branch, Calhoun, Kalamazoo, and St. Joseph counties.
Intervention
The RCT randomly provides some MWS clients with access to a website and smartphone app that provides tailored information about occupations providing relatively high wages and strong employment growth. The control group receives general information. The main steps of the intervention are:
1) Job seekers who visit MWS complete an intake survey as part of the standard operating procedure. All individuals receive a link to the website/app via email. At this time, they are also randomly assigned to study group A, B, or C.
2) Individuals sign into the website/app using a MWS computer or their own computer or smartphone.
3) Individuals sign into the website/app using a MWS computer or their own computer or smartphone. The website/app is called NextUp Jobs. When they sign in, they are directed to their assigned website type (i.e. group A, group B, or group C).
For the first treatment group (group A), the website/app provides tailored information, based on self-reported information from the MWS intake, about the occupations estimated to offer relatively high wages and high amenities. (We describe the statistical model that produces these estimates below.) For each occupation, we provide a range of possible wages and other information on job characteristics. Individuals in the treatment group can use the website/app whenever they want.
For the second treatment group (group C), the website/app provides the same information on occupations estimated to offer relatively high wages and rapid re-employment. This treatment then provides additional information on whether there is additional training required to get into the given jobs and, if so, what the training path looks like (time period, costs, components, etc…) and what the specific steps are to enroll in such a training program in the Southwest Michigan Area.
The job search website displays three main types of information. First, it provides a user-friendly list of 10-40 occupations that are estimated by us to offer the highest levels of expected utility. This list contains the occupation title, a brief description of the occupation, the expected hourly wage for the individual in this occupation, a description of the occupation’s predicted job growth, and whether the occupation typically offers work from home opportunities or employer-sponsored health insurance. Second, individuals can click on an occupation to see more detailed information: the five most common tasks that are done on the job and the 25th and 75th percentiles of the predicted wage distribution for each individual.
We use individuals’ age, education level, and current or prior occupation to construct predicted wages in each potential target occupation. In particular, we estimate generalized random forest models using workers observed in the 2002-2019 and 2021-2024 Current Population Survey (CPS) who make occupational transitions during the eight CPS survey months (covering 16 calendar months). The dependent variable of the generalized random forest models is the log hourly wage. The potential explanatory variables are individuals’ education level, years of potential labor market experience (equal to age minus years of schooling minus six), the month in which they are searching for a job (to capture seasonal patterns), the state in which they are searching for a job, recent employment growth in their state of residence, recent employment growth in the target occupation, recent employment growth in the target occupation in their state, and the skill distance between individuals’ current/prior occupation and the target occupation. We construct skill distance using 35 different measures of skills— such as the level of reading comprehension, mathematics, or equipment maintenance used on the job—from O*NET. We estimate a separate random forest model for each target occupation. The outputs of the random forest model are the conditional mean and conditional quantiles of the predicted hourly wage for each person (as defined by their observed characteristics) in each occupation.
We restrict the potential set of occupations considered for each individual to a set of feasible occupations using information on education and training requirements, observed occupation-to-occupation transitions, and local job postings. Specifically, we only consider occupations for an individual if no more than 90 percent of individuals in that occupation have more education than them, there are at least 150 people employed in the job in Southwest Michigan in 2016, and there was at least 5 job posting on Indeed in Southwest Michigan for the occupation in May, July, and November 2019, and the occupational transition would not require infeasible amounts of retraining given the O*NET occupational training requirements categorization. This procedure prevents us from recommending occupations in which it would be extremely difficult or infeasible to find a job given the individual’s background.
Finally, we incorporate information on individuals’ stated preferences over job characteristics to calculate the expected utility from each occupation. On the intake survey, individuals answer questions about how strongly they value high wages, working similar hours each week, being able to work from home, and exerting no more than moderate physical activity on the job. We combine these preferences with our customized wage prediction, occupational characteristics from the CPS, and estimates of the average willingness to pay for occupational characteristics from Mas and Pallais (2017) and Maestas et al. (2022). The result is a customized ranking of occupations that reflects both individual-specific wage estimates and individual-specific preferences. Incorporating individual preferences further increases the relevance of the recommended occupations.
For the control group (group B), the website/app provides general information - a list of websites where jobs are posted, such as Indeed, Monster, and Pure Michigan Talent Connect - currently provided to MWS clients.
4) Short weekly text message surveys ask individuals about job search activity. An endline text message survey asks about their current employment situation and occupation. An endline online survey asks more questions about their employment situation, their information and beliefs about different job types, their subjective well-being, and other questions. Survey incentives will be provided for both the text message and qualtrics survey. The survey incentives amounts may be randomized.