Elevating Employment Success: Empowering Puerto Rico’s American Job Centers with Innovative Job Search Strategies

A recently commissioned study presents compelling evidence that case management can be improved substantially in Puerto Rico’s American Job Centers—the main provider of labor exchange services to disadvantaged populations both in the island and in the mainland US (DEDC, 2021). The potential benefits are considerable. Currently in Puerto Rico, six out of every ten individuals of working age are neither working, nor looking for employment, one of the lowest labor force participation rates in the world. The situation is influenced by the design of a variety of public transfer programs that generate powerful incentives for low-skilled individuals to remain economically inactive (Burtless and Sotomayor, 2006). In turn, low levels of labor force participation are the main determinants of the high levels of inequality and poverty in the island (Sotomayor, 1996, 2004), where 58 percent of children and 42 percent of the total population lives in poverty. The dire macroeconomic scenario is exacerbated by the increasing number of Puerto Ricans who are relocating to various U.S. states.

Our partner, the Department of Economic Development and Commerce (DEDC), is the lead agency responsible for establishing public policy for economic development in Puerto Rico. Within the agency, the Workforce Development Program (WDP) is tasked with planning and coordinating employment and training programs across the island’s 15 Local Areas of Labor Development, each overseen by a Local Workforce Development Board. The areas share common labor market characteristics and are responsible for providing services through one or more AJCs.

For this intervention, the government partner will hire an external provider of job search and support services from a pool of nationally recognized organizations. Once contracted, the selected provider will be furnished a monthly list of clients who have recently had a programmatic exit from WIOA (from the universe of AJCs). Subsequently, the entity will initiate contact with these individuals, extending a range of job search assistance services as part of their offerings. These services will include: (i) the development and implementation of individual action plans (Abel, Burger, Caranza, 2019), (ii) disseminating local and broader job opportunities to clients (Abebe et al., 2020), (iii) providing information regarding high-demand occupations, earnings, and skill requirements to broaden their search horizons (Belot, Kircher, Muller, 2018), (iv) offering instruction on effectively utilizing job search engines, (v) assistance in crafting resumés, interview preparation, and showcasing their skills (Bassi and Nansamba, 2021), and facilitating opportunities for career advancement. Once employment is secured, (vi) clients will receive guidance to address job retention challenges and the necessary support for advancing their careers over the next six months. The aim of this intervention is to implement a tailored, differentiated, and hands-on approach to job placement assistance and coaching that surpasses the quality of the conventional career services usually provided at AJCs. The study will rigorously evaluate - via a randomized controlled trial - the impact of the intensive and individualized job search assistance and career navigator support.

2024-10-01

2026-06-30

The primary outcomes of interest will be program participant employment status and earnings following treatment. Specifically, the first main outcome concerns employment status (employed=1) two, and four quarters after exiting WIOA programs. The second primary outcome corresponds to earnings two, and four quarters after exiting WIOA programs. These outcomes are currently available in the administrative data shared with the research team. However, in order to measure longer term outcomes, we have petitioned the partner agency to include medium-term follow ups in order to observe labor market outcomes 18, 24 and 30 months after WIOA program exit.

The employment outcomes two and four quarters after WIOA exit are available in the Participant Individual Record Layout (PIRL), which is a federal Department of Labor (DOL) mandated reporting layout that provides a standardized set of data elements, definitions and reporting instructions that are used to describe the characteristics, activities and outcomes of WIOA participants.

In addition to studying impacts 2 and 4 quarters after treatment, we aim to establish potential longer-term effects (performance indicators that go beyond the 4-quarter evaluation required by WIOA, including 18-month, 24-month and 30-month impacts). For this reason, we will petition the partner agency (the Workforce Development Program (WDP)) to include these medium-term and long-term follow ups.

Additional exploratory outcomes of interest will help us understand what features and components of the program might increase take up and strengthen implementation quality. They will also provide additional insights about treatment effect heterogeneity and possible effects on participant’s mobility (as measured by inter-state migration). In addition to mean earnings impact, we will include a binary variable intended to indicate the attainment of a good job. We suggest defining this variable as the median earnings goal negotiated between the Puerto Rico WDP and the US Department of Labor, in accordance with WIOA. We will also use two other stable employment variables defined by earnings levels set at 25% and 50% above the state-negotiated earnings goal. In addition, we are also interested in determining if program participant choose to migrate outside of Puerto Rico.

To measure take-up and actual participation in the program, we will use a second administrative dataset collected by the external provider. This second dataset will contain information about the characteristics of the service provided to those assigned to treatment and their engagement. This data doesn’t exist yet because the government program hasn’t launched (we anticipate that the first monthly cohort will be offered treatment in September 2024). We anticipate that it will include the following variables: (i) frequency, duration and format of interaction between the client and the career navigator/coach assigned (number of times that the client interacted with the coach, format of that interaction, be it in person or virtual, duration of the interaction); (ii) type of service requested (coaching, job search assistance, etc); (iii) characteristics of the coach (gender, years of experience, educational background); (iv) client perceptions, experiences and satisfaction with the service received.

As mentioned, we are interested in determining if program participant choose to migrate outside of Puerto Rico. To achieve this, we will petition to our government partner to link participant records to Unemployment Insurance databases in the mainland US through the State Wage Interchange System. The migration information is not reported in the PIRL database.

On a monthly basis, a randomly assigned group of individuals exiting Title I and Title III WIOA programs will be offered the package of additional, comprehensive/tailored employment services that have shown positive effects when rigorously evaluated in other settings. The intervention will take place for up to 18 months. Each month, the target sample size will be approximately 1,500 individuals (average number of potential subjects with a WIOA exit record that could be randomized into treatment and control).

Once the government partner agency’s data services provider releases the monthly list of WIOA Title I and Wagner Payser program participants with an exit record, an official from the partner agency will the execute a randomization code, stratifying participants by gender and WIOA program of exit category (Adult, Dislocated Worker, Youth, and Wagner-Peyser). Within each stratum, an equal number of clients will be assigned to treatment and control. Once randomized, only the subset assigned to treatment will be shared with the contracted external provider to initiate contacts, thereby ensuring the integrity of the control group. This process will be repeated in each of the subsequent months throughout the intervention period.

Not available

Randomization in government office by a computer.

The unit of randomization is the individual.

No

According to the latest available numbers from the Puerto Rico WIOA & Wagner-Peyser Annual Report (PY 2022) , in 2022 there were 9,153 Title I program exits: 46% were from the Adults Program (n=4,251), 42% Dislocated Workers (n=3,845) and the remaining 12% Youth Program exits (n=1,057). In addition, there were 7,455 Wagner-Peyser (Title III) program exits. Combining these two sub-groups (Title I and Title III participants), we have a planned number of 18,000 potential clients (those individuals with a WIOA exit record) over the course of a year. Thus, we estimate an average monthly flow of 1,500 individuals that could be randomized into treatment and control (planned monthly sample size). The government agency is open to running the intervention over a longer period, as needed (between 12 to 18 months), which might increase the sample size accordingly.

18,000 individuals.

Over the course of an entire year, 9,000 individuals will be assigned to control; 9,000 individuals assigned to treatment.
Note that the operating capacity of the selected provider may require adjusting the treatment allocation (proportion of the sample assigned to treatment).

The weighted average employment rate for clients exiting AJC services stood at 53 percent (dropped to 44 percent three quarters thereafter) (WDP 2022). Assuming a control mean of 50% employment rate, power calculations (power=0.80, α=0.05) establish that with a sample size of 18,000 individuals and an equal split, we will be able to detect a +2.08 pp effect size (equivalent to 0.05 standard deviations). Considering a conservatively modest treatment take-up rate of 20 percent, this will correspond to a +10 pp increase among takers. Second, using compensation data from Puerto Rico Community Service (2017-2021) for a sample of individuals that resemble WIOA participants (average annual compensation of $24,500), and using the residual standard deviation of the regression of compensation on a set of covariates (including gender, age, age2, and education), with a sample size of 18,000 individuals, we will be able to detect an annual increase between $1,094-$1,467 (which corresponds to a 0.04-0.05 SD depending on the proportion of the sample assigned to treatment). Our power analysis is conservative as we will use other baseline variables to reduce the outcome's residual variance since we have access to the PR WIOA Participant Record Information System (PRIS), which allows us to incorporate a ser of rich covariates and multiple data points both before and after the intervention.