An Experimental Evaluation of Philadelphia WorkReady

The Philadelphia Youth Network (PYN) has offered a youth summer jobs programs called WorkReady for over 15 years. Using a blend of government and private funding, PYN contracts with 50-60 local agencies that implement the six-week WorkReady summer program. Youth ages 14 to 21 apply with a common application either to PYN or to a local agency directly. Program youth are assigned to a local provider, which places them in one of three program models (service learning, work experience, or internship). All three models focus on developing “21st-Century Workforce Skills” and offer an hourly wage.

2017-07-03

2017-08-25

Final outcomes will depend on data availability. Outcomes for which we are pursuing data include: criminal behavior, education engagement and performance, labor market involvement, health, fertility, and household functioning. We are also conducting qualitative interviews and observations with a small number of study subjects to help document program implementation and context, understand the counterfactual condition, and generate hypotheses about mechanisms.

Demand regularly outpaces available resources; last year over 16,000 youth applied for a summer job through PYN, while less than half were awarded a working opportunity. Historically, slots have been allocated either by first-come, first-served policies or by the choice of the provider. For the 2017 program, PYN has committed to a more equitable distribution of a subset of program slots through the implementation of a fair lottery. Youth applicants will be blocked by age and geographic location, then individually randomly assigned to be offered the program or not.

Randomization will be conducted by the researchers on a computer in an office.

Individual

No

N/A

We anticipate randomizing around 3,000 applicants for 1,000 program slots.

Youth will be assigned to different program types, but not randomly. So there is a single treatment arm.

With 100% take-up, the MDE would be about 0.1. Based on the pilot study, however, we expect the effective take-up rate to be between 30 and 50 percent. If the resulting power is too low, we may repeat the study in summer 2018.

This paper combines two new summer youth employment experiments in Chicago and Philadelphia with previously published evidence to show how repeated study of an intervention as it scales and changes contexts can guide decisions about public investment. Two sources of treatment heterogeneity can undermine the scale-up and replication of successful human capital interventions: variation in the treatment itself and in individual responsiveness. Results show that these programs generate consistently large proportional decreases in criminal justice involvement, even as administrators recruit additional youth, hire new local providers, find more job placements, and vary the content of their programs. Using both endogeneous stratification within cities and variation in 62 new and existing point estimates across cities uncovers a key pattern of individual responsiveness: impacts grow linearly with the risk of socially costly behavior each person faces. Identifying more interventions that combine this pattern of robustness to treatment variation with bigger effects for the most disconnected could aid efforts to reduce social inequality efficiently.

Heller, Sara B. “When Scale and Replication Work: Learning from Summer Youth Employment Experiments” (2022), Journal of Public Economics, 209(104617)

https://doi.org/10.1016/j.jpubeco.2022.104617

Administrative burden reduces the effectiveness of public social programs by deterring take up among adults, but we know little about the role these burdens play in public programs for young people. This paper uses empirical evidence to assess how different barriers shape adolescents’ take-up of summer jobs programs. In a Philadelphia experiment, we find that reminder emails increased application completion by 1.8 percentage points (12.3 percent), with bigger effects from emphasizing short-term monetary gains. In a non-experimental analysis of Philadelphia and Chicago programs, we show that without individualized support during enrollment, disconnected youth are less likely to participate when offered a slot than their more advantaged peers. However, offering universal personalized support during enrollment makes them as or more likely to participate. These findings suggest administrative burden does constrain the benefits of public spending on youth programs and that reducing burden can increase gains from social programs for young people.

Bhanot, Syon & Sara B. Heller. “Does Administrative Burden Deter Young People? Evidence from Summer Jobs Programs” (2022), Journal of Behavioral Public Administration, 5(1)

https://doi.org/10.30636/jbpa.51.300