National Evaluation of Welfare-to-Work Strategies (NEWWS)

The 11 programs in the NEWWS Evaluation were operated in seven sites across the country. Employment-focused programs were operated in Atlanta, Georgia; Grand Rapids, Michigan; Riverside, California; and Portland, Oregon. Atlanta, Grand Rapids, and Riverside also operated education-focused programs, as did Columbus, Ohio (two programs); Detroit, Michigan; and Oklahoma City, Oklahoma. The four employment-focused programs--the Labor Force Attachment (LFA) programs in Atlanta, Grand Rapids, and Riverside, and Portland's program--assigned most enrollees to job club as their first activity, and they encouraged enrollees to find work as quickly as possible. Further, both Portland's and Riverside's programs employed full-time job developers to help place program enrollees in unsubsidized jobs.

In contrast to the three LFA programs, however, Portland's program offered GED preparation classes to people who case managers thought had a good chance of attaining a GED certificate relatively quickly. Furthermore, Portland case managers, more often than those in the LFA programs, encouraged enrollees to hold out for jobs that paid well above the minimum wage (about 25 percent higher) and that offered the best chance for long-lasting and stable employment. Case managers in the LFA programs, especially Riverside's, stressed the value of starting off with any job, even a low-paying one, and then advancing toward more stable and better-paying jobs in the future.

The Human Capital Development (HCD) programs in Atlanta, Grand Rapids, and Riverside; the Columbus Integrated and Traditional case management programs; and the programs in Detroit and Oklahoma City can be characterized as "education-focused." A large percentage of program enrollees in these programs were initially assigned to some type of skill-building activity. The types of activities to which enrollees were first assigned depended, in part, on the level of educational attainment that individuals had achieved prior to entering the program. Those who had not completed high school or received a GED certificate but who were assessed by case managers as having high-school-level skills were assigned to GED preparation classes. Those with lower reading or math levels were assigned to adult basic skills classes. In addition, non-English speakers could be assigned to English as a Second Language (ESL) programs. Finally, those who had completed high school or held a GED certificate could be assigned to vocational training or employment-oriented skills courses at local community colleges. All in all, however, assignments to GED preparation or basic education courses predominated in these education-focused programs, and assignments to vocational training programs were less common, primarily as a result of welfare recipients' low levels of educational achievement; enrollment in college played an even smaller role.

Other key program features varied across the 11 studied programs as well. All four employment-focused programs and five of the seven education-focused programs can be considered "high enforcement" programs: They worked with a cross-section of the welfare applicants and recipients who were required to participate; monitored participation closely; and, especially in the two programs in Columbus and the two in Grand Rapids, frequently invoked sanctions (reductions in welfare grants) for nonparticipation. The remaining two education-focused programs, in Detroit and Oklahoma City, did not have these characteristics (because of either lack of funds or program philosophy) and can be considered "low enforcement" programs.

The programs also differed in their child care policies and practices (within each site, however, child care assistance policies were identical for program and control group members). During the early to mid-1990s, the Atlanta, Oklahoma City, Portland, and Detroit programs provided the strongest staff support for arranging for child care, and the programs in Atlanta and Oklahoma City emphasized the use of licensed care; in contrast, case managers for both Riverside programs encouraged enrollees to find low- or zero-cost, informal child care.
The programs also differed in their case management strategies. Three programs--Columbus Integrated, Portland, and Oklahoma City--implemented an integrated case management staffing arrangement. The other programs used a traditional case management structure.

1991-06-01

1999-12-31

Primary outcomes included employment rates, earnings amounts, income amounts, and public assistance receipt rates and amounts.

Secondary outcomes included education and training credential attainment, reading and math literacy levels, and child well-being. The study also included a cost-benefit analysis.

To assess the effectiveness of different welfare-to-work strategies, the evaluation used a random assignment research design. In each of the seven sites in the evaluation, people who were required to participate in a welfare-to-work program were assigned, by chance, to one or two program groups, which had access to employment and training services and whose members were required to participate in the program, or to a control group, which received no program services and whose members were not subject to a participation requirement but could seek out similar services on their own in the community. Program group members who did not comply with the participation mandate risked incurring a sanction, that is, having their welfare grant reduced. Sample members in each research group were tracked over a follow-up period of five years after their date of random assignment.

Four of the 11 sites implemented a three-way random assignment research design in order to test the effectiveness of two different program approaches. In the three-way design, each person was assigned, by chance, to either one type of program, another type of program, or a control group.

Three of these four sites (Atlanta, Grand Rapids, and Riverside) operated two programs that were designed--for purposes of the evaluation--to magnify the differences between the employment- and education-focused approaches described above. The Labor Force Attachment (LFA) programs emphasized rapid job placement as the best way for welfare recipients to develop their work habits and skills, even if the job paid low wages. The Human Capital Development (HCD) programs emphasized that welfare recipients had to develop their "human capital," that is, their knowledge and basic skills, through education and training, before seeking employment, in order to have a better chance of finding and keeping jobs and advancing toward well-paid and secure employment.

Unlike in the other sites that used a three-way design, the goal in the three-way Columbus test was to compare the effectiveness of two different case management models. In the Traditional model, one worker handled the welfare department’s employment and training function and another worker handled welfare eligibility and payment issues--often called "income maintenance." Both workers maintained relatively large caseloads. In the Integrated model, a single worker handled both the employment and training and income maintenance functions. In the Integrated model, the worker maintained a smaller caseload than either of the workers in the Traditional model since, on a per client basis, the worker was handling jobs "traditionally" done by two workers.

The remaining three sites in the evaluation (Oklahoma City, Detroit, and Portland) used a two-way random assignment design to test the effectiveness of program models already established in those sites. In other words, instead of implementing a program designed expressly for research purposes, as in the three-way sites, program administrators in each of the two-way sites randomly assigned people to a group that entered their welfare-to-work program or to a control group.

A "two-stage" random assignment design was also included in NEWWS. This was implemented in two of the three LFA/HCD sites -- Riverside and Grand Rapids. In the first stage, over 23,000 welfare recipients were randomly assigned to two groups when they appeared in the welfare office for in-person eligibility (or eligibility redetermination) interviews. One group was required to attend a welfare-to-work program orientation. The other group was excused from such an orientation or any program participation. The comparison of the early experiences of these two groups showed the effects of a mandate to enter a welfare-to-work program without including any effects of program activities and services. Individuals randomly assigned at this first stage to the "required to attend orientation" group were subsequently randomly assigned again (the second stage), if and when they attended orientation, to one of the three research groups described above.

More details are available in the report cited under Post-Trial: Relevant Papers.

Randomization was done by MDRC, via a computer.

individual

No

Over 55,000 individuals

Over 55,000 individuals

15,553 control, 26,162 treatment in the "second stage" random assignment, with number in treatment arms varying by site, as specified below.

Three-way random assignment sample sizes included the following, for the five-year follow-up analysis:
Atlanta: 1,497 control, 1,441 Labor Force Attachment (LFA), 1,495 Human Capital Development (HCD)
Grand Rapids: 1,445 control, 1,557 LFA, 1,542 HCD
Riverside: 3,342 control, 3,384 LFA, 1,596 HCD
Columbus: 2,159 control, 2,513 Integrated case management model, 2,570 Traditional case management model

Two-way RA included the following, for the five-year follow-up analysis:
Portland: 499 control, 3,529 program
Detroit: 2,233 control, 2,226 program
Oklahoma City: 4,368 control, 4,309 program

As noted above, over 23,000 individuals were randomly assigned in the "first stage" of random assignment in Riverside and Grand Rapids.

This research synthesis lays out the lessons learned from the National Evaluation of Welfare-to-Work Strategies (NEWWS) and addresses critical questions in the welfare-to-work policy discussion. Questions addressed include the following:
Control group outcomes: How do welfare recipients fare in the absence of welfare-to-work programs?
Participation in education and training: Can mandatory welfare-to-work programs engage large numbers of people in education and training?
Participation in other activities: What are typical patterns of participation in non-education and training activities?
GED and other credential receipt: Do welfare-to-work programs' investments in education and training result in higher rates of credential attainment?
Gains in skills: Do welfare-to-work programs' investments in education and training result in higher skills?
Adult education: What factors enhance or diminish its beneficial effects?
Net impacts: How effective are different types of welfare-to-work programs?
Relative impacts: Which types of programs are generally most effective?
The Labor Force Attachment approach versus the Human Capital Development approach: In head-to-head tests, which is more effective?
The most effective program: What were its distinguishing features?
Subgroup findings: What types of programs work best for which groups of welfare recipients?
Education and training reconsidered: Can they be made more effective?
Family circumstances: Can programs have long-term spillover effects on family outcomes such as marriage and fertility?
Children's well-being: How might programs that have mandates and services but leave income unchanged affect children in the long run?
Income: How can welfare-to-work programs increase family resources?
Case management: Do different strategies yield different results?
Participation standards: What does it take to engage a substantial proportion of people in welfare-to-work program activities?
Mandate enforcement: What role does enforcing mandates play in program effectiveness?
Costs: What contributes to the cost of welfare-to-work programs?
Costs relative to benefits: What is the government's financial return on its investment in welfare-to-work programs?

Hamilton, Gayle. 2002. Moving People from Welfare to Work: Lessons from the National Evaluation of Welfare-to-Work Strategies. Washington, DC: U.S. Department of Health and Human Services and U.S. Department of Education.

https://www.mdrc.org/publication/moving-people-welfare-work

This research synthesis lays out the lessons learned from the National Evaluation of Welfare-to-Work Strategies (NEWWS) and addresses critical questions in the welfare-to-work policy discussion. Questions addressed include the following:
Control group outcomes: How do welfare recipients fare in the absence of welfare-to-work programs?
Participation in education and training: Can mandatory welfare-to-work programs engage large numbers of people in education and training?
Participation in other activities: What are typical patterns of participation in non-education and training activities?
GED and other credential receipt: Do welfare-to-work programs' investments in education and training result in higher rates of credential attainment?
Gains in skills: Do welfare-to-work programs' investments in education and training result in higher skills?
Adult education: What factors enhance or diminish its beneficial effects?
Net impacts: How effective are different types of welfare-to-work programs?
Relative impacts: Which types of programs are generally most effective?
The Labor Force Attachment approach versus the Human Capital Development approach: In head-to-head tests, which is more effective?
The most effective program: What were its distinguishing features?
Subgroup findings: What types of programs work best for which groups of welfare recipients?
Education and training reconsidered: Can they be made more effective?
Family circumstances: Can programs have long-term spillover effects on family outcomes such as marriage and fertility?
Children's well-being: How might programs that have mandates and services but leave income unchanged affect children in the long run?
Income: How can welfare-to-work programs increase family resources?
Case management: Do different strategies yield different results?
Participation standards: What does it take to engage a substantial proportion of people in welfare-to-work program activities?
Mandate enforcement: What role does enforcing mandates play in program effectiveness?
Costs: What contributes to the cost of welfare-to-work programs?
Costs relative to benefits: What is the government's financial return on its investment in welfare-to-work programs?

Hamilton, Gayle. 2002. Moving People from Welfare to Work: Lessons from the National Evaluation of Welfare-to-Work Strategies. Washington, DC: U.S. Department of Health and Human Services and U.S. Department of Education.

https://www.mdrc.org/publication/moving-people-welfare-work

For the past 30 years, federal and state policymakers have been legislating various types of programs to increase employment among welfare recipients. How people can best move from welfare to work, however, has been the subject of long-standing debate. This report, summarizing the long-term effects of 11 mandatory welfare-to-work programs on welfare recipients and their children, represents a major advance in resolving this debate. The findings are the final ones from the National Evaluation of Welfare-to-Work Strategies (NEWWS), a multi-year study of alternative approaches to helping welfare recipients find jobs, advance in employment, and leave public assistance.

"What works best, and for whom?" is the central question animating this report and the NEWWS Evaluation as a whole. In particular, the evaluation compares the effects of two alternative pre-employment strategies, for different groups of welfare recipients: programs that emphasize short-term job search assistance and encourage people to find employment quickly (referred to as "Labor Force Attachment" [LFA] or, more broadly, "employment-focused" programs); and programs that emphasize longer-term skill-building activities, primarily basic education (referred to as "Human Capital Development" [HCD] or, more broadly, "education-focused" programs). The effects of each approach are estimated from a wealth of data pertaining to over 40,000 single parents (mostly mothers) and their children, and a five-year follow-up period (falling somewhere between 1991 and 1999, depending on the site), using an innovative and rigorous research design based on the random assignment of individuals to one or more program groups (with services) or to a control group (without services).

Findings in Brief

The research designs that were implemented in the NEWWS Evaluation permit many comparisons. The key ones examined the programs' economic effects on adults and the "spillover" effects on noneconomic outcomes and child well-being, as summarized below.

Comparing All 11 Programs to What Would Have Happened in the Absence of the Programs
-- In the absence of any welfare-to-work program over a five-year follow-up period, approximately three-quarters of single-parent welfare recipients found jobs, and more than half left the welfare rolls. Few of the 11 studied programs improved on this already-high rate of job-finding, but nearly all programs helped single parents work during more quarters of the follow-up and earn more than they would have in the absence of a program. Moreover, all programs decreased welfare receipt and expenditures over the five years.
-- Measured combined income, however, was largely not affected: The programs led to individuals' replacing welfare and Food Stamp dollars with dollars from earnings and Earned Income Tax Credits (EITCs), but the programs did not increase income above the low levels of the control group.
-- The programs achieved their economic gains with few spillover effects on such family measures as marriage, fertility, and household composition. Notably, the adults' gains in self-sufficiency (defined as increased employment and decreased welfare receipt) were achieved with few indications of harm or benefit to the well-being of their children. This was particularly true for mothers with young children, who in 1988 were newly mandated to participate in programs. Because the new mandate's implications for children were of considerable concern at the time, these families were the subject of intense study in this evaluation.

Comparing Labor Force Attachment (LFA) and Human Capital Development (HCD) Programs
-- By rigorously comparing LFA and HCD programs--versions of employment-focused and education-focused programs designed to magnify the differences between the two types of strategies and operated side by side in three evaluation sites--it was found that the HCD approach did not produce added economic benefits relative to the LFA approach.
-- Moreover, the LFA approach moved welfare recipients into jobs more quickly than did the HCD approach--a clear advantage when federally funded welfare months are time-limited.
-- Finally, the LFA approach was much cheaper to operate than the HCD approach and, at the same time, did not affect sample members' overall financial well-being or their children's well-being any differently than the HCD approach.
-- Surprisingly, these findings held true for program enrollees who lacked a high school diploma or a General Educational Development (GED) certificate as of study entry--the subgroup of welfare recipients who were expected to derive the greatest benefit from an initial investment in basic education--as well as for those who already possessed these education credentials.

Comparing Employment-Focused and Education-Focused Programs
-- Dividing all 11 programs into two broad categories--employment-focused programs and education-focused programs--programs in the former category generally had larger effects on employment, earnings, and welfare receipt than those in the latter category.
-- Given the large number of programs examined and their variety of served populations, implementation features, and labor markets, these results provide more support for the advantages of employment-focused programs than for education-focused ones.

These results should not be taken as an indictment of the benefits of education and training in general in welfare-to-work programs. Nonexperimental work done as part of the NEWWS Evaluation has suggested that obtaining a GED and, especially, obtaining a GED and then receiving some type of vocational training, can result in employment and earnings gains for those who achieve these milestones. However, in the context of mandatory welfare-to-work programs, few people make it this far, for many reasons, including: people leave welfare and therefore do not stay in welfare-to-work programs, and thus education or training classes, for very long; adults supporting families cannot afford an up-front deferment of employment and earnings that may or may not have a longer-run payoff; and only a small minority of welfare recipients report that, if given a choice, they prefer to go to school to study basic reading and math over going to school to learn a job skill or going to a program to get help looking for a job. It should be noted as well that none of these programs made assignments to or emphasized college.

The Features of the Most Effective Program
-- One program--the Portland (Oregon) one--by far outperformed the other 10 programs in terms of employment and earnings gains as well as providing a return on every dollar the government invested in the program.
-- The Portland employment-focused program, unlike either the LFA or the HCD programs or the other education-focused programs, initially assigned some enrollees to very short-term education or training and others (the majority) to job search. Also, in another departure from the other programs, job search participants in Portland were counseled to wait for a good job, as opposed to taking the first job offered. While other aspects of the Portland program, such as its use of job developers and staff's experience operating job search programs, were also noteworthy, these distinctive features, along with other past research, suggest that a "mixed" approach--one that blends both employment search and education or training--might be the most effective.

Findings for Children
-- Considering the six programs (three sites) in which children who were preschool age at random assignment were studied in depth, impacts were found on a small number of measures of child well-being--predominantly in the area of the young children's social skills and behavior. Overall, the young-child impacts differed more often by site than by welfare-to-work approach.
-- Program effects on child care--one important way in which children might be affected by welfare-to-work programs--diminished from the two-year follow-up point to the end of the five-year follow-up. As of this latter point, only the Portland program was still producing an increase in the use of child care.
-- In the seven programs (four sites) in which a limited number of measures were examined for children of all ages, few effects were evident. Some impacts, however, were found relating to young adolescents' academic functioning (but in only two of the four sites for which data are available), and these impacts on adolescents were predominantly unfavorable. As was the case for young children, impacts on children of all ages did not differ by welfare-to-work program approach.

Comparisons Shedding Light on Other Welfare-to-Work Program Design Issues
-- Of the two programs with low enforcement of the participation mandate, one had no impact on employment and earnings, and the other had only small effects. It appears that a minimum level of enforcement by program staff is required to produce at least moderate employment impacts, likely because this extra "push" is needed in order to engage in program activities those who normally would not participate on their own initiative.
-- Two of the three programs that used "integrated," as opposed to "traditional," case management worked well for those who entered the study without a high school diploma or GED. In integrated case management, one worker fulfills the responsibilities related to the payment of welfare and other benefits, normally performed by income maintenance staff, as well as the responsibilities related to the provision of employment-related services, usually assigned to welfare-to-work program staff. In traditional case management, each welfare recipient has two different case managers. Two programs that implemented different versions of well-funded and well-supported integrated case management produced relatively large impacts for nongraduates; the third program, which also used an integrated case management model but one that was hampered by tight funding, had limited impacts.

The Limits of Pre-Employment Strategies

Average income levels among control group members over the five-year follow-up period were low. Despite the successes of these programs, no program, not even Portland's, met the long-range goal of making enrollees substantially better off financially. Most program group members continued to have low incomes from various combinations of earnings, the EITC, welfare, and Food Stamps. In fact, among individuals who lacked a high school diploma or GED as of study entry, some programs had the five-year result of making them financially worse off. These findings suggest that the challenge of the future is to identify other types of programs or initiatives that can provide welfare recipients with better and more stable jobs, increase their income, and improve the well-being of their children.

Hamilton, Gayle, Stephen Freedman, Lisa A. Gennetian, Charles Michalopoulos, Johanna Walter, Diana Adams-Ciardullo, Anna Gassman-Pines, Sharon McGroder, Martha Zaslow, Jennifer Brooks, and Surjeet Ahluwalia. 2001. How Effective Are Different Welfare-to-Work Approaches? Five-Year Adult and Child Impacts for Eleven Programs. Washington, DC: U.S. Department of Health and Human Services and U.S. Department of Education.

http://www.mdrc.org/publication/how-effective-are-different-welfare-work-approaches