Learning from Administrative Data: NEWWS Evaluation

NEWWS was designed to directly and reliably test the effects of alternative approaches to helping welfare recipients find jobs and leave public assistance. The evaluation examined the effects of 11 mandatory welfare-to-work programs on welfare recipients and their children. The interventions operated in seven sites around the country and took different approaches to helping welfare recipients find jobs, advance in the labor market, and leave public assistance.

There were three approaches tested – two primary preemployment approaches (one that emphasized short-term job search assistance and encouraged people to find jobs quickly and one that emphasized longer-term skill-building activities, primarily basic education, before entering the labor market) — and a third approach that mixed elements of the other two. The programs were operated in seven sites across the country: Atlanta, Georgia; Grand Rapids, Michigan; Riverside, California; Columbus, Ohio; Detroit, Michigan; Oklahoma City, Oklahoma; and Portland, Oregon.

The study found that both the employment- and education-focused approaches substantially increased earnings during the five-year follow-up period, although the employment-focused approach produced larger increases in earnings and the effects emerged earlier in the follow-up period. Most of the programs increased earnings during the second and third years of the follow-up period, and the effects generally faded by the fourth and fifth years. Only two programs continued to produce statistically significant earnings impacts at the end of the fifth year. A subsequent analysis found no effects on work or earnings 10 to 15 years after study entry. The programs reduced public assistance receipt over the five-year period. These reductions offset the increases in earnings, however, with the result that there were no significant impacts on income (measured as earnings plus public assistance and tax credits minus payroll taxes).

See the following report for a summary of effects and a list of publications from the study: Hamilton, Gayle. 2002. Moving People from Welfare to Work: Lessons from the National Evaluation of Welfare-to Work Strategies. Washington, DC: U.S. Department of Health and Human Services. https://www.mdrc.org/sites/default/files/full_453.pdf

1991-06-01

1997-12-30

The project will examine the long-term effects on key outcomes for study participants’ children, including employment and earnings, fertility, marriage, and mortality. Employment and earnings will be measured as average annual earning during the latest several-year age range available in the data. Outcomes related to fertility, marriage and mortality for the children will be measured as follows: had a birth before marriage, had a birth before age 20, married as of age 30 (pending outcome data availability), and mortality as of age 30 (pending outcome data availability). Effects will be estimated by program approach (employment-focused versus education focused) and for subgroups of children, defined by gender and age at exposure to the program.

There are various channels through which welfare policies can affect children. The mediators include financial resources expended on children; parental employment, education and behaviors; the quality and quantity of childcare arrangements; parental stress and mental health; and the quality of the home environment provided to the child. NEWWS had effects on several of these mediators, such as notable increases in employment and earnings among parents, an increase in the use of childcare, and an increase in parents’ education levels (GED receipt). Although in most sites the effects faded by Years 4 and 5, recent research supports the idea, that even a temporary change in family circumstances and childhood experiences can have long-term effects on children.

Secondary outcomes include: 1) employment, earnings and mortality for adult study participants; 2) participation rates in government assistance programs (Medicaid, subsidized housing, TANF, and SNAP for adult participants and their children; and 2) geographic mobility and neighborhood type for adult participants and their children.

Effects on these and the primary outcomes will also be estimated for additional subgroups of participants, defined by parents’ employment history, welfare receipt history, education level, housing status, marital status, and residential locations. Effects will also be estimated for individual subgroups within the program approaches (employment versus education-focused). Effects will also be assessed at various points in the distribution of key outcomes, through quantile regression methods.

Finally, depending on variation across the 11 programs in short-term effects on key mediators (such as parents’ employment and education and childcare use), the analysis will examine the potential mechanisms for any long-run impacts. To what extent do short-term impacts on these mediators help to explain variation in long-term impacts across sites and interventions?

Theory would also predict potential long-term effects on multiple outcomes for adults, owing to early increases in work, earnings, and education. Research suggests, for example, that work experience and education affected how participants weather economic downturns and the timing of retirement, for older sample members. Research has also documented the effects of education, unemployment, and job type on health and mortality.

The evaluation of NEWWS used a random assignment research design, in which welfare recipients were randomly assigned to one of two or three research groups, depending on the site.
The analysis will focus on the full sample that was used in the main evaluation, including 41,715 adult welfare recipients and more than 81,000 children across all 11 sites.

Data sources to be used for the long-term analysis include the following: Employment and earnings from the IRS (if available) and from state UI agencies via the LEHD; federal tax credit (e.g. EITC) receipt, and marital status (based on tax filing status) from the IRS; Medicaid enrollment and utilization from CMS; Social Security disability program participation from SSA; Public housing program participation from HUD; Cash (TANF) and food assistance (SNAP) program participation from selected states; Fertility and death records from SSA; and California Franchise Tax Board data. The long-term analysis will also use data from the original evaluation, including a baseline survey, two- and five-year surveys, a child outcomes study survey, baseline reading and math tests, and a baseline private opinion survey.

Randomization conducted by enrollment staff via computer.

individual

No

41,175 adult participants, approximately 81,000 children

41,175 adult participants, approximately 81,000 children

Site T1 T2 Control

1 1495 1441 1497
2 1542 1557 1455
3 1596 3384 3342
4 2513 2570 2159
5 2226 n/a 2233
6 4309 n/a 4368
7 3529 n/a 499

MDES for children are the following: .017 for the full sample of child participants; .034 for the test of the effects of the education-focused approach; and .031 for the test of the employment-focused approach.

LAD NEWWS analysis plan

LAD NEWWS analysis plan