x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
Learning from Administrative Data: NEWWS Evaluation
Last registered on September 19, 2019

Pre-Trial

Trial Information
General Information
Title
Learning from Administrative Data: NEWWS Evaluation
RCT ID
AEARCTR-0004722
Initial registration date
September 18, 2019
Last updated
September 19, 2019 12:04 PM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
MDRC
Other Primary Investigator(s)
PI Affiliation
UC Berkeley
PI Affiliation
Columbia University
PI Affiliation
UC Berkeley
PI Affiliation
Columbia University
Additional Trial Information
Status
On going
Start date
1991-06-01
End date
2021-06-30
Secondary IDs
Abstract
The Learning from Administrative Data (LAD) initiative seeks to extend the evidence of the long-term effects of various welfare-to-work programs evaluated in the 1990s, programs that included components such as earnings supplements, child care subsidies, work requirements, time limits on benefit receipt, and education services. Evaluations of these programs found that they typically increased mothers’ employment and earnings and childcare use. Some increased incomes and others did not, as increases in earnings were offset by reduced benefits. Questions remain about the long-term effects of these programs on children. Did they increase parents’ employment and/or family incomes enough to reduce poverty over the long-run or helped end the “cycle of poverty” by improving the outcomes of the children of welfare recipients?

The project will take advantage of multiple randomized controlled trials conducted by MDRC in the 1990s, including over 55,000 participants (mostly welfare recipients). It will merge data for the study participants with administrative data held by the U.S. Census’ Center for Administrative Records research and Applications (CARRA) to study the long run (over 20-year) effects of these policies on outcomes such as employment and earnings, fertility, marriage, and mortality, for adult participants and their children. This entry focuses on the analysis of the NEWWS evaluation, which tested the effects of 11 mandatory welfare-to-work programs that tested employment-focused versus education-focused approaches to increasing work and reducing benefit use.
External Link(s)
Registration Citation
Citation
Hoynes, Hilary et al. 2019. "Learning from Administrative Data: NEWWS Evaluation." AEA RCT Registry. September 19. https://doi.org/10.1257/rct.4722-1.0.
Experimental Details
Interventions
Intervention(s)
NEWWS was designed to directly and reliably test the effects of alternative approaches to helping welfare recipients find jobs and leave public assistance. The evaluation examined the effects of 11 mandatory welfare-to-work programs on welfare recipients and their children. The interventions operated in seven sites around the country and took different approaches to helping welfare recipients find jobs, advance in the labor market, and leave public assistance.

There were three approaches tested – two primary preemployment approaches (one that emphasized short-term job search assistance and encouraged people to find jobs quickly and one that emphasized longer-term skill-building activities, primarily basic education, before entering the labor market) — and a third approach that mixed elements of the other two. The programs were operated in seven sites across the country: Atlanta, Georgia; Grand Rapids, Michigan; Riverside, California; Columbus, Ohio; Detroit, Michigan; Oklahoma City, Oklahoma; and Portland, Oregon.

The study found that both the employment- and education-focused approaches substantially increased earnings during the five-year follow-up period, although the employment-focused approach produced larger increases in earnings and the effects emerged earlier in the follow-up period. Most of the programs increased earnings during the second and third years of the follow-up period, and the effects generally faded by the fourth and fifth years. Only two programs continued to produce statistically significant earnings impacts at the end of the fifth year. A subsequent analysis found no effects on work or earnings 10 to 15 years after study entry. The programs reduced public assistance receipt over the five-year period. These reductions offset the increases in earnings, however, with the result that there were no significant impacts on income (measured as earnings plus public assistance and tax credits minus payroll taxes).

See the following report for a summary of effects and a list of publications from the study: Hamilton, Gayle. 2002. Moving People from Welfare to Work: Lessons from the National Evaluation of Welfare-to Work Strategies. Washington, DC: U.S. Department of Health and Human Services. https://www.mdrc.org/sites/default/files/full_453.pdf
Intervention Start Date
1991-06-01
Intervention End Date
1997-12-30
Primary Outcomes
Primary Outcomes (end points)
The project will examine the long-term effects on key outcomes for study participants’ children, including employment and earnings, fertility, marriage, and mortality. Employment and earnings will be measured as average annual earning during the latest several-year age range available in the data. Outcomes related to fertility, marriage and mortality for the children will be measured as follows: had a birth before marriage, had a birth before age 20, married as of age 30 (pending outcome data availability), and mortality as of age 30 (pending outcome data availability). Effects will be estimated by program approach (employment-focused versus education focused) and for subgroups of children, defined by gender and age at exposure to the program.
Primary Outcomes (explanation)
There are various channels through which welfare policies can affect children. The mediators include financial resources expended on children; parental employment, education and behaviors; the quality and quantity of childcare arrangements; parental stress and mental health; and the quality of the home environment provided to the child. NEWWS had effects on several of these mediators, such as notable increases in employment and earnings among parents, an increase in the use of childcare, and an increase in parents’ education levels (GED receipt). Although in most sites the effects faded by Years 4 and 5, recent research supports the idea, that even a temporary change in family circumstances and childhood experiences can have long-term effects on children.
Secondary Outcomes
Secondary Outcomes (end points)
Secondary outcomes include: 1) employment, earnings and mortality for adult study participants; 2) participation rates in government assistance programs (Medicaid, subsidized housing, TANF, and SNAP for adult participants and their children; and 2) geographic mobility and neighborhood type for adult participants and their children.

Effects on these and the primary outcomes will also be estimated for additional subgroups of participants, defined by parents’ employment history, welfare receipt history, education level, housing status, marital status, and residential locations. Effects will also be estimated for individual subgroups within the program approaches (employment versus education-focused). Effects will also be assessed at various points in the distribution of key outcomes, through quantile regression methods.

Finally, depending on variation across the 11 programs in short-term effects on key mediators (such as parents’ employment and education and childcare use), the analysis will examine the potential mechanisms for any long-run impacts. To what extent do short-term impacts on these mediators help to explain variation in long-term impacts across sites and interventions?
Secondary Outcomes (explanation)
Theory would also predict potential long-term effects on multiple outcomes for adults, owing to early increases in work, earnings, and education. Research suggests, for example, that work experience and education affected how participants weather economic downturns and the timing of retirement, for older sample members. Research has also documented the effects of education, unemployment, and job type on health and mortality.

Experimental Design
Experimental Design
The evaluation of NEWWS used a random assignment research design, in which welfare recipients were randomly assigned to one of two or three research groups, depending on the site.
The analysis will focus on the full sample that was used in the main evaluation, including 41,715 adult welfare recipients and more than 81,000 children across all 11 sites.

Data sources to be used for the long-term analysis include the following: Employment and earnings from the IRS (if available) and from state UI agencies via the LEHD; federal tax credit (e.g. EITC) receipt, and marital status (based on tax filing status) from the IRS; Medicaid enrollment and utilization from CMS; Social Security disability program participation from SSA; Public housing program participation from HUD; Cash (TANF) and food assistance (SNAP) program participation from selected states; Fertility and death records from SSA; and California Franchise Tax Board data. The long-term analysis will also use data from the original evaluation, including a baseline survey, two- and five-year surveys, a child outcomes study survey, baseline reading and math tests, and a baseline private opinion survey.
Experimental Design Details
Not available
Randomization Method
Randomization conducted by enrollment staff via computer.
Randomization Unit
individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
41,175 adult participants, approximately 81,000 children
Sample size: planned number of observations
41,175 adult participants, approximately 81,000 children
Sample size (or number of clusters) by treatment arms
Site T1 T2 Control

1 1495 1441 1497
2 1542 1557 1455
3 1596 3384 3342
4 2513 2570 2159
5 2226 n/a 2233
6 4309 n/a 4368
7 3529 n/a 499
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
MDES for children are the following: .017 for the full sample of child participants; .034 for the test of the effects of the education-focused approach; and .031 for the test of the employment-focused approach.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
MDRC
IRB Approval Date
2018-03-15
IRB Approval Number
N/A
IRB Name
MDRC
IRB Approval Date
2018-03-15
IRB Approval Number
N/A
Analysis Plan
Analysis Plan Documents
LAD NEWWS analysis plan

MD5: 23601730063c8c47e6936957033f69f9

SHA1: ba6c57bdc774bb522f836db4e1807d85b0366170

Uploaded At: September 18, 2019

LAD NEWWS analysis plan

MD5: 23601730063c8c47e6936957033f69f9

SHA1: ba6c57bdc774bb522f836db4e1807d85b0366170

Uploaded At: September 18, 2019