WorkAdvance
Last registered on October 04, 2016

Pre-Trial

Trial Information
General Information
Title
WorkAdvance
RCT ID
AEARCTR-0001646
Initial registration date
October 04, 2016
Last updated
October 04, 2016 2:50 PM EDT
Location(s)
Primary Investigator
Affiliation
MDRC
Other Primary Investigator(s)
Additional Trial Information
Status
Completed
Start date
2011-06-01
End date
2015-09-30
Secondary IDs
Abstract
Launched in 2011, WorkAdvance goes beyond the previous generation of employment programs by introducing demand-driven skills training and a focus on jobs that have career pathways. The model is heavily influenced by the positive findings from the Sectoral Employment Impact Study (SEIS) completed in 2010. A major component of the WorkAdvance model, in common with the programs studied in the SEIS, is formal training offering industry-recognized certifications, reflecting the hypothesis that skills acquisition is necessary for advancement. The model also requires providers to be far more employer-facing than traditional training programs, taking into account multiple employers’ changing skill requirements, employee assessment practices, and personnel needs. The evaluation consisted of several analyses: implementation, cost, participation, and two-year economic impacts.The economic results are based on unemployment insurance earnings records and a second-year follow-up survey.

- All providers translated the WorkAdvance model into a set of concrete services, but it took time — more than a year for some components and providers — and a substantial amount of technical assistance and support. As a result, at some sites, later study enrollees were more likely than earlier ones to experience a fully implemented and “mature” WorkAdvance program.

- Overall, WorkAdvance resulted in large increases in participation in every category of services, as well as in training completion, credential acquisition, and employment in the targeted sector, compared with what would have happened in the absence of the program. Expenditures for the operation of WorkAdvance fell between $5,200 and $6,700 per participant at the four providers delivering the program.

- WorkAdvance providers increased earnings, with variation in results that closely matched the providers' experience in running sector-based programs and the extent to which the services they offered were demand driven. The most experienced sectoral provider, Per Scholas, had large and consistent impacts on both primary and secondary outcomes. Madison Strategies Group and Towards Employment, providers new to sectoral training, had promising but less consistent results that grew stronger for later enrollees. One provider, St. Nicks Alliance, did not produce positive impacts. The results did not differ dramatically across subgroups, though encouragingly, WorkAdvance was able to increase earnings among the long-term unemployed.

The evaluation as a whole provides important information for workforce development providers interested in pursuing a sector strategy. The analysis considers the role played by providers’ sector-specific training and preparation and the role played by the nature of the sectors themselves. Future priorities that emerge from the results are (1) understanding how to help the more disadvantaged access the programs and (2) learning how to build service capacity, given how complex the model is to run.
External Link(s)
Registration Citation
Citation
Hendra, Richard. 2016. "WorkAdvance." AEA RCT Registry. October 04. https://doi.org/10.1257/rct.1646-1.0.
Former Citation
Hendra, Richard. 2016. "WorkAdvance." AEA RCT Registry. October 04. http://www.socialscienceregistry.org/trials/1646/history/11046.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
WorkAdvance is a workforce development model designed to help unemployed and low-wage working adults increase their employment and earnings by finding good quality jobs in selected sectors that have room for advancement within established career pathways. The essential theory behind WorkAdvance is that strategic upgrades in human capital - that is, education and employment-related skills and experience - will lead to advancement in the labor market, but only if training and job preparation are directly aligned with specific job openings. The model consists of five key components: (1) intensive screening of program applicants before enrollment; (2) sector-appropriate preemployment and career readiness services; (3) sector-specific occupational skills training intended to impart skills and lead to industry-recognized credentials; (4) sector-specific job development and placement services intended to facilitate entry into positions with opportunities for advancement; and (5) postemployment retention and advancement services intended to help participants retain and advance in their jobs.
Intervention Start Date
2011-06-01
Intervention End Date
2015-06-30
Primary Outcomes
Primary Outcomes (end points)
Sectoral training completion, credentials, employment, earnings, wages, job benefits, advancement, income
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The WorkAdvance programs at the four sites were studied using a random assignment design. Individuals who met WorkAdvance eligbility criteria, as well as the requirements for the specific sector rograms, were assigned at random to the WorkAdvance group or to the control group. Members of the WorkAdvance group were eligible to receive WorkAdvance services, while those in the control group were not eligible for these services but could get other services and support available in the community. Both research groups were tracked over time. The main impact analysis was completed at the site-level due to substantial variation in starting points and organizational emphases. In the subgroup analyses, however, the sites were pooled since there was not enough statistical power to support such analyses at the site level. Impact estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members.

Data sources for the evaluation included a baseline survey, field research, program tracking data provided by the providers, unemployment insurance wage and benefits data (nine quarters of follow-up for all sample members), and a participant survey at about 2 years after random assignment.
Experimental Design Details
Randomization Method
Randomization was done by program staff by computer
Randomization Unit
Individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
No clusters
Sample size: planned number of observations
2,564 individuals
Sample size (or number of clusters) by treatment arms
1,293 individuals in treatment group, 1,271 individuals in control group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Please see Appendix A in the WorkAdvance report: http://www.mdrc.org/sites/default/files/2016_Workadvance_Final_Web.pdf
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
MDRC
IRB Approval Date
2011-05-03
IRB Approval Number
Details not available
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
Yes
Intervention Completion Date
June 30, 2015, 12:00 AM +00:00
Is data collection complete?
No
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
No
Reports and Papers
Preliminary Reports
Relevant Papers
Abstract
This report summarizes the two-year findings of a rigorous random assignment evaluation of the WorkAdvance model, a sectoral training and advancement initiative. Launched in 2011, WorkAdvance goes beyond the previous generation of employment programs by introducing demand-driven skills training and a focus on jobs that have career pathways. The model is heavily influenced by the positive findings from the Sectoral Employment Impact Study (SEIS) completed in 2010. A major component of the WorkAdvance model, in common with the programs studied in the SEIS, is formal training offering industry-recognized certifications, reflecting the hypothesis that skills acquisition is necessary for advancement. The model also requires providers to be far more employer-facing than traditional training programs, taking into account multiple employers’ changing skill requirements, employee assessment practices, and personnel needs. This report presents the implementation, cost, participation, and two-year economic impacts of WorkAdvance. The economic results are based on unemployment insurance earnings records and a second-year follow-up survey.

The WorkAdvance program operations and evaluation are funded through the federal Social Innovation Fund (SIF), a public-private partnership administered by the Corporation for National and Community Service. This SIF project is led by the Mayor’s Fund to Advance New York City and the NYC Center for Economic Opportunity in collaboration with MDRC.

Key Findings
- All providers translated the WorkAdvance model into a set of concrete services, but it took time — more than a year for some components and providers — and a substantial amount of technical assistance and support. As a result, at some sites, later study enrollees were more likely than earlier ones to experience a fully implemented and “mature” WorkAdvance program.

- Overall, WorkAdvance resulted in large increases in participation in every category of services, as well as in training completion, credential acquisition, and employment in the targeted sector, compared with what would have happened in the absence of the program. Expenditures for the operation of WorkAdvance fell between $5,200 and $6,700 per participant at the four providers delivering the program.

- WorkAdvance providers increased earnings, with variation in results that closely matched the providers’ experience in running sector-based programs and the extent to which the services they offered were demand driven. The most experienced sectoral provider, Per Scholas, had large and consistent impacts on both primary and secondary outcomes. Madison Strategies Group and Towards Employment, providers new to sectoral training, had promising but less consistent results that grew stronger for later enrollees. One provider, St. Nicks Alliance, did not produce positive impacts. The results did not differ dramatically across subgroups, though encouragingly, WorkAdvance was able to increase earnings among the long-term unemployed.

The evaluation as a whole provides important information for workforce development providers interested in pursuing a sector strategy. The analysis considers the role played by providers’ sector-specific training and preparation and the role played by the nature of the sectors themselves. Future priorities that emerge from the results are (1) understanding how to help the more disadvantaged access the programs and (2) learning how to build service capacity, given how complex the model is to run.
Citation
Hendra, Richard, David H. Greenberg, Gayle Hamilton, Ari Oppenheim, Alexandra Pennington, Kelsey Schaberg, and Betsy L. Tessler. 2016. Encouraging Evidence on a Sector-Focused Advancement Strategy: Two-Year Impacts from the WorkAdvance Demonstration. New York: MDRC.