x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
National Guard Youth ChalleNGe Evaluation
Last registered on January 13, 2017

Pre-Trial

Trial Information
General Information
Title
National Guard Youth ChalleNGe Evaluation
RCT ID
AEARCTR-0001385
Initial registration date
January 13, 2017
Last updated
January 13, 2017 3:43 PM EST
Location(s)
Region
Primary Investigator
Affiliation
MDRC
Other Primary Investigator(s)
Additional Trial Information
Status
Completed
Start date
2005-07-15
End date
2011-06-15
Secondary IDs
Abstract
An estimated five million Americans between the ages of 16 and 24 are both out of school and unemployed. These youth are more likely than those who work or complete a degree to face long-term unemployment, permanent school dropout, welfare dependence, and criminal involvement and incarceration. The National Guard Youth ChalleNGe Program was developed in the early 1990s to improve the employment and life prospects of youth, ages 16 to 18, who have dropped out of high school and have not found a place in the labor market. It currently operates in 27 states and Puerto Rico, and has served more than 100,000 youth over its history.

ChalleNGe was designed on the premise that programs for at-risk youth must provide comprehensive, long-term instruction and assistance. It builds on a positive youth development model and includes a wide array of activities designed to strengthen young people’s preparedness for work and adult responsibilities. Data from the states in which ChalleNGe operates suggest that the program has strong potential to make a positive difference in participants’ lives.

MDRC, in partnership with the MacArthur Foundation Research Network on the Transition to Adulthood, conducted an evaluation of ChalleNGe study in 10 ChalleNGe sites found that, after three years, participants were more likely than their control group counterparts to have obtained a GED or high school diploma, to have earned college credits, and to be working. Their earnings are also 20 percent higher.
External Link(s)
Registration Citation
Citation
Bloom, Dan. 2017. "National Guard Youth ChalleNGe Evaluation." AEA RCT Registry. January 13. https://doi.org/10.1257/rct.1385-1.0.
Former Citation
Bloom, Dan. 2017. "National Guard Youth ChalleNGe Evaluation." AEA RCT Registry. January 13. http://www.socialscienceregistry.org/trials/1385/history/13100.
Experimental Details
Interventions
Intervention(s)
The National Guard Youth ChalleNGe Program is an intensive residential program designed to “reclaim the lives of at-risk youth” who have dropped out of high school and give them the skills and values to succeed as adults. The program is open to young people between the ages of 16 and 18 who have dropped out of (or been expelled from) school, are unemployed, drug-free, and not heavily involved with the justice system. The 17-month program is divided into three phases: the Pre-ChalleNGe Phase (two weeks), the Residential Phase (20 weeks), and the Postresidential Phase (one year). During the first two phases (totaling 22 weeks), the participants live at the program site, often on a military base.

The first phase, Pre-ChalleNGe, is a physically and psychologically demanding assessment and orientation period. Candidates are introduced to the program’s rules and expectations; learn military bearing, discipline, and teamwork; and begin physical fitness training.

Candidates who complete Pre-ChalleNGe are formally enrolled in the program as “cadets” and move to the second phase. The curriculum for the 20-week Residential Phase is structured around eight core components that reflect current thinking about how to promote positive youth development: Leadership/Followership, Responsible Citizenship, Service to Community, Life-Coping Skills, Physical Fitness, Health and Hygiene, Job Skills, and Academic Excellence. Cadets spend the largest share of each day in the education component. At the time of the study, most programs helped participants prepare for the GED exam, but a few of them offered a high school diploma.

The program environment is described as “quasi-military”: The cadets are divided into platoons and squads, live in barracks, have their hair cut short, wear uniforms, and are subject to military-style discipline. The daily schedule is highly structured with almost no “down time,” and the cadets are closely supervised by staff at all times. While ChalleNGe uses military structure, discipline, facilities, and staff to accomplish its objectives, participation in the program is voluntary, and there are no requirements for military service during the program or afterward.
Toward the end of the Residential Phase, the cadets work with staff to arrange a post-residential “placement.” Acceptable placements include employment, education, and military service.

The cadets who successfully complete the Residential Phase move into the one-year Postresidential Phase, which involves a structured mentoring program. The ChalleNGe mentoring program is unusual, in that young people nominate their own mentors during the application process. ChalleNGe initiates the mentoring relationship partway through the Residential Phase, after the staff screen and train the mentors. The staff then maintain contact with both the program’s graduates and their mentors at least monthly during the Postresidential Phase to help solve problems and to report on the youths’ progress.
Intervention Start Date
2005-07-15
Intervention End Date
2008-12-31
Primary Outcomes
Primary Outcomes (end points)
Educational attainment (high school diploma, GED, college credits, current enrollment); employment and earnings (worked in last 12 months, currently working, average weekly earnings, military enlistment); crime and delinquency (arrests and convictions since random assignment).
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Ten state ChalleNGe programs participated in the study. The evaluation used a random assignment research design in which some of the youth who are eligible were placed, at random, into the ChalleNGe program, and others were placed into a comparison group that did not participate in ChalleNGe but had access to other services in the community.

To assess how the programs are implemented, the research team conducted interviews with program administrators, observed program activities, and reviewed documents on program operations. To understand participants’ characteristics, experiences, and outcomes, the team surveyed ChalleNGe and comparison group youth about 9, 21, and 36 months after they entered the study.
Experimental Design Details
Randomization Method
randomization in office by a computer
Randomization Unit
Individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
n/a
Sample size: planned number of observations
3,074 applicants
Sample size (or number of clusters) by treatment arms
754 individuals control, 2,320 ChalleNGe program
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Yes
Data Collection Completion Date
Final Sample Size: Number of Clusters (Unit of Randomization)
10 programs
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
3,074 individuals
Final Sample Size (or Number of Clusters) by Treatment Arms
754 individuals control, 2,320 ChalleNGe program
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
No
Reports, Papers & Other Materials
Relevant Paper(s)
Abstract
High school dropouts face an uphill battle in a labor market that increasingly rewards skills and postsecondary credentials: they are more likely than their peers to need public assistance, be arrested or incarcerated, and less likely to marry. This report presents results from a rigorous evaluation of the National Guard Youth ChalleNGe Program, an intensive residential program that aims to “reclaim the lives of at-risk youth” who have dropped out. More than 100,000 young people have completed the program since it was launched in the early 1990s. MDRC is conducting the evaluation in collaboration with the MacArthur Foundation Research Network on Transitions to Adulthood. The 17-month ChalleNGe program is divided into three phases: Pre-ChalleNGe, a two-week orientation and assessment period; a 20-week Residential Phase; and a one-year Postresidential Phase featuring a mentoring program. During the first two phases, participants live at the program site, often on a military base. The environment is “quasi-military,” though there are no requirements for military service. The evaluation uses a random assignment design. Because there were more qualified applicants than slots, a lottery-like process was used to decide which applicants were admitted to the program. Those who were admitted (the program group) are being compared over time with those who were not admitted (the control group); any significant differences that emerge between the groups can be attributed to ChalleNGe. About 3,000 young people entered the study in 10 ChalleNGe programs in 2005-2006.

Results

A comprehensive survey was administered to about 1,200 young people in the program and control groups an average of three years after they entered the study, when they were about 20 years old, on average. Key findings from the survey include:
•Members of the program group were much more likely than those in the control group to have obtained a General Educational Development (GED) certificate or a high school diploma and to have earned college credits.
•Members of the program group were more likely to be employed at the time of the survey, and they earned about 20 percent more than their control group counterparts in the year before the survey.
•There were few statistically significant differences between groups on measures of crime, delinquency, health, or lifestyle outcomes.

These results are impressive; few programs for dropouts have produced sustained positive impacts. And yet, both the survey and a series of in-depth telephone interviews with ChalleNGe graduates suggest that many young people struggled to maintain momentum after leaving the residential program and returning home, where they had relatively few supports and also faced unusually challenging labor market conditions. ChalleNGe may want to experiment with ways to bolster its postresidential services to provide more support during this difficult transition.
Citation
Millenky, Megan, Dan Bloom, Sara Muller-Ravett, and Joseph Broadus. 2011. Staying on Course: Three-Year Results of the National Guard Youth ChalleNGe Evaluation. New York: MDRC.
REPORTS & OTHER MATERIALS