Supporting Evidence Building in Child Welfare Project – LifeSet Evaluation

Last registered on August 02, 2021

Pre-Trial

Trial Information

General Information

Title
Supporting Evidence Building in Child Welfare Project – LifeSet Evaluation
RCT ID
AEARCTR-0008015
Initial registration date
July 30, 2021
Last updated
August 02, 2021, 2:00 PM EDT

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
Urban Institute

Other Primary Investigator(s)

PI Affiliation
University of Chicago

Additional Trial Information

Status
In development
Start date
2021-08-02
End date
2026-12-31
Secondary IDs
Abstract
The evaluation of the LifeSet program will follow a sample of youth randomized to be offered the LifeSet program or, alternatively, services as usual. The LifeSet program focuses on supporting youth in successful transitions to independent living by providing supports and services in multiple domains of independent living. The program is structured around a core relationship between each youth and their assigned “LifeSet specialist” who works intensively with the youth throughout his or her time in the program.

This evaluation will study both program impacts as well as program implementation, and will draw on qualitative and quantitative data collection and analysis techniques. The goal of the evaluation is to determine whether LifeSet has significant impacts on key youth outcomes in New Jersey and will address the core evaluation question: does LifeSet have a causal impact on the well-being and successful transition to adulthood of youth in the target population? The primary research questions for the impact study are does LifeSet improve outcomes of target population youth in the domains of:

a. Connections to education and employment
b. Social connections
c. Housing stability
d. Youth well-being (including resilience and social-emotional competence)

The eligible population for LifeSet will include youth 17-21 years old who have an open New Jersey public child welfare agency (DCF) case, are in the custody or guardianship of DCF, are living in the community (i.e. not incarcerated, in residential treatment, or hospitalized), and do not have a disqualifying condition (e.g., serious violent criminal history, residence out of state, severe mental illness, or intellectual disability).

The target population for the LifeSet program includes many more individuals who are in need of, and are eligible for, the intervention than can be accommodated by the program’s limited slots. The evaluation will therefore allocate the limited program slots by lottery, which is a fair way to allocate scarce resources. Identification and randomization of target-age youth will be done by New Jersey DCF based on administrative records and information gathered from youths’ case workers. To prevent one LifeSet provider from randomly having all youth in their catchment area assigned to only one group, randomization will be stratified by provider catchment area using a 1T:1C ratio. We estimate that the program will be able to serve approximately 300 study-enrolled youth during a two-year intake period. With a 1:1 randomization ratio (1 treatment to 1 control, or 1T:1C), the evaluation design will provide us with a sample of 600 youth.

Registration Citation

Citation
Courtney, Mark and Michael Pergamit. 2021. "Supporting Evidence Building in Child Welfare Project – LifeSet Evaluation." AEA RCT Registry. August 02. https://doi.org/10.1257/rct.8015-1.0
Sponsors & Partners

Sponsors

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
The LifeSet program is operated by Youth Villages, a private nonprofit organization. Formerly known as Transitional Living, LifeSet provides at-risk youth and young adults leaving foster care, juvenile justice, and mental health systems with the intensive in-home support and guidance they need to make a successful transition to adulthood. The core of the program is a therapeutic case management relationship between participant youth and “LifeSet specialists,” who work with youth to develop and move towards youth-defined goals in multiple domains of independent living including education, housing, employment and financial security, health and safety, and social connections and support. Youth Villages is currently operating in seven states in the US—Georgia, Massachusetts, Mississippi, North Carolina, Oklahoma, Oregon, and Tennessee—and the organization trains and supports nonprofit partners to offer LifeSet in other communities in six additional states. The CWEST project is evaluating LifeSet in New Jersey, which will be delivered by four local providers in the state.

When a young person enters the LifeSet program, the LifeSet specialist works with the youth to develop an individualized plan based on the youth’s personal context and goals. After the initial assessment, the youth and the LifeSet specialist meet at least weekly for one-hour, face-to-face or virtual sessions. During these sessions, the LifeSet specialist provides clinically focused case management, support, and counseling aimed at helping the youth move towards his or her goals. These meetings occur in a community setting, either at the youth’s home, work, or at whatever other location is most comfortable and convenient for him or her. In addition to these regular, scheduled meetings, LifeSet assures that youth are able to get in touch with a LifeSet specialist 24 hours a day, 7 days a week. The maximum duration youth can receive LifeSet services in New Jersey is 12 months.

LifeSet specialists work with between eight and ten youth at any one time and have either a master’s degree in a relevant field, or a bachelor’s degree and relevant work experience. Team supervisors supervise four LifeSet staff, including conducting group supervision, individual and field supervision, and professional development, and have a master’s degree in a relevant field, or a bachelor’s degree with relevant work experience. Licensed Program Experts are responsible for ensuring model fidelity and ideally have LifeSet implementation experience at the specialist and supervisor level and hold a master’s degree and professional license in a social service field. Licensed Program Experts meet regularly with each LifeSet team to provide clinical supervision and guidance on the program model.
Intervention Start Date
2021-08-16
Intervention End Date
2024-08-31

Primary Outcomes

Primary Outcomes (end points)
The primary research questions for the impact study are does LifeSet improve outcomes of target population youth in the domains of:
a. Connections to education and employment
b. Social connections
c. Housing stability
d. Youth well-being (including resilience and social-emotional competence)
Primary Outcomes (explanation)
Except where noted, survey items used to assess outcomes are generally taken from existing surveys with similar populations including the Midwest Study of the Adult Functioning of Former Foster Youth (Courtney et al., 2011), California Youth Transitions to Adulthood Study (CalYOUTH, Courtney et al., 2020), and the National Longitudinal Survey of Youth (NLSY, Bureau of Labor Statistics, 2013).
a. Connections to education and employment. The outcome will be measured as whether youth in the treatment group had a higher likelihood of (1) receiving a high school diploma or GED certificate, if not already received at baseline; (2) enrollment in or completion of postsecondary education or training; and (3) of being employed within 1 year and 2 years of randomization. Education outcomes will be measured through participants’ responses to a series of survey items and through administrative data from the National Student Clearinghouse. Employment outcomes will be measured through participants’ responses to a series of survey items and through administrative data from state wage records.
b. Social connections. The outcome will be measured as whether youth in the treatment group had a higher sense of social connection and support. This outcome will be measured through participants’ responses to a series of survey items from the Social Support Network Questionnaire (SSNQ) (Gee & Rhodes, 2007) and through data collected by the local LifeSet Provider on youths’ social connections at program exit.
c. Housing stability. The outcome will be measured as whether youth in the treatment group had a lower likelihood of (1) being housing unstable; (2) staying in an emergency homeless shelter at any time point; and (3) being placed in a transitional living arrangement. This outcome will be measured through participants’ responses to a series of survey items and through administrative data from the state child welfare agency and local Homeless Management Information System (HMIS) provider.
d. Youth well-being. The outcome will be measured as whether youth in the treatment group had higher-level feelings of resiliency and social-emotional competence. This outcome will be measured through participants’ responses to a series of survey items from the Youth Thrive survey (Center for the Study of Social Policy, 2020).

Secondary Outcomes

Secondary Outcomes (end points)
The secondary research questions for the impact study are does LifeSet improve outcomes of target population youth in the domains of:
a. Mental health
b. Contact with the criminal justice system
c. Intimate partner violence
d. Economic well-being
Secondary Outcomes (explanation)
a. Mental health. The outcome will be measured as whether youth in the treatment group experienced with less frequency (1) symptoms of depression, anxiety, and stress; (2) symptoms of Post-Traumatic Stress Disorder (PTSD); and (3) psychiatric hospitalization. This outcome will be measured through participants’ responses to a series of survey items, including items from the Depression Anxiety Stress Scale-21 (DASS21) (Lovibond & Lovibond, 1995) and Primary Care PTSD Screen for DSM-5 (PC-PTSD-5) (Prins et al., 2015).
b. Contact with the criminal justice system. The outcome will be measured as whether youth in the treatment group were less likely to (1) be arrested; (2) be convicted of a crime; (3) spend at least one night in a correctional facility; and (4) be sentenced to probation or supervision. This outcome will be measured through participants’ responses to a series of survey items and through administrative data from state criminal history and court records.
c. Intimate partner violence. The outcome will be measured as whether youth in the treatment group were less likely to be victims or perpetrators of intimate partner violence. This outcome will be measured through participants’ responses to survey items from the Conflict Tactic Scale (CTS) (Straus et al., 1996).
d. Economic well-being. The outcome will be measured as whether youth in the treatment group were (1) less likely to experience economic hardships and food insecurity; (2) less likely to receive public benefits; (3) more likely to have a checking or savings account; and (4) more likely to have a higher amount of money in their checking or savings account. This outcome will be measured through participants’ responses to a series of survey items, including items from the USDA Food Security Survey (Carlson, Andrews, & Bickel, 1999) and administrative data from the state public benefits agency.

Experimental Design

Experimental Design
The LifeSet evaluation will employ experimental methods, i.e. a randomized controlled trial (RCT). The eligible population for LifeSet will include youth 17-21 who have an open DCF case, are in the custody or guardianship of DCF, are living in the community (i.e. not incarcerated, in residential treatment, or hospitalized), and do not have a disqualifying condition (e.g., serious violent criminal history, residence out of state, severe mental illness, or intellectual disability). The four local LifeSet providers serve the Southern, Northern, and Central Regions, covering 18 of New Jersey’s 21 counties.

We estimate that the LifeSet program will be able to serve approximately 300 study-enrolled youth during the two-year intake period. Given a 1:1 ratio of assignment to treatment and control group, the study anticipates recruiting a total of 600 youth.

Identification and randomization of target-age youth will be done by New Jersey DCF based on administrative records and information gathered from youths’ case workers. This process will be conducted by age cohort, starting with youth who are at least 19 years-old and moving to younger ages in successive monthly randomization rounds. Youth will be randomly assigned to LifeSet or services as usual prior to obtaining consent/assent for youths’ participation in evaluation data collection. New Jersey DCF will then refer youth in the treatment group to a LifeSet provider; youth in the control group will be referred to, or continue receiving, services as usual.
Experimental Design Details
Not available
Randomization Method
Identification of target-age youth will be done by New Jersey DCF based on administrative records and information gathered from youths’ case workers. New Jersey DCF will use its administrative data to identify youth who meet the age criteria for LifeSet and are not excluded due to other criteria identifiable in DCF administrative data. This list will be sorted first by region and then in descending order of youths’ age in days. The DCF Research Coordinator will conduct a secondary screening by contacting the caseworkers for youth deemed eligible based on administrative records to obtain caseworker confirmation that the youth does not meet any exclusion criteria.

New Jersey DCF will send the list of eligible youth and the number of study slots needed per region to the Urban Institute project team who will perform the random assignment. New Jersey DCF will send this list via CSV file to the Urban Institute research team. Urban will conduct the random assignment in pairs per region using STATA. Urban will then send the list with group assignments added back to DCF.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
NA
Sample size: planned number of observations
The study anticipates recruiting a total of 600 youth
Sample size (or number of clusters) by treatment arms
The study estimates including 300 youth in the treatment group and 300 youth in the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For the power calculation, we assume 300 participants in the treatment group and 300 participants in the control group. An R2 of 0.30 is assumed, as a conservative estimate based on a prior evaluation of LifeSet. We also assume that administrative data will be available for the full sample and that the survey response rate will be 85 percent. Minimum Detectable Effects (MDE)s were calculated for an alpha level of 0.10 or 0.05, two-tailed test with 80 percent power. For outcomes based on administrative data, we estimated MDEs of 0.170 and 0.192 for alpha levels of 0.10 or 0.05, respectively. For outcomes based on survey data, we estimated MDEs of 0.185 and 0.208, given estimated attrition.
IRB

Institutional Review Boards (IRBs)

IRB Name
Urban Institute
IRB Approval Date
2021-07-14
IRB Approval Number
0189