|
Field
Abstract
|
Before
Project Connect is a comprehensive home visitation intervention that targets child welfare-involved, substance-affected families with children and adolescents ages 0 to 17 through home-based services and treatment. Project Connect addresses the complex needs of substance-affected families by providing intensive, long-term services aimed at strengthening families by addressing issues of unhealthy parental substance use and helping parents recover while keeping children safe. The program focuses on maintaining children safely in their homes (preventing admission to care) or facilitating reunification when children have been placed in out-of-home care.
The goal of the evaluation is to determine whether Project Connect increased reunification for families receiving Project Connect services and document how the program is implemented in Rhode Island. This evaluation has two main components, an implementation study and an impact study. The impact study examines the effect that the program had on outcomes, using quantitative data collection and analysis. The impact study will employ a randomized controlled trial (RCT). The evaluation will randomly assign families to be referred either to receive Project Connect services or to receive services as usual. The study will follow all randomized families in the evaluation for one year after the completion of treatment. The implementation study examines how the program is executed, the policies and practices of the state child welfare agency and the context of substance use in the state of Rhode Island.
|
After
Project Connect is a comprehensive home visitation intervention that targets child welfare-involved, substance-affected families with children and adolescents ages 0 to 17 through home-based services and treatment. The program addresses the complex needs of families affected by substance use by providing intensive, long-term services aimed at strengthening families, addressing parental substance use, and helping parents recover while keeping children safe.
The study used a randomized controlled trial (RCT) to evaluate the causal impact of Project Connect services on families experiencing substance use in the Rhode Island child welfare system and to compare the child welfare service trajectories (e.g., placement and length of time in care) of families affected by substance use who received Project Connect services with those who received services as usual. We also collected qualitative data, which included the interviews and focus groups, as part of an implementation study. Families affected by substance use with an open child protective services case in Rhode Island were randomly referred for substance use services – either to Project Connect or to other usual services. The final full study sample includes 230 families, 192 families whose children were in out-of-home care at the start of the study (96 treatment and 96 control) and 34 families who children were in home at the start of the study (17 treatment and 17 control). The majority of our analyses will focus on the 192 “reunification” families. We will examine 24-months post randomization outcomes. If feasible, we will also examine 30-month post randomization outcomes.
|
|
Field
Trial End Date
|
Before
December 31, 2025
|
After
March 31, 2027
|
|
Field
Last Published
|
Before
April 26, 2021 10:41 AM
|
After
April 25, 2025 10:23 AM
|
|
Field
Intervention (Public)
|
Before
This study is an RCT evaluation of Project Connect, a program that services substance-affected families with children ages 0-17 in the child welfare system. The proposed evaluation of the Project Connect program will follow a sample of families randomized to be offered the Project Connect program or, alternatively, services as usual. This evaluation will study program impacts as well as program implementation, and will draw on qualitative and quantitative data collection and analysis. The impact study will examine whether Project Connect has significant impacts on child welfare outcomes in Rhode Island, specifically reduced removals. The implementation study will provide the project team with key pieces of information related to the implementation of Project Connect needed to inform the development and management of the impact study.
|
After
Project Connect is a home-based intervention for families involved in the child welfare system affected by substance use. Eligible families have children or adolescents ages 0 to 17. The program aims to maintain children safely in their homes (preventing admission to out-of-home care) or to facilitate reunification when children have been placed out-of-home. The program’s theory of change is that child safety, well-being, and permanency are achieved when parents have resources that support recovery, enhance parenting capacity, strengthen family relationships and functioning, and improve families’ living conditions. Project Connect works with families to build those resources through substance use assessment; individual and family counseling; parent education; pediatric nursing assessment and services; relapse prevention planning; assistance with transportation; court advocacy; supplemental visitations; 24/7 emergency on call support; domestic violence support; and aftercare services following DCYF involvement. Project Connect also links families with community resources, including substance abuse treatment agencies; child care; health care; housing; mental health; legal services; and medical services. The Project Connect model emphasizes deep and trusting relationships with caregivers as a key to connecting families to resources and achieving intended outcomes.
|
|
Field
Intervention End Date
|
Before
April 01, 2023
|
After
June 28, 2024
|
|
Field
Primary Outcomes (End Points)
|
Before
1. Child reunifications with family from out-of-home care
2.Time to reunification
|
After
1. Child reunifications with family from out-of-home care
2. Child reunifications with family from out-of-home care with no placement disruptions within the prior 6 months
|
|
Field
Primary Outcomes (Explanation)
|
Before
Child reunification will be measured as the probability that a child who is out of the home at the time of randomization is reunited with their parents within 1 year and within 2 years of randomization. We will measure time to reunification as the number of days from randomization to reunification. Primary outcomes will be measured from child welfare administrative data.
|
After
Child reunification will be measured as the probability that a child who is out of the home at the time of randomization is reunited with their parents. We will examine 1) the share of children reunified at 24 months post randomization; and 2) the share of children reunified at 24 months post randomization with no placement disruptions within the prior 6 months. The latter measure represents reunification stability. We also note that higher rates of stable reunifications imply lower rates of reported abuse and neglect.
|
|
Field
Experimental Design (Public)
|
Before
The evaluation will employ a family level randomized controlled trial (RCT) in 1 site. The evaluation site has a target population for Project Connect that includes many more families that are eligible for the program than can be accommodated by the program’s limited number of available service slots. As such, the evaluation will allocate the limited program slots by randomly assigning families to be referred either to receive Project Connect or services as usual.
|
After
The evaluation employed a family level RCT in one site. Randomization occurred between April 2021 and June 2023.Our randomization protocol had caseworkers in the public child welfare agency refer families requiring substance use services to the agency's Central Referral Unit (CRU). The CRU stratified the referrals separating preservation families (i.e., children at-home) from reunification families (i.e., children in out-of-home care). The CRU manager entered information into Urban’s online randomization tool, which provided her the study group assignment. The CRU manager randomized in pairs so the treatment and control groups were of equal size with roughly comparable referral dates. Under our randomization protocol, all families flagged by their caseworker as having a substance use concern and subsequently referred to the agency's CRU were eligible for the study, and randomized.
|
|
Field
Randomization Method
|
Before
The specific referral and randomization strategy leverages an existing referral form that caseworkers complete and send to a central referral unit (CRU) at the public child welfare agency. At the CRU, a staff member reviews each referral form and randomizes all eligible families into either treatment or control. The staff member uses a customized online randomization tool designed by the Urban Institute. The randomization tool collects the family’s identification number, whether they are preservation or reunification, and their referral form and determines whether a family is to be referred to Project Connect or services as usual. The tool uses a one-to-one randomization ratio, with one family randomized to treatment for every control family.
|
After
The specific referral and randomization strategy leveraged an existing referral form that caseworkers completed and sent to a central referral unit (CRU) at the public child welfare agency. At the CRU, a staff member reviewed each referral form and randomized all eligible families into either treatment or control. The staff member used a customized online randomization tool designed by the Urban Institute. The randomization tool collected the family’s identification number, whether they were a preservation or reunification family, and their referral form and determined whether a family would be referred to Project Connect or services as usual. The tool used a one-to-one randomization ratio, with one family randomized to treatment for every control family.
|
|
Field
Randomization Unit
|
Before
Randomization takes place at the household (family) level.
|
After
Randomization took place at the household (family) level.
|
|
Field
Planned Number of Observations
|
Before
The total sample size is 196 "removal" families (That is, families who, at the time of random assignment, have the target child living outside the home)
|
After
The full study sample is 226 families.
The majority of our planned research questions focus only on reunification families – that is, the 192 families whose children were in out-of-home care at the start of the study (96 treatment and 96 control)
The study sample also includes 34 families who had an open child welfare case but whose children remained at home at the start of the study (i.e., preservation families). These families are included in one analysis as part of the list of secondary outcomes.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
-98 "removal" families in the control group
-98 "removal" families in the treatment group
|
After
-96 "reunification" families in the control group
-96 "reunification" families in the treatment group
-17 “preservation” families in the control group
-17 “preservation" families in the treatment group
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
To determine the appropriate sample size, we estimated Minimum Detectable Effect (MDE) sizes for Intent-to-Treat (ITT) estimates. We calculated the MDE as the impact divided by the comparison group standard deviation (sqrt(m(1-m)), where m = the proportion of the control group with a “successful” outcome (i.e., reunification). The MDE was calculated assuming a randomization ratio of one treatment (Project Connect) to one control (services as usual) with an alpha level of 0.05anda two-tailed test with 80 percent power. Based on the data from Project Connect and DCYF, Project Connect children were 18 percentage points more likely than comparison group children to be at home by June 30, 2019. This represents a standardized effect size of.40. Under an intent-to-treat (ITT) estimation strategy, where outcomes for all people referred to Project Connect are included, we estimate that to achieve this minimum detectable effect, a sample of roughly 196participants would be necessary, 98 treatment and 98 control.
|
After
We estimated Minimum Detectable Effect (MDE) sizes for Intent-to-Treat (ITT) estimates. We calculated the MDE as the impact divided by the comparison group standard deviation (sqrt(m(1-m)), where m = the proportion of the control group with a “successful” outcome (i.e., reunification). The MDE was calculated assuming a randomization ratio of one treatment (Project Connect) to one control (services as usual) with an alpha level of 0.05, a two-tailed test with 80 percent power, and a sample size of 192. This produced a minimum detectable effect size of 0.40.
|
|
Field
Keyword(s)
|
Before
Behavior
|
After
Behavior
|
|
Field
Secondary Outcomes (End Points)
|
Before
1. Reunification for all children in a family
2. Child removals into out-of-home care
3. Child Permanency
|
After
1. Differences in the share of children whose caregivers have a re-report of abuse or neglect (reunification families only)
2. Differences in the speed of reunification (reunification families only)
3. Among combined reunification and preservation families at 24 months post randomization, the share of children in a stable in-home placement for the prior 6 months
|
|
Field
Secondary Outcomes (Explanation)
|
Before
For cases where at least one sibling of the focal child has a case open, reunification for all children in a given family will be measured as the probability that all children who are out of the home at the time of randomization are reunited with their parents within 1 year and within 2 years of randomization. Child removals will be measured as the probability that a child who is in the home at the time of randomization has been removed at all at 1 year and 2 years post randomization. Child permanency will be measured as the likelihood that a child is not in foster care and is living in a permanent family setting, which may include living with parents, living with kin, adoption, or other permanent situation.
|
After
A stable, in-home placement will be measured as the likelihood that a child is not in foster care and is living in a permanent family setting, which may include living with parents, living with kin, adoption, or other permanent situation. The full study sample is 226 families, which includes 34 families who had an open child welfare case but whose children remained at home at the start of the study (i.e., preservation families). The last secondary outcome listed above will be an analysis of this full sample.
|
|
Field
Building on Existing Work
|
Before
|
After
No
|