Supporting Evidence Building in Child Welfare: The Evaluation of Project Connect

Last registered on April 26, 2021
View Trial History

Pre-Trial

Trial Information

General Information

Title
Supporting Evidence Building in Child Welfare: The Evaluation of Project Connect
RCT ID
AEARCTR-0007273
Initial registration date
April 23, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 26, 2021, 10:41 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Catherine Kuhns
Affiliation
The Urban Institute
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Mark Courtney
PI Affiliation
University of Chicago
Contact Investigator
PI Name
Michael Pergamit
PI Affiliation
Urban Institute
Contact Investigator

Additional Trial Information

Status
In development
Start date
2021-04-01
End date
2025-12-31
Keywords
Behavior
Additional Keywords
child welfare, substance use
JEL code(s)
Secondary IDs
HHSP233201500064I_HHSP23337014T
Abstract
Project Connect is a comprehensive home visitation intervention that targets child welfare-involved, substance-affected families with children and adolescents ages 0 to 17 through home-based services and treatment. Project Connect addresses the complex needs of substance-affected families by providing intensive, long-term services aimed at strengthening families by addressing issues of unhealthy parental substance use and helping parents recover while keeping children safe. The program focuses on maintaining children safely in their homes (preventing admission to care) or facilitating reunification when children have been placed in out-of-home care.

The goal of the evaluation is to determine whether Project Connect increased reunification for families receiving Project Connect services and document how the program is implemented in Rhode Island. This evaluation has two main components, an implementation study and an impact study. The impact study examines the effect that the program had on outcomes, using quantitative data collection and analysis. The impact study will employ a randomized controlled trial (RCT). The evaluation will randomly assign families to be referred either to receive Project Connect services or to receive services as usual. The study will follow all randomized families in the evaluation for one year after the completion of treatment. The implementation study examines how the program is executed, the policies and practices of the state child welfare agency and the context of substance use in the state of Rhode Island.
External Link(s)

Registration Citation

Citation
Courtney, Mark, Catherine Kuhns and Michael Pergamit. 2021. "Supporting Evidence Building in Child Welfare: The Evaluation of Project Connect." AEA RCT Registry. April 26. https://doi.org/10.1257/rct.7273-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This study is an RCT evaluation of Project Connect, a program that services substance-affected families with children ages 0-17 in the child welfare system. The proposed evaluation of the Project Connect program will follow a sample of families randomized to be offered the Project Connect program or, alternatively, services as usual. This evaluation will study program impacts as well as program implementation, and will draw on qualitative and quantitative data collection and analysis. The impact study will examine whether Project Connect has significant impacts on child welfare outcomes in Rhode Island, specifically reduced removals. The implementation study will provide the project team with key pieces of information related to the implementation of Project Connect needed to inform the development and management of the impact study.
Intervention Start Date
2021-04-01
Intervention End Date
2023-04-01

Primary Outcomes

Primary Outcomes (end points)
1. Child reunifications with family from out-of-home care
2.Time to reunification
Primary Outcomes (explanation)
Child reunification will be measured as the probability that a child who is out of the home at the time of randomization is reunited with their parents within 1 year and within 2 years of randomization. We will measure time to reunification as the number of days from randomization to reunification. Primary outcomes will be measured from child welfare administrative data.

Secondary Outcomes

Secondary Outcomes (end points)
1. Reunification for all children in a family
2. Child removals into out-of-home care
3. Child Permanency
Secondary Outcomes (explanation)
For cases where at least one sibling of the focal child has a case open, reunification for all children in a given family will be measured as the probability that all children who are out of the home at the time of randomization are reunited with their parents within 1 year and within 2 years of randomization. Child removals will be measured as the probability that a child who is in the home at the time of randomization has been removed at all at 1 year and 2 years post randomization. Child permanency will be measured as the likelihood that a child is not in foster care and is living in a permanent family setting, which may include living with parents, living with kin, adoption, or other permanent situation.

Experimental Design

Experimental Design
The evaluation will employ a family level randomized controlled trial (RCT) in 1 site. The evaluation site has a target population for Project Connect that includes many more families that are eligible for the program than can be accommodated by the program’s limited number of available service slots. As such, the evaluation will allocate the limited program slots by randomly assigning families to be referred either to receive Project Connect or services as usual.
Experimental Design Details
Not available
Randomization Method
The specific referral and randomization strategy leverages an existing referral form that caseworkers complete and send to a central referral unit (CRU) at the public child welfare agency. At the CRU, a staff member reviews each referral form and randomizes all eligible families into either treatment or control. The staff member uses a customized online randomization tool designed by the Urban Institute. The randomization tool collects the family’s identification number, whether they are preservation or reunification, and their referral form and determines whether a family is to be referred to Project Connect or services as usual. The tool uses a one-to-one randomization ratio, with one family randomized to treatment for every control family.
Randomization Unit
Randomization takes place at the household (family) level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
The total sample size is 196 "removal" families (That is, families who, at the time of random assignment, have the target child living outside the home)
Sample size (or number of clusters) by treatment arms
-98 "removal" families in the control group
-98 "removal" families in the treatment group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
To determine the appropriate sample size, we estimated Minimum Detectable Effect (MDE) sizes for Intent-to-Treat (ITT) estimates. We calculated the MDE as the impact divided by the comparison group standard deviation (sqrt(m(1-m)), where m = the proportion of the control group with a “successful” outcome (i.e., reunification). The MDE was calculated assuming a randomization ratio of one treatment (Project Connect) to one control (services as usual) with an alpha level of 0.05anda two-tailed test with 80 percent power. Based on the data from Project Connect and DCYF, Project Connect children were 18 percentage points more likely than comparison group children to be at home by June 30, 2019. This represents a standardized effect size of.40. Under an intent-to-treat (ITT) estimation strategy, where outcomes for all people referred to Project Connect are included, we estimate that to achieve this minimum detectable effect, a sample of roughly 196participants would be necessary, 98 treatment and 98 control.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Urban Institute IRB
IRB Approval Date
2021-04-14
IRB Approval Number
0189
Analysis Plan