Behavioral Interventions to Advance Self Sufficiency – Next Generation (BIAS-NG), Allegheny County Family Engagement

Last registered on February 05, 2021

Pre-Trial

Trial Information

General Information

Title
Behavioral Interventions to Advance Self Sufficiency – Next Generation (BIAS-NG), Allegheny County Family Engagement
RCT ID
AEARCTR-0007126
Initial registration date
February 04, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 05, 2021, 10:06 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
MDRC

Other Primary Investigator(s)

PI Affiliation
MDRC
PI Affiliation

Additional Trial Information

Status
On going
Start date
2018-08-01
End date
2022-03-15
Secondary IDs
Abstract
The Behavioral Interventions to Advance Self-Sufficiency-Next Generation (BIAS-NG) project, sponsored by the Office of Planning, Research, and Evaluation in the Administration for Children and Families (ACF), U.S. Department of Health and Human Services and led by MDRC in collaboration with its research partners MEF Associates, Child Trends, and Public Strategies Inc., aims to improve human services program design and outcomes through the application of behavioral science principles. BIAS-NG partners with state and local agencies to diagnose behavioral barriers to program success, design interventions to address those barriers, and test the efficacy and cost efficiency of those behaviorally informed interventions relative to status quo service delivery.
In Allegheny County, PA, BIAS-NG is working with The Office of Children, Youth and Families (CYF) in the Allegheny County Department of Human Services.
Child welfare agencies around the country have been taking steps to improve how they work with families and to promote greater engagement between families and caseworkers. However, there have been a limited number of randomized trials focused on family engagement once a family’s case has been accepted for service. For this test, the research team is working with a county-administered child welfare agency to test whether a behaviorally informed intervention, utilized early in the case planning process, can increase engagement of families in their cases. Cases are randomized to receive a standard notification letter only or that letter plus a supplemental flyer coupled with text messages. For the intervention cases, the flyer and text messages frame case opening as a “fresh start”; offer simplified information; clearly explain next steps; and include prompts and reminders for action. The trial tests whether initial information plus ongoing communication for up to eight weeks improves engagement of families and caseworkers within the first 60 days of case opening.
External Link(s)

Registration Citation

Citation
Balu, Rekha, Lawrence Katz and Clint Key. 2021. "Behavioral Interventions to Advance Self Sufficiency – Next Generation (BIAS-NG), Allegheny County Family Engagement." AEA RCT Registry. February 05. https://doi.org/10.1257/rct.7126-1.0
Experimental Details

Interventions

Intervention(s)
When a child welfare referral is accepted for service and a new case is opened in the county, the caseworker works with the family to complete a series of steps, culminating with the family plan. These steps typically must be completed within 60 days for cases in which the children have not been removed from the home (in-home) and 30 days for cases in which they have (removal). Family engagement during this period is critical; if the family is not home for a caseworker visit or does not attend a family conference, the family plan may not contain steps that are meaningful to resolving the family’s challenges and the family and caseworker may not agree on priority steps to make progress on the family’s case.
To design the intervention, the BIAS-NG team worked with the county to assess and understand the problem of family engagement using qualitative and quantitative data, and then identified process bottlenecks for families involved with the Allegheny County agency.
The research team’s primary goal is to increase engagement of families early in their case planning.
To this end, the two-stage intervention focused on the first one to two months of a newly established case. The first stage of the intervention involved mailing families a behaviorally informed orientation flyer explaining the milestones in the first 60 days for cases in which the child remains in the home, and 30 days for cases in which the child has been removed from the home. This flyer leveraged several concepts from behavioral science including the “fresh start” effect, simplification, and prompts to make a plan. The flyer capitalized on the transition from the case investigation to the case opening phase to communicate a fresh start for the family; the fresh start effect is conveyed on the flyer by establishing the case to be at “day 0.” The flyer simplified the information conveyed to families by clearly identifying four key steps that happen in the first 60 days. The flyer encouraged families to make concrete plans using a “roadmap” as a calendar to write important dates and events. The flyer accompanied a required mailing notifying families of case opening.
Stage two of the intervention was a series of informational messages and preparatory reminders to families related to key steps in their case. During the first weeks of the case, these behaviorally informed messages and reminders were sent via text message to clients, often coinciding with key milestones in their case. Messages included, but were not limited to:
• Introduction when case is accepted
• Information about next steps, plus links to short videos and relevant phone numbers for county help lines and case workers.
• Reminders for key steps, including contacting caseworkers
• Encouragement and reassurances that their experiences are common to other families with open cases.
Information about what will happen and reminders gave families the opportunity to plan for their case steps in partnership with their caseworker. They also distilled key information into concrete steps, simplifying a complex process on a week-by-week basis.
Intervention Start Date
2018-08-01
Intervention End Date
2020-03-31

Primary Outcomes

Primary Outcomes (end points)
A constructed measure for whether families engaged in early case milestones (binary)
Primary Outcomes (explanation)
The study will construct a measure that combines variables that correspond to three milestones listed in the flyer roadmap: a home visit, a family conference, and a family plan. The measure indicates whether two or more milestones were achieved (the reference category is none or one was achieved): whether a home visit was completed, a family conference was scheduled, and whether a family plan includes at least one outcome assigned to a birth parent.

For the family plan outcome, a data set of family conference attendees is matched to a demographic information data set using unique case-level and person-level identification numbers already present in both data sets. The demographic information file contains a field that identifies the biological father and the biological mother.

Secondary Outcomes

Secondary Outcomes (end points)
For confirmatory analysis, we will examine a distal outcome: Subsequent incident of child neglect and/or abuse with a valid/substantiated allegation, measured at 6 months after the case has opened (binary indicator, reverse coded).

For exploratory analysis, the study will analyze the following proximal outcomes:
i) a constructed measure that captures whether one of more milestones were achieved (the reference category is none were achieved).
ii) each of the three milestone variables as individual outcomes:
- completed home visit within 75 days of case opening (binary)
- family conference scheduled within 75 days of case opening (binary)
- family plan entered in case management system with at least one outcome assigned to a birth parent (binary)

Subgroup analysis
The study will examine differences in impact on the composite variable between subgroups: between case re-openings vs. cases completely new to CYF, between cases classified as child removal vs. non-removal at case opening, between cases with vs. without court involvement in the first month of case opening.
Secondary Outcomes (explanation)
For the family plan outcome, a data set of family conference attendees is matched to a demographic information data set using unique case-level and person-level identification numbers already present in both data sets. The demographic information file contains a field that identifies the biological father and the biological mother.

Experimental Design

Experimental Design
As new cases opened over the course of nearly 22 months, those cases were randomly assigned, within each of 5 regional offices, to receive either the usual case opening letter or that letter supplemented by two elements: i) a mailed informational flyer with an explanation of case steps, followed by ii) up to 8 weeks of text messages containing follow-up information, reminders, and tips about preparing for upcoming case steps.

For cases that opened in or after February 2020, randomization continued through the end of March but those cases may experience some COVID-related disruptions in typical services or the opportunity to complete certain steps.
Experimental Design Details
Randomization Method
Computerized randomization done by MDRC, with randomization of cases done within each of 5 office randomization blocks.
Randomization Unit
Randomization occurs at the level of the case ( not the individual, as multiple adults and/or children may be associated with a single case).
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Blocked within 5 regional offices
Multiple adults may be clustered within a single case, but the unit of assignment and analysis is case
Sample size: planned number of observations
2,150 cases
Sample size (or number of clusters) by treatment arms
1,125 treatment, 1,125 control cases
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
MDES = .099 standard deviation units (assuming an R-sq between 0- 0.1 from covariates without a pre-test variable, 5 randomization blocks, and a two-tailed test with significance level of 0.1). This would represent a roughly 5-6 percentage point effect, which would be modest to large for a behavioral mailing/text intervention when families are also contending with complicated familial relationships and situations with many structural factors outside of the control of the study. These factors include challenges such as housing instability, behavioral health issues, and other factors that contribute to a family’s involvement in the child welfare system but that cannot be addressed through this type of intervention.
IRB

Institutional Review Boards (IRBs)

IRB Name
MDRC IRB, New York, NY
IRB Approval Date
2018-03-16
IRB Approval Number
MDRC IRB #0003522

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials