Behavioral Interventions to Advance Self Sufficiency – Next Generation (BIAS-NG): Washington State, Temporary Assistance for Needy Families

Last registered on October 08, 2021

Pre-Trial

Trial Information

General Information

Title
Behavioral Interventions to Advance Self Sufficiency – Next Generation (BIAS-NG): Washington State, Temporary Assistance for Needy Families
RCT ID
AEARCTR-0008341
Initial registration date
October 07, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 08, 2021, 10:07 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
MDRC

Other Primary Investigator(s)

PI Affiliation
Harvard University

Additional Trial Information

Status
On going
Start date
2018-10-01
End date
2022-07-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The Behavioral Insights to Advance Self-Sufficiency-Next Generation (BIAS-NG) project, sponsored by the Office of Planning, Research, and Evaluation in the Administration for Children and Families (ACF), U.S. Department of Health and Human Services, uses principles of behavioral science in an effort to improve human services program design and outcomes. BIAS-NG partners with state and local agencies to diagnose behavioral barriers to program success, design interventions to address those barriers, and test the efficacy and cost efficiency of those behaviorally informed interventions relative to status quo service delivery. In Washington state, BIAS-NG worked with the Department of Social and Health Services (DSHS). DSHS administers Washington’s Temporary Assistance for Needy Families (TANF) program and the associated WorkFirst (WF) program.



Working closely with DSHS, BIAS-NG identified behavioral barriers that TANF clients face to successful participation in WF activities. DSHS provides a robust set of participation options to WF clients through other agencies with whom they contract to provide these WF activities. The WF activities are designed to help clients obtain jobs of interest (Job Search activity track), access education and skill training (Education and Training activity track), remove barriers to employment (Resolving Issues activity track), build language skills (Limited English Proficiency activity track), and gain work experience (Work Experience activity track). The goal of WF in the state of Washington is to support clients to become independent of public financial support, and participating in WF activities is viewed as key to achieving this financial independence.



However, approximately half of all TANF clients assigned to a WF activity never attend their first meeting or their WF Activity orientation, limiting their access to WF services that are hypothesized to increase clients’ likelihood of securing gainful employment and be in a position to no longer need TANF cash grants. Disengagement at this step in enrollment contributes to clients exiting TANF. Possible trajectories of clients who disengage after assignment to a WF activity are that they find employment on their own, or lose benefits for lack of participation. In both cases, clients may lose out on an opportunity to obtain new or strengthen existing skills that could help them to obtain and sustain employment of interest. This disengagement may also contribute to clients cycling back on to TANF within a short period.

To determine the assignment of an appropriate WF activity, WF Program Specialists conduct a Comprehensive Evaluation (CE) with clients in a one-on-one meeting to thoroughly assess clients’ resources and supports, aspirations, skills, and barriers to employment. At the culmination of the CE, WF Program Specialists prepare an Individual Responsibility Plan (IRP) which is a “living contract” through which WF Program Specialists build and update a customized plan for clients’ participation in WF activities based on their situation. Participation plans include the required number of participation hours in and referrals to WF activities providers, in addition to consequences of non-participation. Clients are required to make contact with their WF activity provider within 30 days of assignment and document hours of participation on a regular basis to continue to receive TANF benefits.



During the first phase of the BIAS-NG project, identified as the behavioral diagnosis and design phase, the research team became more knowledgeable about Washington state’s TANF WF program and enrollment processes, focusing attention on attrition between the referral to WF activity providers and participation in these WF activities. To conduct the investigation, the research team interviewed clients, staff, and administrators; observed interactions of interest (group DSHS orientation, CE between WF Program Specialists and clients, and WF activity orientation); and reviewed materials used in enrollment processes. The research team observed that the WF enrollment process was long and contained a lot of information, and that a primary message conveyed is that DSHS wants clients to obtain employment. Data showed the Job Search activity track was the most common referral and had low completion rates. Staff interviews suggest that Job Search is the default referral for people determined eligible to get a job. The hypothesized behavioral barriers based on these observations were that: 1) incoming clients receive and share a lot of important information at once, and therefore may not share all relevant information, or fully process and recall participation requirements and how to access available resources intended to support participation; and 2) the participation plan and planning process could be interpreted as having a contractual and transactional tone, which may not motivate clients to share all relevant information or participate in available activities.



BIAS-NG designed, implemented, and tested an intervention to address the behavioral barriers hypothesized to contribute to drop-off between assignment to WF activities and participation in those activities. The intervention asked WF Program Specialists to use a “Blueprint” planning card with clients that they would fill out during the individual assessment interview (i.e., the CE). The aim of the card was to provide clients with a place to 1) articulate and write down goals to accomplish in the WF program, and 2) to help them plan and identify the steps they would need to take to attend the assigned orientation for their WF activity. In addition, WF Program Specialists were provided a set of flashcards to help demonstrate each of the WF activity options. The intervention was designed to address behavioral bottlenecks in the enrollment process by more fully engaging TANF recipients during the CE in service of motivating them to attend the agreed-upon activity as delineated in their IRP. More participant engagement in the enrollment process might change the quality or quantity of information exchanged during the CE and outcome of the IRP. This exploratory hypothesis is measured by the percent of people referred to the Job Search activity track – considered the default referral for eligible applicants deemed by WFPS as work-ready—versus other activity tracks. The long-term goal was to increase the number of TANF clients who successfully exit TANF, measured by reduced “cycling” (i.e., churn) within 7-12 months of case opening.



For the impact evaluation, clients were randomly assigned to be served by staff trained to deliver the intervention (behavioral group) or assigned to staff delivering status quo services (standard group). In addition to the impact study analysis, BIAS-NG will conduct implementation and cost analyses to document how the intervention was delivered and at what cost.
External Link(s)

Registration Citation

Citation
Katz, Lawrence and Clinton Key. 2021. "Behavioral Interventions to Advance Self Sufficiency – Next Generation (BIAS-NG): Washington State, Temporary Assistance for Needy Families." AEA RCT Registry. October 08. https://doi.org/10.1257/rct.8341
Sponsors & Partners

Sponsors

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
The behavioral intervention attempted to address the problems identified in the diagnosis of behavioral barriers to program success by inserting behaviorally informed materials into the WF-TANF enrollment process. The intervention design aimed to make the connection more explicit between clients’ goals and how the WF activity options to which they are assigned could help them in accomplishing their goals, and in doing so, address their economic situations. The intervention also aimed to clarify participation expectations and facilitate planning to attend the WF Activity Orientation as a starting point to ongoing engagement.
Intervention Start Date
2018-10-01
Intervention End Date
2020-03-13

Primary Outcomes

Primary Outcomes (end points)
% of WF households containing clients participating in at least one activity component within 30 days of Random Assignment (RA).

% of WF households containing clients participating in at least one activity component within 180 days of RA.

% of WF households containing clients who enter sanctions within 180 days of RA.

% of WF households containing clients who return back on to TANF/State funded assistance (SFA) between 7 and 12 months after RA.
Primary Outcomes (explanation)
% of WF households containing clients participating in at least one activity component within 30 days of Random Assignment (RA).

The behavioral intervention is designed to increase participation within the first 30 days.

DSHS refers to WF activities as "activity components". DSHS WF Program Specialists enter a component code and assignment date in the DSHS electronic Jobs Automation System (eJAS) for every activity to which clients are assigned in their IRP. For most activities, when clients attend the corresponding orientation or complete the next step, WF partner staff update the component code with a "contractor" start date. In the case of the Job Search activity, the component assignment date indicates the initial meeting attendance. We will create a binary (yes/no) variable indicating attendance at any assigned activity within 30 days of RA. Attendance date is indicated by the component assignment date for Job Search activity and the "contractor" start date for all other activities. Exemptions from WF activities and being sanctioned for not attending WF activities are included as components in eJAS but will not be counted as activities in this measure.



% of WF households containing clients participating in at least one activity component within 180 days of RA.



The behavioral intervention is designed to increase participation within 180 days.



This will use the same logic as the first outcome, but covering within 180 days of RA. Exemptions and sanctioning will not be included in this measure.



% of WF households containing clients who enter sanctions within 180 days of RA.

The behavioral intervention is designed to increase ongoing participation measured by a reduction in entering sanctions within 180 days.

Sanctioning is indicated in eJAS with a component code and start date.

% of WF households containing clients who return back on to TANF/State Funded Assistance (SFA) between 7 and 12 months after RA.

The behavioral intervention is designed to increase the number of TANF/SFA clients who successfully exit TANF/SFA, measured by reducing returning to TANF/SFA within 7-12 months of RA.



Defined as households who left TANF/State Funded Assistance (SFA) on or after RA and then returned between 7 and 12 months after RA. 

Secondary Outcomes

Secondary Outcomes (end points)
% of WF households containing participants assigned to any WF activity not including Job Search within 180 days of RA (includes households with and without Job Search, as long as they also received a non-Job Search assignment).

Secondary Outcomes (explanation)
% of WF households containing participants assigned to any WF activity not including Job Search within 180 days of RA (includes households with and without Job Search, as long as they also received a non-Job Search assignment). The behavioral intervention is designed to engage participants more during the CE. To observe whether increased engagement meaningfully changed outcomes of the CE we explore whether there is a difference between groups in percent assigned to the default referral – Job Search – versus other activity tracks.



Assigned activities are indicated in eJAS with a component code and start date.

Experimental Design

Experimental Design
We used a one-stage random assignment design that leveraged the system DSHS offices currently use to manage workflow to assign clients to a staff person to conduct the CE. All WF Program Specialists are assigned specific hours during the day where they meet with clients by pulling the case file from a queue of people who have completed the orientation and are waiting to meet with a WF Program Specialist. The WF Program Specialists do not have a choice of which case they pull; as soon as they indicate they have finished the previous case, they are assigned the next case on the queue. Prior to the launch of the intervention, the research team randomly assigned staff to either deliver the intervention (the behavioral-serving staff) or to conduct business as usual (the standard-serving staff). The behavioral-serving staff were trained on how to deliver the intervention. The queue system mirrored random assignment in that clients who complete orientation and need a CE were randomly divided into two groups: (1) those who completed their Comprehensive Evaluation with staff who are trained to deliver the intervention components (i.e., the behavioral group); and (2) those who completed their Comprehensive Evaluation with staff who followed the status quo process (i.e., standard group). All applicants who were assigned to a WF Specialist and who began completing a CE were eligible for study enrollment.

Clients who had been previously randomly assigned continued to receive the intervention associated with their initial study group assignment condition. A small number of clients were later assigned a different study group condition than their original condition. For these clients, their initial study group condition and random assignment month is retained for the analysis.



Individuals in two-client households in the same Assistance Unit (AU) were assigned to the same study group condition. Since some households contained two adults who each completed the CE separately and were assigned to WorkFirst activities once both clients completed the CE, and because some two-adult households only contained one adult who was assigned to WorkFirst activities, we will assess outcomes at the household level to measure participation by at least one adult in the household.



For those individuals in the research sample not actively receiving TANF – for example, if either parent in the household did not pursue enrollment to TANF following the CE-- household data will be unavailable. These individuals will be treated as their own household for the measures.



Since WF activity exemptions are determined during the CE, we are unable to exclude clients who were exempt from WF activity participation in the CE. Thus, the sample will include clients who are “employment-ready,” clients with significant barriers to remove before they are employment-ready, and clients who are exempt from participating in WF activities altogether.
Experimental Design Details
Not available
Randomization Method
The point of random assignment is the moment when a given staff member is assigned a particular client through the queue system. If the WF Specialist is a trained, behavioral-serving staff member, the client becomes part of the behavioral group; if the WF Program Specialist is a standard-serving staff member, the client becomes part of the standard group.



This method leverages the current system of “assigning” clients to WF Program Specialists. In this existing system, all applicants who have completed their group WorkFirst orientation and are ready to complete their CE are entered into a set queue and WF Program Specialists who conduct CEs must work with the client at the top of that queue in a given office when they become available for a task. Staff cannot manipulate this assignment once they are tasked to complete the CE for a given client.



In order to reduce the risk that staff ability or other factors besides their randomly assigned service condition may affect impacts, we randomly assign WF Program Specialists at each participating office to either (a) deliver the intervention as designed throughout all CEs they conduct during the study period (i.e., behavioral-serving staff); or (b) continue delivering the CE as usual for all CEs they conduct during the study period (i.e., standard-serving staff).
Randomization Unit
Randomization occurred by WF Program Specialist, with analysis done at the household level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
44 WF Program Specialists.
Sample size: planned number of observations
4,000 households
Sample size (or number of clusters) by treatment arms
About 50% in each group, so, approximately 2,000 households in the behavioral group and 2,000 households in the standard group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Accounting for sample design and clustering, what is the minimum detectable effect size for main outcomes. Specify the unit, standard deviation, and percentage The formula to detect the minimum detectable effect size (MDES) in a cluster randomized trial is: MDES=M∗ρP(1−P)J+(1−ρ)P(1−P)Jn−−−−−−−−−−−−−−−−−−−−−−√ MDES=M∗𝜌P1−PJ+1−𝜌P1−PJn where: M is a multiplier determined by the desired levels of precision and power, ρ is the intraclass correlation coefficient, a ratio of the variance across WF Program Specialists to the variance across centers plus the variance across WF Program Specialists P is the proportion of the sample assigned to the behavioral group J is the total number of clusters n is the average number of households per WF Program Specialist M is derived from calculations for 80% power and 10% level of significance, with the two research groups of equal size. ρ is set at .05, a conservative estimate. There are 44 WF Program Specialists which corresponds to, J, the total number of clusters. And n is derived by dividing the 4,000 households by the 44 clusters, for 91 households per WF Program Specialists. Multiplying the MDES by the standard deviation for each primary outcome results in the following minimum detectable effects (MDEs): Primary Outcome Base rate MDES Standard Dev MDE Participation within 30 days 0.40 0.188 0.49 0.09 Participation within 180 days 0.45 0.188 0.50 0.09 Sanctioned within 180 days 0.16 0.188 0.37 0.07 Returned to TANF within 7-12 months 0.10 0.188 0.29 0.06
IRB

Institutional Review Boards (IRBs)

IRB Name
MDRC IRB
IRB Approval Date
2018-07-19
IRB Approval Number
MDRC IRB #0003522
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information