Ticket to Work Notice Optimization project

Last registered on September 11, 2020
View Trial History

Pre-Trial

Trial Information

General Information

Title
Ticket to Work Notice Optimization project
RCT ID
AEARCTR-0006380
Initial registration date
September 10, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 11, 2020, 12:00 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
Sebastian Jilke
Affiliation
Office of Evaluation Sciences / General Services Administration
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Rekha Balu
PI Affiliation
Office of Evaluation Sciences / General Services Administration
Contact Investigator
PI Name
Elizabeth Bell
PI Affiliation
Office of Evaluation Sciences / General Services Administration
Contact Investigator
PI Name
Jeffrey Hemmeter
PI Affiliation
Social Security Administration
Contact Investigator
PI Name
Onika Richardson
PI Affiliation
Social Security Administration
Contact Investigator
PI Name
Elana Safran
PI Affiliation
Office of Evaluation Sciences / General Services Administration
Contact Investigator

Additional Trial Information

Status
In development
Start date
2020-09-16
End date
2022-12-23
Keywords
Behavior, Labor, Welfare
Additional Keywords
Social Policy, Disability
JEL code(s)
Secondary IDs
Abstract
The Ticket to Work (TTW) program provides Social Security Disability Insurance (SSDI) beneficiaries and Supplemental Security Income (SSI) recipients with disabilities with choices for receiving employment services while providing incentives to providers to serve these individuals. The goal of the TTW legislation was to expand the availability of employment services for beneficiaries beyond the existing state vocational rehabilitation (VR) agencies, reducing dependency on disability benefits. The TTW Notice Optimization project seeks to increase participation in the TTW program by developing an evidence-based approach to targeting outreach to eligible beneficiaries. The project will test changes to Ticket notices including the types of notices sent and the language used. The project aims to increase the number of beneficiaries who “assign a Ticket” with an Employment Network (EN) or VR agency. The assignment of a Ticket constitutes an agreement between the beneficiary and a service provider of their choice for employment support services. More intermediate goals of the project include an increase in initial calls to the TTW Help Line. The intervention is tested in a 2 x 2 factorial design where the re-designed letter constitutes factor #1 and an added cardstock Ticket factor #2. The intervention will run for 18 months, starting September 16, 2020, and outcomes will be measured at 9 months post-intervention for each individual.
External Link(s)

Registration Citation

Citation
Balu, Rekha et al. 2020. "Ticket to Work Notice Optimization project." AEA RCT Registry. September 11. https://doi.org/10.1257/rct.6380
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
CURRENT PRACTICE: To notify beneficiaries of their eligibility to participate in the TTW program, SSA currently mails notices to beneficiaries approximately two months after award (start-up notices) and at the 12-month and 36-month anniversary dates of their award. The notices include standard language about the TTW program and include a TTW brochure, titled “Your Ticket to Work” (available in English and Spanish), and an “inline, paper Ticket.” The inline Ticket is currently an additional page following the letter with additional TTW program information.
The intervention consists of updated communication materials that SSA sends to people with disabilities (SSDI beneficiaries and/or SSI recipients) between the ages of 18 and 55.
INTERVENTION #1 (Notice Structure): First, the project will modify both the start-up and one-year notices and eliminate the three-year letter notice. In place of the three-year notice, we are introducing a notice at two years post-award. The newly designed notices include a more succinct description of the program, which emphasizes saliency, and clear, plain-language action steps encouraging beneficiaries to call the TTW Help Line.
INTERVENTION #2 (Hardcopy Ticket): In addition to testing the amended notices, we will test the inclusion of a cardstock Ticket. These “Tickets” will be printed on 3.5” by 8” cardstock paper. The goal of the newly designed cardstock Tickets is to make the benefits of participating in the TTW program more salient and concise than the letter and, specifically, to help people think of themselves as ready to work (identity priming) and take the specific step of calling the TTW Help Line (implementation prompts). The Ticket format—small, durable, and easily posted on a refrigerator or bulletin board—also makes key information about the program easier to save and access later when necessary. Ultimately, the cardstock Ticket and notice are designed to encourage beneficiaries to consider the benefits of returning to work (or for those not ready to return to work right now, encouraging them to consider it later).
Intervention Start Date
2020-09-16
Intervention End Date
2022-03-23

Primary Outcomes

Primary Outcomes (end points)
Ticket assignment (9 months post mailing)
Primary Outcomes (explanation)
Beneficiary “assigns a Ticket,” which constitutes an agreement between the beneficiary and a service provider of their choice for employment support services.

Secondary Outcomes

Secondary Outcomes (end points)
Help Line calls (9 months post mailing)
Secondary Outcomes (explanation)
Beneficiary (or payee) calls the TTW Help Line.

Experimental Design

Experimental Design
We are launching three simultaneous experiments, each with one of three distinct populations. The three distinct populations represent three touch points the TTW program has with potential work participants:
Startup cohort (never been contacted),
1-year cohort (contacted once), and
2-year cohort (contacted twice).
For the first two experiments, we employ a factorial design with two main effects for each experiment, one pertaining to the inclusion of the revised language in the notice (factor 1) and one pertaining to the cardstock Ticket (factor 2), leading to the following confirmatory tests:
Experiment 1 (start-up cohort): The main effect of the optimized start-up notice, and the main effect of the cardstock Ticket among the start-up cohort.
Experiment 2 (1-year cohort): The main effect of the optimized start-up notice, and the main effect of the cardstock Ticket among the 1-year cohort.
Experiment 3 (2-year cohort): The main effect of the optimized start-up notice and cardstock Ticket together, compared to no notice at two-year anniversary.
Experimental Design Details
Not available
Randomization Method
SSA’s Office of Systems (Systems) will automate the random assignment process using data system partitions based on the last two digits of SSNs to assign eligible beneficiaries to one of the four study groups. The use of partitions ensures roughly even numbers between the experimental conditions. Systems will provide a print vendor with a file with the randomized lists; the vendor, in turn, prints and mails the notices.
Randomization Unit
Individual beneficiaries (nested in data system partitions).
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
n/a
Sample size: planned number of observations
Between 288,000 - 450,000 beneficiaries per experiment.
Sample size (or number of clusters) by treatment arms
Between 72,000-112,500 beneficiaries for experiments 1 and 2, and between 144,000-225,000 beneficiaries for experiment 3.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Depending on sample size and baseline ticket assignment rate (between 0.57-1.14 per cent), we anticipate the MDE to range between 0.056 - 0.098 percentage point.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan