Ticket to Work Notice Optimization project

Last registered on September 11, 2020

View Trial History

Pre-Trial

Trial Information

General Information

Title

Ticket to Work Notice Optimization project

RCT ID

AEARCTR-0006380

Initial registration date

September 10, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

September 11, 2020, 12:00 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

United States of America

Region

Primary Investigator

Name

Sebastian Jilke

Affiliation

Office of Evaluation Sciences / General Services Administration

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Rekha Balu

PI Affiliation

Office of Evaluation Sciences / General Services Administration

Contact Investigator

PI Name

Elizabeth Bell

PI Affiliation

Office of Evaluation Sciences / General Services Administration

Contact Investigator

PI Name

Jeffrey Hemmeter

PI Affiliation

Social Security Administration

Contact Investigator

PI Name

Onika Richardson

PI Affiliation

Social Security Administration

Contact Investigator

PI Name

Elana Safran

PI Affiliation

Office of Evaluation Sciences / General Services Administration

Contact Investigator

Additional Trial Information

Status

In development

Start date

2020-09-16

End date

2022-12-23

Keywords

Behavior, Labor, Welfare

Additional Keywords

Social Policy, Disability

JEL code(s)

Secondary IDs

Abstract

The Ticket to Work (TTW) program provides Social Security Disability Insurance (SSDI) beneficiaries and Supplemental Security Income (SSI) recipients with disabilities with choices for receiving employment services while providing incentives to providers to serve these individuals. The goal of the TTW legislation was to expand the availability of employment services for beneficiaries beyond the existing state vocational rehabilitation (VR) agencies, reducing dependency on disability benefits. The TTW Notice Optimization project seeks to increase participation in the TTW program by developing an evidence-based approach to targeting outreach to eligible beneficiaries. The project will test changes to Ticket notices including the types of notices sent and the language used. The project aims to increase the number of beneficiaries who “assign a Ticket” with an Employment Network (EN) or VR agency. The assignment of a Ticket constitutes an agreement between the beneficiary and a service provider of their choice for employment support services. More intermediate goals of the project include an increase in initial calls to the TTW Help Line. The intervention is tested in a 2 x 2 factorial design where the re-designed letter constitutes factor #1 and an added cardstock Ticket factor #2. The intervention will run for 18 months, starting September 16, 2020, and outcomes will be measured at 9 months post-intervention for each individual.

External Link(s)

Registration Citation

Citation

Balu, Rekha et al. 2020. "Ticket to Work Notice Optimization project." AEA RCT Registry. September 11. https://doi.org/10.1257/rct.6380-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

CURRENT PRACTICE: To notify beneficiaries of their eligibility to participate in the TTW program, SSA currently mails notices to beneficiaries approximately two months after award (start-up notices) and at the 12-month and 36-month anniversary dates of their award. The notices include standard language about the TTW program and include a TTW brochure, titled “Your Ticket to Work” (available in English and Spanish), and an “inline, paper Ticket.” The inline Ticket is currently an additional page following the letter with additional TTW program information.
The intervention consists of updated communication materials that SSA sends to people with disabilities (SSDI beneficiaries and/or SSI recipients) between the ages of 18 and 55.
INTERVENTION #1 (Notice Structure): First, the project will modify both the start-up and one-year notices and eliminate the three-year letter notice. In place of the three-year notice, we are introducing a notice at two years post-award. The newly designed notices include a more succinct description of the program, which emphasizes saliency, and clear, plain-language action steps encouraging beneficiaries to call the TTW Help Line.
INTERVENTION #2 (Hardcopy Ticket): In addition to testing the amended notices, we will test the inclusion of a cardstock Ticket. These “Tickets” will be printed on 3.5” by 8” cardstock paper. The goal of the newly designed cardstock Tickets is to make the benefits of participating in the TTW program more salient and concise than the letter and, specifically, to help people think of themselves as ready to work (identity priming) and take the specific step of calling the TTW Help Line (implementation prompts). The Ticket format—small, durable, and easily posted on a refrigerator or bulletin board—also makes key information about the program easier to save and access later when necessary. Ultimately, the cardstock Ticket and notice are designed to encourage beneficiaries to consider the benefits of returning to work (or for those not ready to return to work right now, encouraging them to consider it later).

Intervention Start Date

2020-09-16

Intervention End Date

2022-03-23

Primary Outcomes

Primary Outcomes (end points)

Ticket assignment (9 months post mailing)

Primary Outcomes (explanation)

Beneficiary “assigns a Ticket,” which constitutes an agreement between the beneficiary and a service provider of their choice for employment support services.

Secondary Outcomes

Secondary Outcomes (end points)

Help Line calls (9 months post mailing)

Secondary Outcomes (explanation)

Beneficiary (or payee) calls the TTW Help Line.

Experimental Design

We are launching three simultaneous experiments, each with one of three distinct populations. The three distinct populations represent three touch points the TTW program has with potential work participants:
Startup cohort (never been contacted),
1-year cohort (contacted once), and
2-year cohort (contacted twice).
For the first two experiments, we employ a factorial design with two main effects for each experiment, one pertaining to the inclusion of the revised language in the notice (factor 1) and one pertaining to the cardstock Ticket (factor 2), leading to the following confirmatory tests:
Experiment 1 (start-up cohort): The main effect of the optimized start-up notice, and the main effect of the cardstock Ticket among the start-up cohort.
Experiment 2 (1-year cohort): The main effect of the optimized start-up notice, and the main effect of the cardstock Ticket among the 1-year cohort.
Experiment 3 (2-year cohort): The main effect of the optimized start-up notice and cardstock Ticket together, compared to no notice at two-year anniversary.

Experimental Design Details

Randomization Method

SSA’s Office of Systems (Systems) will automate the random assignment process using data system partitions based on the last two digits of SSNs to assign eligible beneficiaries to one of the four study groups. The use of partitions ensures roughly even numbers between the experimental conditions. Systems will provide a print vendor with a file with the randomized lists; the vendor, in turn, prints and mails the notices.

Randomization Unit

Individual beneficiaries (nested in data system partitions).

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

n/a

Sample size: planned number of observations

Between 288,000 - 450,000 beneficiaries per experiment.

Sample size (or number of clusters) by treatment arms

Between 72,000-112,500 beneficiaries for experiments 1 and 2, and between 144,000-225,000 beneficiaries for experiment 3.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Depending on sample size and baseline ticket assignment rate (between 0.57-1.14 per cent), we anticipate the MDE to range between 0.056 - 0.098 percentage point.

Supporting Documents and Materials

IRB