SNAP Take-Up Evaluation

“Low Touch” Interventions: BDT will send a letter to individuals who are likely to be eligible for SNAP informing them of their potential eligibility, and providing contact information for the Pennsylvania Department of Human Services, which is the agency in charge of processing SNAP applications. If the household has not called the number listed on the letter within 8 weeks, additional outreach, this time a postcard, will be sent with the same information.

“High Touch” Interventions: BDT will send a letter to individuals who are likely to be eligible for SNAP informing them of their potential eligibility, and providing contact information for their in-house call center. As in the “low touch” interventions, if the household has not called the number listed on the letter within 8 weeks, a postcard will be sent with the same information.

If the individual or household is interested in applying for benefits, BDT will help them determine if they are eligible, complete the application on their behalf, help them assemble the necessary documentation, and submit the completed application to the appropriate government agency. BDT simplifies the application process by completing applications by telephone and allowing applicants to sign the application via telephonic signature.

Control: The control group will not receive any letters or contact information.

Low Touch and High Touch study groups will be further randomized into six unique treatment arms that each receive letters and postcards with varying content and format, as summarized below:

Treatment 1: High Touch BDT Standard (n=7972)
Treatment 2: High Touch Marketing (n=2657)
Treatment 3: Low Touch Standard (n=2657)
Treatment 4: Low Touch Standard without follow-up postcard (n=2658)
Treatment 5: Low Touch Marketing (n=2657)
Treatment 6: Low Touch Framing (n=2657)
Control: No intervention (n=10630)

The “Standard” letters and postcards present information in a way that is “standard” for most of BDT’s current outreach.

The “Marketing” letters and postcards are modified to attract clients using a "marketing" approach. They are printed in color rather than black & white, and have a slightly different layout and tone.

The “Framing” letters and postcards differ from “standard” only in the way they describe the SNAP benefits amounts that the average 60+ SNAP recipient in Pennsylvania receives. While the standard letter describes this as an “average of $119 a month” and “hundreds of dollars each year,” the framing letters state that “thousands of older Pennsylvanians already get up to $200 a month” and “thousands of dollars each year.” Both are factually accurate statements: the only difference is the framing of the amount.

We will analyze whether enrollment rates vary across the various arms and whether the “types” or characteristics of individuals who respond to these letters vary by the type of letter they receive.

2015-11-04

2016-05-30

Primary:
Number of SNAP enrollees [Timeframe: 9 months after initial mailing]

Secondary:

Baseline characteristics of enrollees (e.g. demographics, measures of economic well-being, measures of health etc) [Timeframe: 9 months after initial mailing]

Number of SNAP applications [Timeframe: 9 months after initial mailing]

Baseline characteristics of applicants [Timeframe: 9 months after initial mailing]

Number of responses to outreach letters (i.e., phone calls to the number listed on the outreach letter) [Timeframe: 9 months after initial mailing]

We are interested in measuring characteristics of the enrollees, for example measures of economic well-being, demographics and health status. Which characteristics we measure and how we measure them will depend largely on the quality and availability of data.

Individuals were selected from a list of Pennsylvania residents 60 and over who received Medicaid in the last year, but did not receive SNAP. Residents from Philadelphia were excluded from the sample because BDT has frequently conducted similar outreach campaigns in Philadelphia. From this list, households were randomly assigned in equal proportions to Control, Low Touch, or High Touch interventions.

Randomization done in office by a computer (Stata)

Household

No

31,888 households

31,888 households

10,630 households control, 10,629 households low touch, 10,629 households high touch

Using historical statistics provided by BDT on outreach to Medicaid recipients in Philadelphia and Maryland, preliminary power calculations suggest that the minimum detectable effect size is a 2 percentage point change (34%) or less in enrollment when comparing any two treatment arms, assuming conventional 80% power and 95% confidence intervals. We assume 4 percent of the control group enrolls in SNAP absent any intervention (standard deviation = .2). This effect size is smaller than effects seen in historical program data and in the literature for similar outreach interventions. Using the same assumptions noted above, we estimate power to detect differences between treatment arms in average characteristics of those who apply and/or enroll in SNAP. Instead of specifying particular characteristics, we explore power to detect percentage point changes in a standard indicator that half of the control group is assumed to display (e.g. has less than median income). Thus, by construction, the standard deviation of the outcome is .5 for the control group. We find that for comparison of the High Touch and Control groups we are powered to minimum effect sizes of 8.3 percentage points, and for comparison of other any other two study arms we are powered to detect less than 16 percentage point differences. In light of the few studies that examine similar questions average enrollee characteristics, this study appears sufficiently powered to detect meaningful differences in characteristics.

Analysis Plan

We develop a framework for welfare analysis of interventions designed to increase take-up of social safety net programs in the presence of potential behavioral biases. We calibrate the key parameters using a randomized field experiment in which 30,000 elderly individuals not enrolled in—but likely eligible for—the Supplemental Nutrition Assistance Program (SNAP) are either provided with information that they are likely eligible, provided with this information and offered assistance in applying, or are in a “status quo” control group. Only 6% of the control group enrolls in SNAP over the next nine months, compared to 11% of the Information Only group and 18% of the Information Plus Assistance group. The individuals who apply or enroll in response to either intervention have higher net income and are less sick than the average enrollee in the control group. We present evidence consistent with the existence of optimization frictions that are greater for needier individuals, which suggests that the poor targeting properties of the interventions reduce their welfare benefits.

Amy Finkelstein, Matthew J Notowidigdo, Take-Up and Targeting: Experimental Evidence from SNAP, The Quarterly Journal of Economics, Volume 134, Issue 3, August 2019, Pages 1505–1556, https://doi.org/10.1093/qje/qjz013

https://doi.org/10.1093/qje/qjz013