NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
SNAP Take-Up Evaluation
Initial registration date
October 23, 2015
September 18, 2019 11:01 AM EDT
Other Primary Investigator(s)
Additional Trial Information
Benefits Data Trust (BDT) is a national not-for-profit organization based in Philadelphia committed to transforming how individuals in need access public benefits. Using a data-driven and technology-based approach, BDT provides targeted outreach and comprehensive applications assistance to individuals who are likely eligible for public benefits.
This randomized trial investigates the impact of BDT’s outreach and application assistance to individuals aged 60 and over who are likely to be eligible for the Supplemental Nutrition Assistance Program (SNAP). Through random assignment to receive BDT’s assistance, outreach only, or to control, we plan to study the impact of outreach and application assistance on the level of take-up and characteristics of the marginal individual induced to enroll due to the intervention. Registration Citation
“Low Touch” Interventions: BDT will send a letter to individuals who are likely to be eligible for SNAP informing them of their potential eligibility, and providing contact information for the Pennsylvania Department of Human Services, which is the agency in charge of processing SNAP applications. If the household has not called the number listed on the letter within 8 weeks, additional outreach, this time a postcard, will be sent with the same information.
“High Touch” Interventions: BDT will send a letter to individuals who are likely to be eligible for SNAP informing them of their potential eligibility, and providing contact information for their in-house call center. As in the “low touch” interventions, if the household has not called the number listed on the letter within 8 weeks, a postcard will be sent with the same information. If the individual or household is interested in applying for benefits, BDT will help them determine if they are eligible, complete the application on their behalf, help them assemble the necessary documentation, and submit the completed application to the appropriate government agency. BDT simplifies the application process by completing applications by telephone and allowing applicants to sign the application via telephonic signature.
Control: The control group will not receive any letters or contact information.
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
Number of SNAP enrollees [Timeframe: 9 months after initial mailing] Secondary: Baseline characteristics of enrollees (e.g. demographics, measures of economic well-being, measures of health etc) [Timeframe: 9 months after initial mailing] Number of SNAP applications [Timeframe: 9 months after initial mailing] Baseline characteristics of applicants [Timeframe: 9 months after initial mailing] Number of responses to outreach letters (i.e., phone calls to the number listed on the outreach letter) [Timeframe: 9 months after initial mailing]
Primary Outcomes (explanation)
We are interested in measuring characteristics of the enrollees, for example measures of economic well-being, demographics and health status. Which characteristics we measure and how we measure them will depend largely on the quality and availability of data.
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Individuals were selected from a list of Pennsylvania residents 60 and over who received Medicaid in the last year, but did not receive SNAP. Residents from Philadelphia were excluded from the sample because BDT has frequently conducted similar outreach campaigns in Philadelphia. From this list, households were randomly assigned in equal proportions to Control, Low Touch, or High Touch interventions.
Experimental Design Details
Randomization done in office by a computer (Stata)
Was the treatment clustered?
Sample size: planned number of clusters
Sample size: planned number of observations
Sample size (or number of clusters) by treatment arms
10,630 households control, 10,629 households low touch, 10,629 households high touch
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Using historical statistics provided by BDT on outreach to Medicaid recipients in Philadelphia and Maryland, preliminary power calculations suggest that the minimum detectable effect size is a 2 percentage point change (34%) or less in enrollment when comparing any two treatment arms, assuming conventional 80% power and 95% confidence intervals. We assume 4 percent of the control group enrolls in SNAP absent any intervention (standard deviation = .2). This effect size is smaller than effects seen in historical program data and in the literature for similar outreach interventions.
Using the same assumptions noted above, we estimate power to detect differences between treatment arms in average characteristics of those who apply and/or enroll in SNAP. Instead of specifying particular characteristics, we explore power to detect percentage point changes in a standard indicator that half of the control group is assumed to display (e.g. has less than median income). Thus, by construction, the standard deviation of the outcome is .5 for the control group. We find that for comparison of the High Touch and Control groups we are powered to minimum effect sizes of 8.3 percentage points, and for comparison of other any other two study arms we are powered to detect less than 16 percentage point differences. In light of the few studies that examine similar questions average enrollee characteristics, this study appears sufficiently powered to detect meaningful differences in characteristics.
INSTITUTIONAL REVIEW BOARDS (IRBs)
Committee on the Use of Humans as Experimental Subjects (MIT COUHES)
IRB Approval Date
IRB Approval Number
NBER Institutional Review Board
IRB Approval Date
IRB Approval Number
Analysis Plan Documents
March 28, 2016
Post Trial Information
Is the intervention completed?
Intervention Completion Date
May 27, 2016, 12:00 AM +00:00
Is data collection complete?
Reports, Papers & Other Materials
We develop a framework for welfare analysis of interventions designed to increase take-up of social safety net programs in the presence of potential behavioral biases. We calibrate the key parameters using a randomized field experiment in which 30,000 elderly individuals not enrolled in—but likely eligible for—the Supplemental Nutrition Assistance Program (SNAP) are either provided with information that they are likely eligible, provided with this information and offered assistance in applying, or are in a “status quo” control group. Only 6% of the control group enrolls in SNAP over the next nine months, compared to 11% of the Information Only group and 18% of the Information Plus Assistance group. The individuals who apply or enroll in response to either intervention have higher net income and are less sick than the average enrollee in the control group. We present evidence consistent with the existence of optimization frictions that are greater for needier individuals, which suggests that the poor targeting properties of the interventions reduce their welfare benefits.
Amy Finkelstein, Matthew J Notowidigdo, Take-Up and Targeting: Experimental Evidence from SNAP, The Quarterly Journal of Economics, Volume 134, Issue 3, August 2019, Pages 1505–1556, https://doi.org/10.1093/qje/qjz013
REPORTS & OTHER MATERIALS
In the United States, enrollment in social safety programs is not automatic, and many social safety programs experience low take-up; many individuals who are eligible for certain programs fail to become enrolled. Researchers studied the impact of providin
Finkelstein, Amy, Matthew J Notowidigdo. "Take-Up and Targeting: Experimental Evidence from SNAP," J-PAL Evaluation Summary, January 30, 2018.