Material hardship and mental health among low-income families during COVID-19: experimental evidence from an unconditional cash transfer

As part of its charitable mission, the nonprofit organization GiveDirectly, is providing a sample of low-income families in the U.S. a one-time cash transfer of $1,000 to help mitigate the financial stress associated with the COVID-19 pandemic. The intervention is a one-time, $1,000 lump-sum cash payment. Families may use the money as they choose (i.e., there are no restrictions on how they spend the money). As a one-time, lump sum payment this aid does not impact other public assistance benefits. Families are selected randomly from a sample of individuals who are receiving federal assistance through the Supplemental Nutrition Assistance Program (SNAP). The payment is completely unanticipated and is described as an effort to provide cash relief to Americans in need during the COVID-19 pandemic. Individuals are not required to participate in the research project, or fulfill any other obligations, in order to receive the cash payment.

2020-09-09

2020-10-10

The primary outcomes in this study will be composite measures of material hardship and mental health. We will measure these outcomes roughly 6 weeks post intervention. In addition to looking at the effect of the intervention on the full sample for both outcomes, we also plan to examine the impact of the intervention on particularly disadvantaged individuals in our sample. We plan to define this subgroup in two ways: (i) using a simple measure of total household income last month; (ii) using a composite measure of predicted material hardship based on other individual responses.

Test 1 - Material hardship, Full sample
Test 2 - Material hardship, low-income sample
Test 3 - Material hardship, low-income with children
Test 4 - Mental health, full sample
Test 5 - Mental health, low-income sample

To create composites, we follow the approach outlined in Anderson (2008) and calculate a weighted mean of the standardized survey items within the domain. We will first switch the signs of all items so that the positive direction always indicates a “better” outcome. We will then standardize each outcome using the mean and standard deviation of that outcome among the control group. We will then create a weighted average of these standardized outcomes using all of the outcomes in the domain. For weights we will use the inverse of the covariance matrix of the transformed outcomes in the domain.

We intend to conduct a variety of secondary (i.e., exploratory) analyses guided by theory and prior literature.

We intend to conduct a variety of secondary (i.e., exploratory) analyses guided by theory and prior literature. The discussion below is meant to be illustrative, but not exhaustive. First, we will examine several other outcomes as relevant for the respondent: child well-being, relationship with partner and parenting practices. Second, we will examine the impact of the treatment on individual items or groups of items within domains. In the case of material hardship, for example, we will explore whether the cash payment has a larger impact on certain hardships (e.g., food insecurity vs. inability to pay utilities). In the case of mental health, we will separately examine impacts on depression versus anxiety. Third, we will explore treatment effect heterogeneity along several dimensions beyond those specified in the primary hypotheses. For example, we will examine heterogeneity in treatment effects based on (i) the generosity of a state’s social welfare system, (ii) extent to which the COVID pandemic negatively impacted the individual’s geography area; (iii) the demographic characteristics of an individual’s zipcode including whether it is a rural or urban area.

The sample will be drawn from a set of families receiving benefits through the Supplemental Nutrition Assistance Program (SNAP) who utilize the FreshEBT mobile app (“app users”) to manage their benefits. App users who live in zipcodes with poverty rates of at least 35% are eligible for the intervention. Among this set, we will randomly select treatment and control individuals. Individuals selected for the treatment group will be notified via a banner on the FreshEBT app; control individuals will receive no such banner. Roughly six weeks following the intervention, we will reach out to individuals in both the treatment and control groups via a banner in the app, inviting them to participate in a research study about their experiences during COVID. Individuals who agreed to participate in the research study will be sent a link to a short online survey. Participants will be compensated $10 for completing the survey.

We will draw a random sample of individuals, stratifying on whether or not the individual began using the FreshEBT app prior to March 1, 2020 and a three-category measure of the generosity of the social welfare benefits in which the individual lives (as measured by the maximum TANF benefits for a three-person household in the state). This will give us a total of six strata. We plan to use the re-randomization method outlined in Lock and Rubin (2012) to ensure covariate balance. To analyze the results of the experiment, we will use randomization inference that is appropriate for the re-randomization scheme we conducted.

The individual

No

Roughly 20,000 individuals

Roughly 20,000 individuals

10,000 individuals in each of two arms (one treatment and one control)

We have funding to survey 10,000 individuals. In order to calculate the total number of individuals to survey, we are assuming a 50% response rate. This means we will plan to distribute surveys to roughly 20,000 individuals (10K for each group). Because we are interested in judging the success of the intervention on each outcome individually, we will be adjusting for multiple hypothesis testing. For the purpose of power calculations, we take a conservative approach and use a Bonferroni correction to control for the Familywise Error Rate. For each individual hypothesis test, we assume a probability of Type I error (alpha) of 0.05, a statistical power (1-beta) of 0.8, and a two-tailed hypothesis test. Given that we will be testing six outcomes, the Bonferroni adjustment dictates that we need to power our study to detect effects given an alpha of 0.0083. We assume the proportion of the sample randomized to treatment will be 0.5. Based on our analysis of data on a similar population, we assume that by including the available baseline data, the R-squared in our outcome models will be 0.05. Assuming an equal split of treatment and control observations, the MDES is roughly 0.068 SD for tests involving the full sample. Based on earlier work, we assume that roughly 50% of the sample will fall into the low-income subgroup. For tests involving this subgroup, the MDES will be 0.096 SD. Take-Up Rate - Based on recent prior experience with the cash payments, we expect that roughly 90 percent of those offered the payment will ultimately receive the money. Because we are interested in the effect of receiving the money as opposed to simply being offered the money, we will estimate the instrumental variables models described below. Given a 90% take-up rate, the MDES for the LATE estimate will be .068/.9 = .075 SD for full sample tests and 0.096/.9 = .107 SD for subsample tests.