Evaluating form modifications to make it easier to report income on benefits applications – Evidence from multiple income households

Last registered on August 06, 2024

Pre-Trial

General Information

Title

RCT ID

AEARCTR-0014101

Initial registration date

July 30, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

August 06, 2024, 1:23 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

United States of America

Region

Primary Investigator

Name

Stephanie Tepper

Affiliation

Office of Evaluation Sciences

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Blair Read

PI Affiliation

Office of Evaluation Sciences

PI Name

Michael Hand

PI Affiliation

Office of Evaluation Sciences

PI Name

Mary Clair Turner

PI Affiliation

Office of Evaluation Sciences

Additional Trial Information

Status

Completed

Start date

2024-06-10

End date

2024-08-06

Keywords

Welfare

Additional Keywords

Social Policy, Safety Net, Social Welfare Programs, Poverty Alleviation

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

This project aims to improve applicants’ experiences with reporting their income on the Supplemental Nutrition Assistance Program (SNAP) application. The SNAP application requires applicants to provide information about their income in the past 30 days to assess eligibility and calculate benefits amounts. Additionally, applicants have to upload verification documents that confirm their responses on the application. This project uses an application digital assister, designed by Code For America (CfA), to help individuals in a large U.S. state complete the SNAP application. This evaluation examines which of two different income reporting options are more effective at encouraging applicants with multiple jobs in their household to complete the SNAP application. In the federal context, multiple benefits applications ask applicants to report their income, and administrative burdens in form completion can impede access to federal programs among eligible beneficiaries. This project also will build generalizable evidence regarding how to ask people about income that can be applied across forms, benefits applications, and surveys administered by the federal government that modules on income reporting.

External Link(s)

Registration Citation

Citation

Hand, Michael et al. 2024. "Evaluating form modifications to make it easier to report income on benefits applications – Evidence from multiple income households." AEA RCT Registry. August 06. https://doi.org/10.1257/rct.14101-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Intervention Start Date

2024-06-10

Intervention End Date

2024-08-06

Primary Outcomes

Primary Outcomes (end points)

The primary outcome of interest is application completion, a binary measure equal to 1 if the applicant entered in a signature on the last page of the application and hit “sign and submit application” and 0 otherwise.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Secondary outcomes of interest are:
Total amount of income reported
Time spent on the application
Uploading of verification documents
Whether client was shown the screen telling them they might be over the income threshold for SNAP eligibility
Screen of application drop-off
Applicant satisfaction rating
Whether client is approved/denied for SNAP

Secondary Outcomes (explanation)

Experimental Design

In this evaluation, SNAP applicants will be randomly assigned to see one of two different income reporting conditions: an unstructured reporting condition (control) or a structured reporting condition (intervention). In the unstructured condition, applicants will be asked to report their total household income from the last 30 days in a single text box. They will also be asked if they expect to earn less money in the following month and, if so, how much they expect to earn. At the end of the application, they will be asked to review and confirm that the income they reported is correct. Note that this condition is the business-as-usual process for reporting income in the digital assister application. In the structured reporting condition, applicants will be shown a screen with the names of every person in their household. One by one, they will select a person in the household and enter each of that person’s jobs and the income they earn from each job. They will also be asked if they expect to earn less money in the following month and, if so, how much they expect to earn. Applicants will go through this process job-by-job and person-by-person until all income from the household has been entered. In this condition, applicants will also have the option to estimate the income for each earner in the household if they prefer not to enter income job-by-job. At the end of the application, they will be asked to review and confirm that the income they reported is correct.

The income reporting module occurs after applicants have answered basic questions about themselves, including the county they live in, their disability status, their student status, their citizenship status, and if they receive income support in the form of SSI or disability payments. Applicants are then asked if they are self-employed, if there are multiple jobs in their household, and if they have stable or variable income. Only applicants who have multiple jobs in their household are included in the evaluation.

Experimental Design Details

Randomization Method

Applicants will be randomized using Vanity, an Experiment Driven Development platform. Each eligible applicant had an equal chance of being assigned to each of the intervention arms. See more about Vanity randomization at https://vanity.labnotes.org/ab_testing.html.

Randomization Unit

Units are randomized at the level of individual application session.

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

N/A

Sample size: planned number of observations

~20,000

Sample size (or number of clusters) by treatment arms

10,000 applicants in the control group and 10,000 applicants in the treatment group.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

The minimum detectable effect size for our primary outcome is approximately 2 percentage points with a sample of approximately 20,000. This power analysis assumes 80% power and a baseline submission rate of 58%, which is based on historical data from the digital assister.

Supporting Documents and Materials

IRB