Survey Nonresponse Bias Among University Students Receiving Charitable Food Assistance

Food for Change (FFC) is an initiative run by Houston Food Bank to help people at-risk of food insecurity find a path towards long-term stability. One of the programs operated through this initiative is the Food Scholarship (FS) program, where Houston Food Bank partners with colleges and universities to provide students with groceries to lower the financial stress of students’ grocery costs. FS distributes up to 30 pounds of groceries per week from FFC markets to students who may be at-risk of food insecurity. By distributing free food to students, the program aims to encourage continued enrollment in higher education and support academic success.

On May 12, 2026 the HFB will contact FS participants via text and email, inviting them to complete an end-of-semester survey by clicking a personalized link that takes the participant to a Qualtrics survey. Participants will be randomized into three groups. One group (control) will not be offered any financial incentive to complete the survey, while the other two groups (treatment) will be offered $10 and $20, respectively, for completion. The research team will then test for survey nonresponse bias by comparing survey completers and non-completers across the three groups along several sets of characteristics. All students are enrolled at one of three college partners with the Houston Food Bank: The University of Houston, The University of Houston Downtown, and Houston City College.

2026-05-12

2026-05-29

Completion of the end-of-semester survey

The Qualtrics survey contains 20 questions, each on separate pages. Every question has a “prefer not to answer” option, but participants cannot advance to the next question without providing an answer. Survey completion is defined as answering every survey question and therefore reaching the last “survey complete” page.

Opening the survey by clicking the survey link.
Answering at least one survey question.

Survey opened: indicator equal to one if Qualtrics or the survey platform records that the participant accessed the personalized survey link during the survey field period.
Answered at least one survey question: indicator equal to one if the participant submits a response, including “prefer not to answer,” to at least one substantive survey question during the survey field period.

Each of the three participating schools conducted their standard outreach and marketing activities for the Food Scholarship program. This study focuses on students enrolled in the program through April 29, 2026, including a subset of students who participated in a Spring 2026 pilot intervention that utilized text message nudges designed to encourage continued use of campus food pantries.

Participants will be randomized into one control group and two different treatment arms. The control group will not be offered a financial incentive for survey completion, while the two treatment groups will be offered either a $10 or $20 financial incentive for completing the endline survey. The study will assess whether financial incentives for survey completion affect survey response behavior, the composition of survey respondents, and the extent of survey nonresponse bias.

Beginning May 12, 2026, all eligible participants will receive the endline survey via email and text message, with up to three reminder contacts (two text messages and one email). The survey includes questions related to food insecurity, financial well-being, mental health, human flourishing, self-reported academic outcomes (including GPA and credit hours), and experiences using campus food pantry resources. Responses will be collected through May 29, 2026. Survey outreach messages will be identical across groups except that participants assigned to treatment groups will receive additional language describing their financial incentive offer.

Not available

Randomization will occur prior to survey outreach using the roster of all students enrolled in the Food Scholarship program through April 29, 2026. Randomization will be stratified based on three pre-treatment characteristics collected through program enrollment records: (1) monthly income reported at enrollment (measured as high (i.e., above $300, the sample median) / low / missing), (2) completion of an optional baseline survey at enrollment, and (3) whether the participant previously received text message nudges through the pilot intervention in the Spring 2026 semester. The combination of these variables results in 12 distinct strata (3 income levels × 2 baseline survey completion statuses × 2 previous text message receipt statuses). Within each stratum, we assign each participant a randomly generated value between 0 and 1 and sort participants in ascending order. We then assign the first two thirds of the participants in the strata to the control group (No Incentive), the next one sixth to the first treatment arm (Offered $10 incentive), and the remaining one sixth to the latter treatment arm (Offered $20 incentive). When stratum sizes are not divisible by six, treatment counts within strata will be rounded to preserve the target allocation shares as closely as possible.

The list of eligible survey participants and their corresponding treatment assignments will be generated prior to survey distribution. Following randomization, balance checks will be conducted across treatment groups using standardized differences in the three variables used to determine strata, as well as other baseline characteristics collected at program enrollment, including university, whether the student was new to the program in the spring of 2026, age, gender, ethnicity, whether the student has a bachelor’s degree, marital status, primary language, income source, and monthly income. Rerandomization will occur if any standardized differences exceed the threshold of 0.1.

Individual

No

N/A - not clustered; 4,153 individual units of randomization

The sample covers 4,153 students enrolled at The University of Houston, The University of Houston Downtown, and Houston City College. To be included in the study, students must meet one of two criteria: (i) they enrolled in the FS program during the spring semester (i.e., enrollment occurred between 1/1/2026 and 4/29/2026) – or – (ii) they enrolled in the FS program in a prior semester and visited a campus market during the spring semester (i.e., between the two previously mentioned dates). In other words, we exclude any previously enrolled students who had no interaction with the campus markets during the spring 2026 semester.

There are two treatment arms and one control group, with the respective sample shares outlined below:
Control (No incentive): 2/3
Treatment 1 (Offered $10): 1/6
Treatment 2 (Offered $20): 1/6

The total sample size is 4,153. Of these, 2,769 participants will receive no financial incentive for survey completion, while 692 participants will be offered a $10 incentive and another 692 participants will be offered a $20 incentive.

The total sample is composed of those who (i) visited a food market and/or (ii) enrolled in the FS program at one of the three schools over the past semester. This represents a sample of 4,153 students. The target assignment is 692 participants to the $10 incentive arm, 692 participants to the $20 incentive arm, and 2,769 to control. Thus, the expected sample sizes are: Nc = 2,769 N10 = 692 N20 = 692 N$ = 1,384 The primary outcome is an indicator for survey completion. Power calculations are based on two-sided tests with significance level alpha = 0.05 and 80% power. Unless otherwise noted, calculations use a normal approximation for differences in proportions. We assume a completion rate of 25% in the control group, 45% in the $10 incentive group, and 55% in the $20 incentive group. These assumptions imply that the pooled incentive group has an expected completion rate of approximately 50%. 1. Main effect of any financial incentive on survey completion The first power calculation concerns the pooled effect of receiving any financial incentive, combining the $10 and $20 incentive arms. The primary estimating equation for this contrast is: Yi =s+T$i+i where Yi is an indicator for survey completion, T$i equals one for participants assigned to either incentive arm, and s are randomization-stratum fixed effects. Under the assumed completion rates of 25% in the control group and 50% in the pooled incentive group, the approximate standard error for the difference in completion rates is: SE=sqrt(0.5(1-0.5)/1,384) + sqrt(0.25(1-0.25)/2,769) This implies an approximate minimum detectable effect of 4.4 percentage points. 2. Difference between the $10 and $20 incentive arms The second power calculation concerns whether the $20 incentive has a larger effect on survey completion than the $10 incentive. The relevant test compares completion rates between the two incentive arms: H0: B10=B20 The corresponding estimating equation is: Yi = as + B10*T10i + B20*T20i + ei where T10i and T20i indicate assignment to the $10 and $20 incentive arms, respectively. For this comparison, both incentive arms have approximately 700 participants. Assuming completion rates of 45% in the $10 arm and 55% in the $20 arm, the approximate standard error is: SE=sqrt(0.45(1-0.45)/692) + sqrt(0.55(1-0.55)/692) This implies an approximate minimum detectable difference of 7.5 percentage points. 3. Heterogeneity of the incentive effect across groups The third power calculation concerns whether the effect of receiving any financial incentive on completion differs across observed characteristics. We will make many such comparisons, but an important test will be to compare whether individuals of different baseline income levels are more or less responsive to survey incentives. For analysis of this sort, the relevant estimating equation is: Yi = as + B1 T$i + B2(T$i*HighInci) + B3 (T$i*MissInci) + ei where low income is the omitted category. The coefficient B2 measures whether the effect of any incentive on completion differs between individuals with high vs. low income. The coefficient B3 captures the corresponding difference for participants with missing income, but this comparison is secondary. Categorizing participants into high and low income groups using the sample mean of $300, the corresponding sample sizes and MDEs (comparing one group to all other groups) are presented in the following table: Income Group Breakdown (N | Incentive N | Control N | MDE): Low Income: 1,821 total (607 Inc. / 1,214 Ctrl) | MDE: 6.7 pp High Income: 1,822 total (607 Inc. / 1,215 Ctrl) | MDE: 6.7 pp Missing Income: 510 total (170 Inc. / 340 Ctrl) | MDE: 12.6 pp When comparing the high income group to the low group, the MDE is 9.4 percentage points.