Jumpstarting College: Do financial aid and summer school information nudges to high school seniors improve college outcomes?

The survey will implement two information nudges, which will be randomly assigned to students within the Qualtrics-based survey. The specific questions are outlined below:

Treatment #1: Summer School Nudge
The following sequence of questions will nudge students on the potential benefits of attending summer school at a California Community Colleges. All students will be asked question 8, with one-half of students randomly assigned to received the below information, with follow-up question 8b.

Q8. Do you plan to take college classes this summer?
• Definitely not
• Probably not
• Not sure
• Probably will
• Definitely will

(One-half of respondents received the following information nudge):

Did you know that taking college classes this summer could help you finish college faster and save money along the way?

At your local community college:
• There are many in-person and online options
• Credits from most classes can be transferred to a UC/CSU.
• Many students qualify for free tuition

Q8b. After taking this information into account, how likely are you to take college classes this summer?
• Definitely not
• Probably not
• Not sure
• Probably will
• Definitely will

Treatment #2: Full-time California Community College Enrollment Nudge
The following information will nudge students who indicated they plan to attend a California Community College in the fall on the potential financial aid benefits of enrolling full time.

Q15. You indicated you plan to enroll in a California Community College next year. Community college students eligible for the Cal Grant can receive as much as $8000 a year in extra grant aid for taking 15 units each term (4-5 classes) This money could be used for living expenses and other costs, and would not need to be paid back.

Prior to taking this survey, did you know that Cal Grant recipients could get this additional money for enrolling in 15 units each term?
• Yes
• No

To find out if you’re eligible for the Cal Grant, contact your community college financial aid office.

2023-05-16

2023-06-15

For the summer school nudge, the primary outcomes of interest will be whether the student enrolled in one or more summer school courses at a California Community College during the summer of 2023 as well as the number of units attempted and completed. For the full-time enrollment nudge, the primary outcomes of interest will be: for both the fall and spring semesters following high school graduation, whether the student enrolled in at least 12 units (i.e., full-time enrollment), at least 15 units, the total number of units attempted, and the total number of units completed.

For both treatments, we are additionally interested in a number of important short and longer-run outcomes, including enrollment, grades, persistence, transfer, completion, major, and time-to-degree.

Multivariate models measuring the impact of treatment on the outcomes listed above will be employed to reduce residual variance and improve power. Specifically, for each nudge we will regress our various outcome variables on a treatment indicator variable. We plan to include additional covariates that reflect characteristics such as age, race/ethnicity, sex/gender, family income (e.g., EFC), and achievement (e.g., 11th grade test scores). Multiple hypothesis testing will be accounted for using standard correction methods.

Randomization is conducted automatically through the Qualtrics survey.

Individual

No

0

Approximately 32,000: Assumes a 10 percent response rate to the survey which will be sent to the 323,555 California high school seniors who submitted a FAFSA.

Treatment Arm #1: 16,000 treatment & 16,000 control. Estimate based on 32,000 expected number of respondents. 50% (16,000) will be assigned to the treatment and 50% (16,000) will be assigned to control.

Treatment Arm #2: 7,200 treatment & 7,200 control. Estimate based on 32,000 expected number of respondents. Based on historical data, we estimate that 45% (14,400) of respondents will indicate they plan to attend a community college. 50% (7,200) will be assigned to the treatment and 50% (7,200) will be assigned to control.

Treatment Arm #1: With a sample size of 32,000 and an expected control group summer school enrollment rate of 26%, at 0.9 power, we will be able to detect treatment effects at the .05-level of significance as small as 0.008 (0.8 percentage points). This calculation is based on a proportional means test without any control variables. Treatment Arm #2: With a sample size of 14,400 and an expected control group enrollment rate of 9% in 15 or more units, at 0.9 power, we will be able to detect treatment effects at the .05-level of significance as small as 0.0052 (0.52 percentage points). This calculation is based on a proportional means test without any control variables.