Increasing the Take-Up of Cal Grant Awards Through Improved Notification Letters II

In November, the California Student Aid Commission (CSAC) began mailing preliminary notification letters to notify students that they have qualified for a Cal Grant. These letters include information on the size of the Cal Grant award, instructions on how to claim the award, and some additional information on other potential sources of aid.

For the 2019-20 year, we have designed four variants of the notification letter. These vary in the language that they include and in the information they provide about college costs. A baseline letter will be compared to three variants:

1. The first variant will remove elements of the baseline letter designed to explicitly bolster the students’ sense of belonging in college.

2. The second variant will feature language designed to show receipt of the Cal Grant as a social norm. This letter will add a sentence to the baseline letter noting that many students went to the recipient’s own high school.

3. The third variant will be similar to the baseline letter, but will replace the information about the value of the Cal Grant with a table with customized information about the estimated cost and estimated aid at up to ten California public colleges and universities. The colleges will correspond to the ones that the student listed on his/her FAFSA to receive his/her financial information for use in financial aid calculations. CSAC will use the financial aid information in the FAFSA to populate these colleges’ net price calculators and generate a customized estimate of the full cost of attendance at the institution, the aid that a student can expect to receive if he/she enrolls, and the net cost that the student can expect to pay for their first year of attendance.

CSAC is creating and mailing all communications, recording which version of each letter is sent to students at a given school. California Policy Lab researchers at the University of California, Berkeley, will conduct the data analysis.

2018-11-21

2019-09-03

(1) Creation of account on CSAC website
(2) Claiming of an award
(3) Enrollment in college
(4) Type of institution attended

(1) Did student create an account (Y/N); Date on which student created account
(2) Was an award paid to the student? (Y/N)
(3) Did the student enroll in college (Y/N); Full-time vs. Part-time enrollment.
(4) Possible institutions include: California Community College; Cal State University; University of California; private non-profit WASC-accredited college; private for-profit non-WASC accredited college.

(1) Graduation
(2) Persistence
(3) Ongoing claiming of Cal Grant awards beyond the first year of enrollment

(1) Did student graduate or transfer from a two-year to a four-year college (Y/N); type of degree/certificate earned; time to completion.
(2) Re-enrollment after first semester, first year, and subsequent milestone points; credit accumulation at various milestone points.
(3) Did student claim Cal Grant in years following initial enrollment?

This study will employ a randomized design to test the causal effects of the different letter variants relative to the baseline. High schools will be randomly assigned to four groups, with all students at a given high school receiving the letter type assigned to their school.

We will look for differences among the groups in the number of students who register for a webgrants4students account, the time between receiving the letter and creating the account, whether students enroll in college and claim their award, and the types of institutions students attend. Additionally, we will look for differences in longer-term outcomes including persistence, graduation, and continued claiming of Cal Grants in subsequent years.

Not available

Randomization took place on a computer in the CPL offices. CSAC provided a list of high schools at which at least one student was deemed eligible for a Cal Grant award in either the 2015-16 or 2016-17 academic years. CPL researchers returned the list with randomly generated treatment/control assignments, as well as assignments for schools that did not appear in the previous years’ data.

Randomization was done at the school level. Schools were divided into seven strata based on the number of notification letters sent to students from the school in 2015-16 and 2016-17 and share of those students on whose behalf grants were paid out in 2016-17 and 2017-18, respectively:
(a) Schools at which 10 or more students received notification letters in 2016-17:
a. Schools at which 0-59% of those students received CalGrants (i.e., grants were paid out on their behalf). Number of schools = 288.
b. Schools at which 60-75% of those students received CalGrants. Number of schools = 860.
c. Schools at which 76% or more of students received CalGrants. Number of schools = 439.
(b) Schools at which less than 10 students received notification letters in 2016-17 but more than 10 received letters when aggregated across 2015-16 and 2016-17:
a. Schools at which 0-59% of those students (aggregating across 2015-16 and 2016-17) received CalGrants. Number of schools = 129.
b. Schools at which 60-75% of those students received CalGrants. Number of schools = 71.
c. Schools at which 76% or more of students received CalGrants. Number of schools = 34.
(c) Schools at which the aggregate number of letters sent in 2015-16 and 2016-17 is greater than zero but less than ten. Number of schools = 709.
(d) Schools that do not appear in the 2016 or 2017 data. Number of schools unknown.

One quarter of schools within each stratum were randomly assigned to the control or to one of the three treatment conditions (probability = 1/4 for each).

Randomization was done at the school level.

Yes

2,530 schools

140,000 students

Treatment 1: 630 schools (anticipated)
Treatment 2: 633 schools (anticipated)
Treatment 3: 632 schools (anticipated)
Control: 635 schools (anticipated)

Based on 2017 data on the number of students and the share paid at each school, we anticipate an MDE on the Cal Grant claiming rate of 1.3 percentage point (95% confidence, 80% power, assuming analysis is run at the school level) for each treatment relative to the control.