NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Increasing the Take-Up of Cal Grant Awards Through Improved Notification Letters II
Initial registration date
November 29, 2018
November 30, 2018 10:15 AM EST
This section is unavailable to the public. Use the button below
to request access to this information.
University of California, Berkeley
Other Primary Investigator(s)
California Policy Lab/UC Berkeley
California Policy Lab/UC Berkeley
California Policy Lab/UC Berkeley
Additional Trial Information
Each year, over 150,000 California high school students receive letters notifying them that they qualify for Cal Grants, grant aid for college that is assigned based on family income and high school GPA. Historically, less than two-thirds claim their awards. The California Student Aid Commission (CSAC) believes this is in part because some students do not understand their eligibility.
In 2018-19, high school seniors who qualify for a Cal Grant will be randomly assigned (at the high school level) to receive one of four distinct letters, which vary in their language and in the information contained. Samples of the letters are included below.
Primary outcomes of interest include whether students create accounts in CSAC’s online portal, enroll in college, and claim their awards, as well as which type of college (community college vs. 4-year) they choose. Registration Citation
Davis, Charles et al. 2018. "Increasing the Take-Up of Cal Grant Awards Through Improved Notification Letters II." AEA RCT Registry. November 30.
In November, the California Student Aid Commission (CSAC) began mailing preliminary notification letters to notify students that they have qualified for a Cal Grant. These letters include information on the size of the Cal Grant award, instructions on how to claim the award, and some additional information on other potential sources of aid.
For the 2019-20 year, we have designed four variants of the notification letter. These vary in the language that they include and in the information they provide about college costs. A baseline letter will be compared to three variants:
1. The first variant will remove elements of the baseline letter designed to explicitly bolster the students’ sense of belonging in college. 2. The second variant will feature language designed to show receipt of the Cal Grant as a social norm. This letter will add a sentence to the baseline letter noting that many students went to the recipient’s own high school. 3. The third variant will be similar to the baseline letter, but will replace the information about the value of the Cal Grant with a table with customized information about the estimated cost and estimated aid at up to ten California public colleges and universities. The colleges will correspond to the ones that the student listed on his/her FAFSA to receive his/her financial information for use in financial aid calculations. CSAC will use the financial aid information in the FAFSA to populate these colleges’ net price calculators and generate a customized estimate of the full cost of attendance at the institution, the aid that a student can expect to receive if he/she enrolls, and the net cost that the student can expect to pay for their first year of attendance. CSAC is creating and mailing all communications, recording which version of each letter is sent to students at a given school. California Policy Lab researchers at the University of California, Berkeley, will conduct the data analysis.
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
(1) Creation of account on CSAC website
(2) Claiming of an award
(3) Enrollment in college
(4) Type of institution attended
Primary Outcomes (explanation)
(1) Did student create an account (Y/N); Date on which student created account
(2) Was an award paid to the student? (Y/N)
(3) Did the student enroll in college (Y/N); Full-time vs. Part-time enrollment.
(4) Possible institutions include: California Community College; Cal State University; University of California; private non-profit WASC-accredited college; private for-profit non-WASC accredited college.
Secondary Outcomes (end points)
(3) Ongoing claiming of Cal Grant awards beyond the first year of enrollment
Secondary Outcomes (explanation)
(1) Did student graduate or transfer from a two-year to a four-year college (Y/N); type of degree/certificate earned; time to completion.
(2) Re-enrollment after first semester, first year, and subsequent milestone points; credit accumulation at various milestone points.
(3) Did student claim Cal Grant in years following initial enrollment?
This study will employ a randomized design to test the causal effects of the different letter variants relative to the baseline. High schools will be randomly assigned to four groups, with all students at a given high school receiving the letter type assigned to their school.
We will look for differences among the groups in the number of students who register for a webgrants4students account, the time between receiving the letter and creating the account, whether students enroll in college and claim their award, and the types of institutions students attend. Additionally, we will look for differences in longer-term outcomes including persistence, graduation, and continued claiming of Cal Grants in subsequent years.
Experimental Design Details
Randomization took place on a computer in the CPL offices. CSAC provided a list of high schools at which at least one student was deemed eligible for a Cal Grant award in either the 2015-16 or 2016-17 academic years. CPL researchers returned the list with randomly generated treatment/control assignments, as well as assignments for schools that did not appear in the previous years’ data.
Randomization was done at the school level. Schools were divided into seven strata based on the number of notification letters sent to students from the school in 2015-16 and 2016-17 and share of those students on whose behalf grants were paid out in 2016-17 and 2017-18, respectively: (a) Schools at which 10 or more students received notification letters in 2016-17:
a. Schools at which 0-59% of those students received CalGrants (i.e., grants were paid out on their behalf). Number of schools = 288.
b. Schools at which 60-75% of those students received CalGrants. Number of schools = 860.
c. Schools at which 76% or more of students received CalGrants. Number of schools = 439.
(b) Schools at which less than 10 students received notification letters in 2016-17 but more than 10 received letters when aggregated across 2015-16 and 2016-17:
a. Schools at which 0-59% of those students (aggregating across 2015-16 and 2016-17) received CalGrants. Number of schools = 129.
b. Schools at which 60-75% of those students received CalGrants. Number of schools = 71.
c. Schools at which 76% or more of students received CalGrants. Number of schools = 34.
(c) Schools at which the aggregate number of letters sent in 2015-16 and 2016-17 is greater than zero but less than ten. Number of schools = 709.
(d) Schools that do not appear in the 2016 or 2017 data. Number of schools unknown. One quarter of schools within each stratum were randomly assigned to the control or to one of the three treatment conditions (probability = 1/4 for each).
Randomization was done at the school level.
Was the treatment clustered?
Sample size: planned number of clusters
Sample size: planned number of observations
Sample size (or number of clusters) by treatment arms
Treatment 1: 630 schools (anticipated)
Treatment 2: 633 schools (anticipated)
Treatment 3: 632 schools (anticipated)
Control: 635 schools (anticipated)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Based on 2017 data on the number of students and the share paid at each school, we anticipate an MDE on the Cal Grant claiming rate of 1.3 percentage point (95% confidence, 80% power, assuming analysis is run at the school level) for each treatment relative to the control.
INSTITUTIONAL REVIEW BOARDS (IRBs)
The Committee for the Protection of Human Subjects, CA, Health and Human Services Agency
IRB Approval Date
IRB Approval Number