Increasing the Take-Up of Cal Grant Awards Through Improved Notification Letters
Last registered on June 13, 2018

Pre-Trial

Trial Information
General Information
Title
Increasing the Take-Up of Cal Grant Awards Through Improved Notification Letters
RCT ID
AEARCTR-0003031
Initial registration date
June 09, 2018
Last updated
June 13, 2018 8:37 PM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
University of California, Berkeley
Other Primary Investigator(s)
Additional Trial Information
Status
On going
Start date
2017-11-13
End date
2024-10-31
Secondary IDs
Abstract
Each year, over 150,000 California high school students receive letters notifying them that they qualify for Cal Grants, grant aid for college that is assigned based on family income and high school GPA. Less than two-thirds claim their awards. The California Student Aid Commission (CSAC) believes this is in part because some students do not understand their eligibility. This study will investigate whether a simplified letter, in terms of both format and the use of language that behavioral science research suggests would nudge students to take action, will induce higher take-up of the Grants and will consider how different letters impact students’ college choices decision making.

In 2017-18, students were randomly assigned (at the high school level) to receive CSAC’s usual notification letter or one of two variants designed with behavioral science principles in mind. Primary outcomes of interest include whether students create accounts in CSAC’s online portal, enroll in college, and claim their awards, as well as which type of college (community college vs. 4-year) they choose. Analysis of these outcomes will occur beginning in Spring 2018.
External Link(s)
Registration Citation
Citation
Rothstein, Jesse. 2018. "Increasing the Take-Up of Cal Grant Awards Through Improved Notification Letters." AEA RCT Registry. June 13. https://www.socialscienceregistry.org/trials/3031/history/30718
Experimental Details
Interventions
Intervention(s)
In November, the California Student Aid Commission (CSAC) began mailing preliminary notification letters to notify students that they have qualified for a Cal Grant. These letters include information on the size of the Cal Grant award, instructions on how to claim the award, and some additional information on other potential sources of aid. In 2017-18, instead of sending the same letter to all Cal Grant recipients, CSAC is sending three variants of this letter. One letter – the control letter –closely resembles the letters sent in previous years but has been updated to reflect this year’s Cal Grant award amounts. The other letters have been designed to be simpler, both in terms of the language used in the text, and in the way in which they are formatted. While these variants have been designed to be simpler, the research team has taken care to ensure all relevant information on the award and how to claim it is included in the letter. The two variants differ in that one includes language designed to “nudge” students into college attendance. Samples of the three letters are included below.

High schools were randomly assigned to one or another letter variant, so that all students at a school who appear to be Cal Grant eligible will receive the same letter but students at different schools may receive different variants.

CSAC is creating and mailing all communications, recording which version of each letter is sent to students at a given school. California Policy Lab researchers at the University of California, Berkeley, will conduct the data analysis.
Intervention Start Date
2017-11-13
Intervention End Date
2018-08-31
Primary Outcomes
Primary Outcomes (end points)
(1) Creation of account on CSAC website
(2) Claiming of an award
(3) Enrollment in college
(4) Type of institution attended
Primary Outcomes (explanation)
(1) Did student create an account (Y/N); Date on which student created account
(2) Was an award paid to the student?
(3) Did the student enroll in college (Y/N); Full-time vs. Part-time enrollment.
(4) Possible institutions include: California Community College; Cal State University; University of California; private non-profit WASC-accredited college; private for-profit non-WASC accredited college.
Secondary Outcomes
Secondary Outcomes (end points)
(1) Graduation
(2) Persistence
(3) Ongoing claiming of CalGrant awards beyond the first year of enrollment
Secondary Outcomes (explanation)
(1) Did student graduate or transfer from a two-year to a four-year college (Y/N); type of degree/certificate earned; time to completion.
(2) Re-enrollment after first semester, first year, and subsequent milestone points; credit accumulation at various milestone points.
(3) Did student claim Cal Grant in years following initial enrollment
Experimental Design
Experimental Design
This study employed a randomized control trial to test whether a letter designed using insights from behavioral science elicits a better response than the letter previously mailed to students who are eligible to receive a Cal Grant award. High schools were randomized into three groups, one control group and two treatment groups, with students at a given high school receiving the letter variant assigned to their school.

We will look for differences among the treatment groups and the control group in terms of the number of students who register for a webgrants4students account, the time between receiving the letter and creating the account, whether students enroll in college and claim their award, and the types of institutions students attend. Additionally, we will look for differences in longer-term outcomes including persistence, graduation, and continued claiming of Cal Grants in subsequent years.
Experimental Design Details
Not available
Randomization Method
Randomization took place on a computer in the CPL offices. CSAC provided a list of high schools at which at least one student was deemed eligible for a Cal Grant award in either the 2014-15 or 2015-16 academic years. CPL researchers returned the list with randomly generated treatment/control assignments, as well as assignments for schools that did not appear in the previous years’ data.

Randomization was done at the school level. Schools were divided into seven strata based on the number of notification letters sent to students from the school in 2014-15 and 2015-16 and share of those students on whose behalf grants were paid out in 2015-16 and 2016-7, respectively:
(a) Schools at which 10 or more students received notification letters in 2015-16:
a. Schools at which 0-59% of those students received CalGrants (i.e., grants were paid out on their behalf). Number of schools = 391.
b. Schools at which 60-75% of those students received CalGrants. Number of schools = 917.
c. Schools at which 76% or more of students received CalGrants. Number of schools = 233.
(b) Schools at which less than 10 students received notification letters in 2015-16 but more than 10 received letters when aggregated across 2014-15 and 2015-16:
a. Schools at which 0-59% of those students (aggregating across 2014-15 and 2015-16) received CalGrants. Number of schools = 163.
b. Schools at which 60-75% of those students received CalGrants. Number of schools = 59.
c. Schools at which 76% or more of students received CalGrants. Number of schools = 19.
(c) Schools at which the aggregate number of letters sent in 2014-15 and 2015-16 is greater than zero but less than ten. Number of schools = 613.
(d) Schools that do not appear in the 2015 or 2016 data. Number of schools unknown.

One third of schools within each stratum were randomly assigned to the control or to one of the two treatment conditions (probability = 1/3 for each).
Randomization Unit
Randomization was done at the school level.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
2,395 schools (anticipated)
Sample size: planned number of observations
100,000 (anticipated)
Sample size (or number of clusters) by treatment arms
Treatment 1: 795 (anticipated)
Treatment 2: 798 (anticipated)
Control: 802 (anticipated)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Based on 2015 data on the number of students and the share paid at each school, we anticipate an MDE on the Cal Grant claiming rate of 1.3 percentage point (95% confidence, 80% power, assuming analysis is run at the school level) for each treatment relative to the control.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
The Committee for the Protection of Human Subjects, CA, Health and Human Services Agency
IRB Approval Date
2017-10-06
IRB Approval Number
2017-050