Performance-Based Scholarship Demonstration

Last registered on May 26, 2017

Pre-Trial

Trial Information

General Information

Title
Performance-Based Scholarship Demonstration
RCT ID
AEARCTR-0001761
Initial registration date
May 26, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 26, 2017, 4:11 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
MDRC

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2008-07-01
End date
2015-12-31
Secondary IDs
Abstract
MDRC launched the Performance-Based Scholarship Demonstration in 2008 to test an innovative strategy for addressing two policy objectives: increasing the financial support available to low-income students, and creating an incentive for such students to complete their courses and make more timely progress toward degrees. The idea is to provide a supplement to existing federal and state financial aid that is contingent on enrolling in a minimum number of credit hours and making passing grades. The performance-based scholarships are paid directly to students (rather than to the colleges or universities they attend) in order to reward students for their progress and to allow them to make choices of how best to support their schooling. For some, this may mean buying books or paying for transportation to campus; for others, it may mean cutting back on work hours or hiring a babysitter for their children during finals week.

MDRC tested variations of the performance-based scholarship at eight colleges and one intermediary across six states. The program at each college was targeted to low-income students with high unmet need, based on the cost of attendance and gaps in state financial aid.
The Performance-Based Scholarship Demonstration was designed to produce rigorous evidence of the programs' impacts and to answer policy-relevant questions:

-- Do performance-based scholarships increase short- or long-term academic achievement?
-- How does variation in the amount and duration of scholarships affect academic outcomes? For example, how do the impacts of a $1,000 scholarship compare with those of a $2,000 scholarship?
-- For what target population do the scholarships have the greatest impacts?
-- What are the effects of bundling the scholarships with enhanced student services, such as advising and tutoring?

The results show that these scholarships improved students' academic progress during the program--effects that remained evident several years after the program ended. The effects on students' academic progress appear generally consistent across the different programs and student subgroups. In addition, one program targeted high school seniors and succeeded in increasing their matriculation in college, and three of the programs reduced students' dependency on loans. Most important, this evaluation finds that the programs modestly increased degree completion, measured after five years.

These results show that even relatively moderate investments in low-income students' education can have modest but long-lasting impacts on their academic outcomes. These findings may be especially relevant to states, institutions, and private scholarship providers seeking purposeful and efficient ways to give low-income students additional financial aid that can also help them succeed academically.

Registration Citation

Citation
Sommo, Colleen. 2017. "Performance-Based Scholarship Demonstration." AEA RCT Registry. May 26. https://doi.org/10.1257/rct.1761-1.0
Former Citation
Sommo, Colleen. 2017. "Performance-Based Scholarship Demonstration." AEA RCT Registry. May 26. https://www.socialscienceregistry.org/trials/1761/history/18049
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2008-07-01
Intervention End Date
2012-06-30

Primary Outcomes

Primary Outcomes (end points)
enrollment, credits earned, degree/certificate attainment
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Multiple colleges participated in this study, and several colleges had multiple campuses; more than 5,600 students received scholarships at two- and four-year institutions in six states. In California, students were recruited from four geographic areas throughout the state but could attend any college. Multiple cohorts of students were also recruited into the study. A cohort refers to a group of students entering at the beginning of a specific semester (for example, Borough of Manhattan Community College had a fall 2008 cohort, spring 2009 cohort, and fall 2009 cohort). In total, there were 45 unique cohort combinations by college/campus and geographic region. Individual students were randomly assigned to the program or control groups, and this random assignment was conducted separately for each unique combination of cohorts and campuses, or cohorts and geographic regions.

Because the scholarship dollars are limited, the demonstration can use a random assignment research design to compare the outcomes of students who receive a performance-based scholarship (on top of their regular financial aid) with a control group who would not receive the performance-based scholarship but would have access to other scholarships that would otherwise be available. MDRC is using variety of quantitative data sources, including transcript data, other administrative data, and surveys, to determine the impacts of the scholarships. Qualitative data from focus groups and interviews are being employed to understand why (or why not) the scholarship intervention had an effect on outcomes.
Experimental Design Details
Randomization Method
Randomization done by computer by MDRC.
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
11,613 students
Sample size: planned number of observations
11,613 students
Sample size (or number of clusters) by treatment arms
8 colleges plus Cash for College workshops across California. In all, there were 5,654 program group members and 6,477 control group members.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Yes
Data Collection Completion Date
November 01, 2015, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
11,613 students
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
11,613 students
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Performance-based scholarships have two main goals: to give students more money for college and to provide incentives for academic progress. They are designed to reduce the financial burden on low-income students and help them progress academically by offering financial aid contingent upon meeting pre-specified academic benchmarks. The scholarships are intended to cover a modest amount of students' educational costs during the semesters they are offered--generally between 15 and 25 percent of students' unmet financial need, the difference between students' calculated financial need to attend college and the financial aid they are awarded. The money is paid directly to students, on top of their existing federal and state need-based financial aid, and the students themselves decide how best to use the funds.

MDRC launched the Performance-Based Scholarship Demonstration in 2008 to evaluate the effectiveness of these scholarships for as broad a range of low-income students as possible, in a variety of settings, and with varying incentive structures. As such, the evaluation includes more than 12,000 students in institutions across six states to test different performance-based scholarship designs. Each program was developed for a different population of students and had a different scholarship structure; the scholarship amounts ranged from a few hundred dollars to $1,500 per term, depending in part on the benchmarks being tested. Institutions created performance-based scholarship programs tailored to what they perceived to be the specific needs of their students, by targeting the incentive, academic benchmarks, and in some cases additional services to address those needs.

Each of the six programs in the demonstration was evaluated using a randomized controlled trial--the highest standard of evidence for evaluation research. Students were randomly assigned by researchers either to receive only their usual financial aid package and services or to be eligible to receive supplemental financial aid and services in the form of a performance-based scholarship, contingent upon meeting the given academic benchmarks.

The results show that these scholarships improved students' academic progress during the program--effects that remained evident several years after the program ended. The effects on students' academic progress appear generally consistent across the different programs and student subgroups. In addition, one program targeted high school seniors and succeeded in increasing their matriculation in college, and three of the programs reduced students' dependency on loans. Most important, this evaluation finds that the programs modestly increased degree completion, measured after five years.

These results show that even relatively moderate investments in low-income students' education can have modest but long-lasting impacts on their academic outcomes. These findings may be especially relevant to states, institutions, and private scholarship providers seeking purposeful and efficient ways to give low-income students additional financial aid that can also help them succeed academically.
Citation
Mayer, Alexander, Reshma Patel, Timothy Rudd, Alyssa Ratledge. 2015. Designing Scholarships to Improve College Success: Final Report on the Performance-Based Scholarship Demonstration. New York: MDRC.

Reports & Other Materials