Wallin Scholars Randomized Controlled Trial

Last registered on December 20, 2024

Pre-Trial

Trial Information

General Information

Title
Wallin Scholars Randomized Controlled Trial
RCT ID
AEARCTR-0015067
Initial registration date
December 19, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 20, 2024, 2:41 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
MDRC

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2025-03-01
End date
2031-09-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
For over 30 years, Wallin Education Partners has operated the Wallin Scholars Program, a program that provides financial aid, advising, and career supports to Minnesota high school seniors from low-income backgrounds as they transition to attending 4-year and 2-year colleges. The study will use a randomized controlled trial (RCT) design to estimate the causal effect of the opportunity to participate in the Wallin Scholars program. The program contains three main components: one-on-one advising support; financial aid for college and college-related expenses; and access to career supports such as internships and networking opportunities. Access to the program is contingent on being an actively enrolled student at an eligible college and remaining engaged with Wallin program staff and advising.  

The study plans to enroll approximately 1,600 students into two randomly assigned experimental arms, a program group and a control group, across two cohorts, fall 2025 and fall 2026. To answer whether the Wallin Scholars Program improves academic and career outcomes, the team will estimate the average (across individuals) intent-to-treat (ITT) effect of the program on retention in college and degree attainment through five years after random assignment. The program will also follow outcomes related to employment and earnings through administrative data sources and student surveys.  
External Link(s)

Registration Citation

Citation
Sommo, Colleen. 2024. "Wallin Scholars Randomized Controlled Trial." AEA RCT Registry. December 20. https://doi.org/10.1257/rct.15067-1.0
Experimental Details

Interventions

Intervention(s)
The Wallin Scholars Program includes three main components: advising supports, financial supports, and career supports.

Advising supports: After being accepted into the Wallin Scholars Program, scholars are immediately matched with a trained advisor, employed by Wallin, to receive one-on-one advising support. Scholars meet with advisors monthly during their first year in the program and then consistently, in an advisor-scholar agreed upon cadence, in subsequent years.

Advisors are trained to facilitate conversations that relate to four dimensions of scholar success including:

Degree planning and academic support: Advisors facilitate conversations to help scholars understand the requirements of their degree program, improve time management, and learn study skills.

Financial aid: Advisors and scholars discuss the financial aid process including the disbursement of Wallin funds. Scholars work to develop and demonstrate knowledge and skills required for aligning degree completion with financial planning.

Career development: Advisors and scholars discuss how scholars’ degree program and major relate to specific career and internship opportunities including those offered by Wallin. Advisors help scholars gain the knowledge and skills to access and utilize relevant career and internship resources on campus and in their communities. Advisors may also direct scholars to utilize Wallin career supports such as networking events, career fairs, and internship opportunities.

Personal support: Advisors and scholars discuss how to access and utilize relevant resources on campus or in the community for mental and physical health and well-being. Additionally, advisors can provide generalized support and listen to scholars as they discuss challenges that they are having either with their studies or with external issues that might be affecting their academic abilities.

Financial supports: The Wallin Scholars Program provides a last-dollar scholarship to students spread over their four years of college. Four-year Scholars are eligible for up to $16,000 over four years. Funds are usually disbursed evenly over the four years of school, but advisors have the discretion to request a different disbursement cadence depending on a Scholar’s individual circumstances. Two-year Scholars are eligible to receive up to $6,000 for those who attend a partner community college through the Opportunity Pathway.

Career supports: Wallin Scholars hosts career fairs and workshops that are available to Scholars. Wallin also connects Scholars to community and career opportunities through their network of partners, including many of their donors. Wallin scholars are also eligible to apply to several paid internship and project opportunities through the Wallin-operated UpTurnships program.
Intervention Start Date
2025-04-01
Intervention End Date
2030-06-30

Primary Outcomes

Primary Outcomes (end points)
Initial enrollment and persistence in college, degree attainment
Primary Outcomes (explanation)
Persistence: Persistence will be defined as enrollment in any college at years 1-3 post-random assignment.

Degree attainment: Degree attainment will be defined as receipt of either an associate or bachelor’s degree, evaluated in years 4+.

Secondary Outcomes

Secondary Outcomes (end points)
College credits earned, participation in career preparation activities, graduate school enrollment and completion, earnings, employment rates, sector of employment, self-reported student loan debt, personal savings, sense of economic well-being
Secondary Outcomes (explanation)
College credits earned, graduate school enrollment and completion, and earnings and employment will be obtained from administrative data sources.

Data on participation in career preparation activities, student loan debt, savings, and well-being will be obtained from the student surveys.

Experimental Design

Experimental Design
The evaluation will use individual random assignment design, in which eligible applicants will be assigned at random to the Wallin group, eligible for the program, or a control group, not eligible for the program but eligible to apply for existing scholarships in the community. Random assignment will occur within blocks, as described further below. This blocking may improve the precision of the impact estimator. It also enables the study of variation in program effects across programs (and blocks), since it enables unbiased estimation of program effects for each type of program.

The random assignment process will work as follows: in the fall before the Wallin Scholars Program opens their application process, Wallin Educational Partners and partner high school staff will share information about the Wallin Scholars Program and the RCT with prospective applicants. While on-the-ground recruitment efforts will be conducted like in years past, the application process will be simplified to expand the pool of recruitment. Language about the RCT will be included in information publicizing the program.

If students are interested in the program, they will complete an application. Embedded within the application, applicants will fill out baseline information, an informed consent form, and additional application materials.

Students complete the forms online during the open application period. Upon conclusion of each program’s application cycle, students will then be divided into groups, or blocks, based on whether they fit the typical Wallin Scholar profile (traditional track) or not (pilot track), as determined by Wallin staff criteria. Within these two blocks, students may be divided further based on other characteristics, such as type of program (2- versus 4-year), and other characteristics (e.g., race/ethnicity, field of study, etc.) in order to satisfy scholarship donor preferences. Within each block, students will be assigned, using MDRC’s random assignment system, to the program or control group. The program group will have access to the Wallin Scholar Program and will be sent an offer of the scholarship and services. The control group will not be offered the Wallin Scholars Program, but they will be able to pursue scholarships available to all students in the community. Students in the control group will not receive Wallin services, to prevent crossovers and ensure appropriate treatment contrast in the study. If a student from the program group declines to participate in the Wallin Scholars Program for any reason, a student from the control group may be randomly selected from the same block to be re-assigned to the program group to ensure as many students are served as possible.
Experimental Design Details
Not available
Randomization Method
Randomization done in office by computer (MDRC random assignment system)
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
0
Sample size: planned number of observations
1600
Sample size (or number of clusters) by treatment arms
600 Program, 1000 control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For the expected sample size of 1,600 students, the minimum detectable effect (MDE) on enrollment or college completion (or any given binary outcome) is about 6 percentage points. This assumes a mean value for the control group of 50 percent and thus is a conservative estimate. The Minimum detectable effect size is .122 standard deviation units. Although exploratory at this point, the MDE for annual earnings can also be calculated and depends on the expected variance of earnings in the absence of the program, which tends to increase as the follow-up period increases. If the standard deviation of annual earnings ranges from about $15,000 to about $20,000 (based on data from the recent long-term follow-up of the ASAP Ohio program), the MDE for earnings will range from $1,845 to $2,460. Assuming annual earnings in the first few years after college of about $50,000,4 these MDEs represent increases of 3.7 percent to 4.9 percent.
IRB

Institutional Review Boards (IRBs)

IRB Name
MDRC IRB
IRB Approval Date
2024-10-17
IRB Approval Number
2174392-1
Analysis Plan

Analysis Plan Documents

Wallin Scholars RCT Analysis Plan

MD5: 07ebf64264a03de69f889e3c3a0b0a11

SHA1: b537b15cee4494dfbe00463990853e697c4a9dfc

Uploaded At: December 19, 2024