Experimental Design
The evaluation will use individual random assignment design, in which eligible applicants will be assigned at random to the Wallin group, eligible for the program, or a control group, not eligible for the program but eligible to apply for existing scholarships in the community. Random assignment will occur within blocks, as described further below. This blocking may improve the precision of the impact estimator. It also enables the study of variation in program effects across programs (and blocks), since it enables unbiased estimation of program effects for each type of program.
The random assignment process will work as follows: in the fall before the Wallin Scholars Program opens their application process, Wallin Educational Partners and partner high school staff will share information about the Wallin Scholars Program and the RCT with prospective applicants. While on-the-ground recruitment efforts will be conducted like in years past, the application process will be simplified to expand the pool of recruitment. Language about the RCT will be included in information publicizing the program.
If students are interested in the program, they will complete an application. Embedded within the application, applicants will fill out baseline information, an informed consent form, and additional application materials.
Students complete the forms online during the open application period. Upon conclusion of each program’s application cycle, students will then be divided into groups, or blocks, based on whether they fit the typical Wallin Scholar profile (traditional track) or not (pilot track), as determined by Wallin staff criteria. Within these two blocks, students may be divided further based on other characteristics, such as type of program (2- versus 4-year), and other characteristics (e.g., race/ethnicity, field of study, etc.) in order to satisfy scholarship donor preferences. Within each block, students will be assigned, using MDRC’s random assignment system, to the program or control group. The program group will have access to the Wallin Scholar Program and will be sent an offer of the scholarship and services. The control group will not be offered the Wallin Scholars Program, but they will be able to pursue scholarships available to all students in the community. Students in the control group will not receive Wallin services, to prevent crossovers and ensure appropriate treatment contrast in the study. If a student from the program group declines to participate in the Wallin Scholars Program for any reason, a student from the control group may be randomly selected from the same block to be re-assigned to the program group to ensure as many students are served as possible.