Closing Gaps in Higher Education Trajectories: The Effect of Targeted Information and Mentorship

Last registered on August 16, 2021

Pre-Trial

Trial Information

General Information

Title
Closing Gaps in Higher Education Trajectories: The Effect of Targeted Information and Mentorship
RCT ID
AEARCTR-0007303
Initial registration date
April 14, 2021
Last updated
August 16, 2021, 10:56 PM EDT

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
Centre for Economics Performance (LSE), VATT Institute for Economic Research

Other Primary Investigator(s)

PI Affiliation
Harvard Graduate School of Education
PI Affiliation
Fundación Luksic

Additional Trial Information

Status
In development
Start date
2021-04-14
End date
2022-09-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This document describes the pre-analysis plan to evaluate the effects of two complementary interventions designed to improve students' understanding of the postsecondary education opportunities they have available. These interventions focus on students enrolled in vocational high schools in Chile. Our sample consists of 210 schools organized in 160 school networks. To evaluate the interventions, high schools will be randomly assigned (at the school network level) into three groups: a pure a control group, an information-only treatment group, and a combination of information and mentoring treatment group. Students in any of the treatment groups will receive a personalized information package describing funding opportunities and the benefits of attending higher education. The mentoring treatment will be randomly assigned to four students per class. The design of the intervention includes features that allow us to study the spillovers of the mentoring program on the social network of treated individuals. With this last part of the experiment, we aim to understand to which extent social spillovers could be used to design more efficient and effective interventions to allow students to make informed decisions about their post-secondary education trajectories. This pre-analysis plan provides some background information on the intervention, outlines the study design, and describes the inference and estimation procedures.
External Link(s)

Registration Citation

Citation
Barrios Fernández, Andrés, Josefina Eluchans Errázuriz and Fernanda Ramirez-Espinoza. 2021. "Closing Gaps in Higher Education Trajectories: The Effect of Targeted Information and Mentorship." AEA RCT Registry. August 16. https://doi.org/10.1257/rct.7303-2.0
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
The RCT will randomly assign a third of the high school networks to the control group, a third to an information-only treatment group, and a third to a combination of information and mentoring treatment group. All interventions are focused on high school senior students enrolled in schools that offer a vocational track. All the students enrolled in treated schools will receive an information package either addressed to their homes when possible or sent through their schools when the address is not available/unreachable by the postal service (e.g. rural addresses). The package will provide information on three central aspects:

(1) Funding opportunities.
(2) Important variables to consider when choosing higher education programs.
(3) Steps they need to follow to apply for funding and to higher education.

Since all the students in our sample come from relatively disadvantaged settings, all of them are eligible for public funding. Thus, the information package will highlight the fact that, if they decide to enroll in higher education, they will receive generous support from the government.

The information package will also provide statistics on the benefits of attending higher education in terms of employment and earnings. However, considering that employment and wages greatly depend on the quality of the match between student and major-institution, the information package will also highlight the heterogeneity that exists in these variables across different fields and institutions. Students will be provided with links to official sources that offer tools to compare programs and institutions along with different variables including the labor market performance of recent graduates.

Finally, the information package will contain information on applications to funding and to higher education. The material will stress important deadlines and the requirements to apply to different types of funding and institutions. As in the previous case, it will also direct students to official sources providing additional details.

The package will contain three information pieces. First, an informative brochure that will contain the same information for everyone. Second, a calendar highlighting important deadlines and providing some guidelines to students who want to apply to higher education. Finally, a personalized letter that will reinforce the information in the package, customized based on typical higher education trajectories followed by students in that high school in previous years. Customization will focus on the Labor market outcomes in the four areas of study in which students from their same high school (in previous years) were more interested.

On top of the information package, 4 students of each last-year class in high schools allocated to the second treatment group will receive additional support from mentors. Mentors will be psychologists and school counselors who will contact the selected group of students 9 times between August and December. Each contact will have a specific goal in mind, but the general idea is to discuss with students their plans after completing high school. Apart from providing information about higher education opportunities, mentors will be available to answer any questions that students may have about different aspects of higher education (e.g., applications, funding and admissions requirements, important deadlines, variables to consider when deciding whether to enroll and where to enroll in higher education, etc). All the information provided by mentors will come from official sources. Finally, mentors will have 4 sessions with groups of 4 students, to cover topics like deciding what to do after high school and navigating the financial aid and application system. Although we will try to implement these sessions in person when possible, some of them might have to be done online due to Covid-19 restrictions. To make sure that mentoring affects students through either mentor interaction or peer interactions, we will make sure that mentors do not interact with other students in their school's visits and that mentorship materials are not shared with the school or other students.

Finally, students in the control group will no receive either the information package or the mentorship program. However, like the rest of the students in our project, they will answer an entry and exit survey.
Intervention Start Date
2021-08-16
Intervention End Date
2022-01-31

Primary Outcomes

Primary Outcomes (end points)
1. Understanding of the financial aid system (see next point for additional details).
2. Understanding of the higher education system (see next point for additional details).
Primary Outcomes (explanation)
1. Understanding of the financial aid system: requisites, available funding, application procedures.
2. Understanding of the higher education system:
2.1 Application procedures: requisites to apply to different types of institutions.
2.2 Average labor market outcomes: employment, wages.
2.3 Heterogeneity in the system: graduation rates, over-duration, employment, earnings.

Secondary Outcomes

Secondary Outcomes (end points)
I. Postsecondary education trajectories:
1. Applies to funding for higher education.
2. Applies to higher education.
3. Enrolls in higher education.
4. Characteristics of the institutions and programs in which they enroll: retention rates, over-duration, accreditation, expected earnings, expected employability.

II. Higher Education Aspirations
1. Desire of continuing studying after high school graduation.
2. Self-efficacy beliefs
3. Characteristics of the institutions and programs in which they wish to enroll: retention rates, over-duration, accreditation, expected earnings, expected employability.
Secondary Outcomes (explanation)
In addition to study differences in these outcomes, we plan to study heterogeneity along two important dimensions:
- Both treatments: baseline knowledge of the system, academic potential, and students' gender.
- Mentorship treatment: the size of the spillovers depending on the social distance to treated individuals.

Experimental Design

Experimental Design
The RCT will randomly assign a third of the high school networks to the control group, a third to an information-only treatment group, and a third to a combination of information and mentoring treatment group. The treatments aim to affect the understanding that senior high school students have of their higher education opportunities and, as a consequence, their aspirations and post-secondary education trajectories. Randomization will be done in an office by a computer. Schools in the control group will not be treated. Senior students enrolled in schools in any of the treated groups will receive an information package. In addition, 4 randomly chosen students per class in the second treatment group will receive on top of the information package a mentorship program. The effects of the mentorship program will be measured on both mentored students and also on their classmates. Randomly assigning students will allow us to test for spillover effects and if there is any heterogeneity in these spillovers.
Experimental Design Details
Not available
Randomization Method
Randomization done in the office by a computer.
Randomization Unit
School network
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
160 school networks
Sample size: planned number of observations
21,000 pupils.
Sample size (or number of clusters) by treatment arms
- Control: 53 school networks.
- Information-only treatment: 54 school networks (personalized information send to all senior students).
- Information and mentoring program: 53 school networks (personalized information send to all senior students and mentorship program allocated to highly connected individuals within each class).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Unfortunately, we do not have complete information to make accurate power calculations for our outcomes measuring how well students understand their funding and higher education opportunities at the moment. The best we can do is to rely on figures reported by previous studies in Chile. We base the following power analyses on the results of Hastings et al. (2016). The survey used in this paper was applied to students registered for the university admission exam (PSU) who accepted an email invitation to answer it. Our study focuses on schools in which an important share of the students do not even register for the PSU. Even those who take the PSU perform well below those in the sample of Hastings et al. (2016). For instance, while the average score of students in Hastings et al. (2016) was 539, the average performance of students from schools in our sample in 2018 was 439 (this is a difference of approximately 1SD). Since we focus on students from particularly disadvantaged settings, their knowledge of the system is likely to be lower than the one in Hastings et al. (2016). Having these caveats in mind, we can assume an error in the expected earnings Benchmark of between 0.43 to 0.50. Assuming an intercluster correlation of 0.15, these figures imply we would be able to identify effects above 0.17 at a significance level of 5% and with a power of 80%. When studying the effect of the mentorship program only on those receiving that treatment, then the minimum detectable effect increases to 0.22. Below, a list of binary outcomes that we will study, their means in previous cohorts of schools in our sample, and the minimum detectable effect in percentage points (power 80%, significance 95%). All the outcomes are measured at the individual level, and since they are reported in administrative records we can use previous cohorts to make very precise power calculations. 1. Information-only intervention: • Application to funding: 64% (mean), 7.4 pp (MDE). • Registration in university admission exam: 87% (mean), 5.4 pp (MDE). • Enrollment in higher education: 43% (mean), 7 pp (MDE). • Enrollment in university: 12% (mean), 3.8 pp (MDE) 2. Information + mentorship on treated students (individuals receiving the mentorship treatment and their controls): • Application to funding: 64% (mean), 8.9 pp (MDE). • Registration in university admission exam: 87% (mean), 6.3 pp (MDE). • Enrollment in higher education: 43% (mean), 8.8 pp (MDE). • Enrollment in university: 12% (mean), 5.5 pp (MDE) Our ability to detect effects on the connections of students receiving the mentorship treat- ment depends on the number of close connections included in the analyses. When focusing only on the closest connection the MDE is the same described in 2. While we include more individuals, the MDE starts to approach the ones described in 1.
IRB

Institutional Review Boards (IRBs)

IRB Name
Aalto University Research Ethical Comittee
IRB Approval Date
2021-04-12
IRB Approval Number
N/A
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information