|
Field
Experimental Design (Public)
|
Before
The RCT contemplates randomly assigning 80 high schools to the control group, 80 high schools to an information-only treatment group, and 80 high schools to a combination of information and mentoring treatment group. The treatments aim to affect the understanding that senior high school students have of their higher education opportunities and as a consequence their post-secondary education trajectories. Randomization will be done in the office by a computer. Schools in the control group will not be treated. Senior students enrolled in schools in any of the treated groups will receive an information package. In addition, 10% of the students in the second treatment group will receive on top of the information package a mentorship program. These students will be selected according to a measure of network centrality and academic potential. The criteria used to assign students to the mentorship program will depend on observables. The effects of the mentorship program will be measured on both mentored students, but also on their classmates.
|
After
The RCT will randomly assign a third of the high school networks to the control group, a third to an information-only treatment group, and a third to a combination of information and mentoring treatment group. The treatments aim to affect the understanding that senior high school students have of their higher education opportunities and, as a consequence, their aspirations and post-secondary education trajectories. Randomization will be done in an office by a computer. Schools in the control group will not be treated. Senior students enrolled in schools in any of the treated groups will receive an information package. In addition, 4 randomly chosen students per class in the second treatment group will receive on top of the information package a mentorship program. The effects of the mentorship program will be measured on both mentored students and also on their classmates. Randomly assigning students will allow us to test for spillover effects and if there is any heterogeneity in these spillovers.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
- Control: 80 schools.
- Information-only treatment: 80 schools (personalized information send to all senior students).
- Information and mentoring program: 80 schools (personalized information send to all senior students and mentorship program allocated to highly connected individuals within each class).
|
After
- Control: 53 school networks.
- Information-only treatment: 54 school networks (personalized information send to all senior students).
- Information and mentoring program: 53 school networks (personalized information send to all senior students and mentorship program allocated to highly connected individuals within each class).
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
Unfortunately, we do not have complete information to make accurate power calculations for our outcomes measuring how well students understand their funding and higher education opportunities at the moment. The best we can do is to rely on figures reported by previous studies in Chile. We base the following power analyses on the results of Hastings et al (2016). The survey used in this paper was applied to students registered for the university admission exam (PSU) who accepted an email invitation to answer it. Our study focuses on schools in which an important share of the students do not even register for the PSU. Even those who take the PSU perform well below those in the sample of Hastings et al (2016). For instance, while the average score of students in Hastings et al (2016) was 539, the average performance of students from schools in our sample in 2018 was 439 (this is a difference of approximately 1SD). Since we focus on students from particularly disadvantaged settings, their knowledge of the system is likely to be lower than the one in Hastings et al (2016). Having these caveats in mind, we can assume an error in expected earnings ((Expected − Benchmark)/Benchmark) of between 0.43 to 0.50. Assuming an intercluster correlation of 0.15, these figures imply we would be able to identify effects above 0.17 at a significance level of 5% and with a power of 80%. When studying the effect of the mentorship program only on those receiving that treatment (i.e., around 10% of a cohort), then the minimum detectable effect increases to 0.22.
Below, a list of binary outcomes that we will study, their means in previous cohorts of schools in our sample, and the minimum detectable effect in percentage points (power 80%, significance 95%). All the outcomes are measured at the individual level, and since they are reported in administrative records we can use previous cohorts to make very precise power calculations.
A. Information-only intervention:
(1) Application to funding: 64% (mean), 6 pp (MDE).
(2) Registration in university admission exam: 87% (mean), 5 pp (MDE).
(3) Enrollment in higher education: 43% (mean), 5.4 pp (MDE).
(4) Enrollment in university: 12% (mean), 3.4 pp (MDE)
B. Information + mentorship on treated students (individuals receiving the mentorship treatment and their controls):
(1) Application to funding: 64% (mean), 8.7 pp (MDE).
(2) Registration in university admission exam: 87% (mean), 6.2 pp (MDE).
(3) Enrollment in higher education: 43% (mean), 8.7 pp (MDE).
(4) Enrollment in university: 12% (mean), 5.9 pp (MDE)
(*) Our ability to detect effects on the connections of students receiving the mentorship treatment depends on the number of close connections included in the analyses. When focusing only on the closest connection the MDE is the same described in B. While we include more individuals, the MDE starts to approach the ones described in A.
|
After
Unfortunately, we do not have complete information to make accurate power calculations for
our outcomes measuring how well students understand their funding and higher education
opportunities at the moment. The best we can do is to rely on figures reported by previous
studies in Chile. We base the following power analyses on the results of Hastings et al.
(2016). The survey used in this paper was applied to students registered for the university
admission exam (PSU) who accepted an email invitation to answer it. Our study focuses on
schools in which an important share of the students do not even register for the PSU. Even
those who take the PSU perform well below those in the sample of Hastings et al. (2016). For
instance, while the average score of students in Hastings et al. (2016) was 539, the average
performance of students from schools in our sample in 2018 was 439 (this is a difference of
approximately 1SD). Since we focus on students from particularly disadvantaged settings,
their knowledge of the system is likely to be lower than the one in Hastings et al. (2016).
Having these caveats in mind, we can assume an error in the expected earnings Benchmark
of between 0.43 to 0.50. Assuming an intercluster correlation of 0.15, these figures imply we would be able to identify effects above 0.17 at a significance level of 5% and with a power of 80%. When studying the effect of the mentorship program only on those receiving that treatment, then the minimum detectable effect increases to 0.22.
Below, a list of binary outcomes that we will study, their means in previous cohorts of schools in our sample, and the minimum detectable effect in percentage points (power 80%, significance 95%). All the outcomes are measured at the individual level, and since they are reported in administrative records we can use previous cohorts to make very precise power calculations.
1. Information-only intervention:
• Application to funding: 64% (mean), 7.4 pp (MDE).
• Registration in university admission exam: 87% (mean), 5.4 pp (MDE).
• Enrollment in higher education: 43% (mean), 7 pp (MDE).
• Enrollment in university: 12% (mean), 3.8 pp (MDE)
2. Information + mentorship on treated students (individuals receiving the mentorship
treatment and their controls):
• Application to funding: 64% (mean), 8.9 pp (MDE).
• Registration in university admission exam: 87% (mean), 6.3 pp (MDE).
• Enrollment in higher education: 43% (mean), 8.8 pp (MDE).
• Enrollment in university: 12% (mean), 5.5 pp (MDE)
Our ability to detect effects on the connections of students receiving the mentorship treat- ment depends on the number of close connections included in the analyses. When focusing only on the closest connection the MDE is the same described in 2. While we include more individuals, the MDE starts to approach the ones described in 1.
|