Factors affecting university application decisions

The intervention will involve providing information and mentorship to secondary school students in the UK, with the treatment arms varying the content and nature of each.

2024-07-03

2026-01-01

Short-run outcomes (collected at endline survey):
- Interest in mentorship / further exposure to researcher-assigned universities*
- Stated interest in applying to researcher-assigned universities*

Long-run outcomes (collected by schools after university applications are submitted):
- Applications / attendance to researcher-assigned universities*
- Applications / attendance to mentor's university

*Note: researcher-assigned universities are sets of universities that are suitable for the student based on their grade profiles (universities to which students with similar grades have attended based on national data). Researcher-assigned universities will exclude universities already familiar to the students such as the universities their parents and/or siblings have attended as well as universities that are commonly attended by students at their school.

We will look into heterogeneity of treatment effects on the primary outcomes by demographic similarity between participating students mentors (where demographics include socioeconomic status, ethnicity, gender, and location). For instance, we will examine whether female students matched with female mentors were more likely to apply to their mentor's university compared to female students matched to male mentors.

The experimental design consists of two treatment arms and a control group with randomization at the individual student level. All students will complete baseline and endline surveys before and after the intervention.

Not available

Randomization will be conducted on a computer. Within each participating school, we will receive a list of participating students, and then randomly assign students to either the control arm, treatment 1, or treatment 2. Students in treatment 2 will then be randomized into 2A and 2B following the workshop and assigned mentors.

Individual, stratified by school

No

As the randomization is at the individual level, the number of clusters at the unit of randomization level is the same as the number of individuals. However, we will be sampling particular schools, and our central estimate is that we will recruit 20 schools.

Our central estimate is that we will have a sample size of 2000 individuals, but we may get fewer or more as we are still in the process of recruiting schools to the study. We present MDEs as a function of sample size below to give a sense of how power will depend on the sample size.

We plan to randomize students to workshops in fixed proportions: ¼ of the sample in each school that we recruit will be randomized to the control group and to treatments 1, 2A, and 2B respectively, so we will have 500 students in each under our target sample size of 2000 individuals. However, we may not be able to find mentor matches for as much as a quarter of the sample, in which case we will reduce the assignment of students to arm 2B and correspondingly increase the assignment of students to 2A.

Our primary outcomes are all binary / probability outcomes, so for power calculations we use the standard formula for the variance of a proportion. The minimum detectable effect size then depends on the baseline probability of the outcome and the sample size in each arm. We have limited information about the baseline probabilities for most primary outcomes, but we expect many to fall between 0.05 and 0.2. Below we present power under baseline probabilities of 0.05, 0.1, and 0.2. Our primary comparison compares the control group pooled with arm 1 against arm 2 (pooling 2A and 2B). For this comparison, at our central sample size estimate of 2000 individuals, with individual-level randomization, and given the baseline probabilities defined above, our minimum detectable effect sizes are: Baseline 0.2: 5.0 percentage points Baseline 0.1: 3.8 percentage points Baseline 0.05: 2.7 percentage points We will also compare each of the individual arms against each other. Power for these comparisons is smaller as we are comparing the 500 students in each arm against each other, as opposed to the 1000 students in each group that result once we pool arms as pre-specified above. For these comparisons, we have power of: Baseline 0.2: 7.1 percentage points Baseline 0.1: 5.3 percentage points Baseline 0.05: 3.9 percentage points The figure in the attached supporting documents and materials illustrates power under different assumptions about sample size and the baseline outcome level.