Factors affecting university application decisions

The intervention will involve providing information and mentorship to secondary school students in the UK, with the treatment arms varying the content and nature of each.

Low-income students in the UK “undermatch”, attending lower-ranked universities despite good academic credentials and uniform tuition fees. This project looks into whether and how social exposure to individuals who attend high-ranked universities shapes university aspirations. Using administrative data, we document that school-to-university pipelines are persistent and dense; on average, 32% of students attend the modal university attended by students in the prior cohort at their school. These effects are more pronounced in economically disadvantaged schools, where students who attend pipeline universities are also more likely to undermatch. Additionally, students are more likely to attend a university their schoolmates have historically attended, even when they have offers from higher-ranked universities. In light of these patterns, we propose learning about university opportunities through schoolmates as one driver of undermatching. To obtain causal evidence and disentangle mechanisms of this type of social learning, we will run an RCT on ~2,000 students in the UK applying to university. Mechanisms we will test include the effectiveness of social exposure compared to standard information interventions, whether students respond more to information from individuals who are demographically similar, and the importance of coordination within friend groups.

The intervention will involve in-person workshops in secondary schools in the UK providing information about the university application process to participants, video clip anecdotes from current university students, and matching participants with current university student mentors to discuss life at university and the application process. Our treatment arms will vary the content and nature of the workshops and whether students are matched to mentors. The target population is Year 12 and/or Year 13 secondary school students in the UK in the year before they apply to university (typically October - January of Year 13). We plan on conducting the intervention in multiple waves in order to maximize the number of schools that can be included in the study. The first wave will take place in June and July 2024, the second wave will take place in September through November 2024, and the third wave will take place sometime between April through September 2025.

2024-07-03

2026-01-01

Short-run outcomes (collected at endline survey):
- Interest in mentorship / further exposure to researcher-assigned universities*
- Stated interest in applying to researcher-assigned universities*

Long-run outcomes (collected by schools after university applications are submitted):
- Applications / attendance to researcher-assigned universities*
- Applications / attendance to mentor's university

*Note: researcher-assigned universities are sets of universities that are suitable for the student based on their grade profiles (universities to which students with similar grades have attended based on national data). Researcher-assigned universities will exclude universities already familiar to the students such as the universities their parents and/or siblings have attended as well as universities that are commonly attended by students at their school.

We will look into heterogeneity of treatment effects on the primary outcomes by demographic similarity between participating students mentors (where demographics include socioeconomic status, ethnicity, gender, and location). For instance, we will examine whether female students matched with female mentors were more likely to apply to their mentor's university compared to female students matched to male mentors.

The experimental design consists of two treatment arms and a control group with randomization at the individual student level. All students will complete baseline and endline surveys before and after the intervention.

The intervention will be randomized at the student level and consist of three treatment arms and a control group. All participating students will attend in-person workshops led by school alumni, teachers, or other university or school-adjacent professionals and fill out a baseline survey ahead of the workshops about their university aspirations, and an endline survey at the end of the workshop. Based on their responses to the baseline survey as well as their demographics and academic records, participating students will be “assigned” to a set of academically suitable university-major pairs that they may lack exposure to. The endline survey will ask students to state universities from which they would like mentors or otherwise be further exposed to as an incentive for their responses. Students in the control group will attend an in-person workshop that includes generic information and application tips such as information about the university application process in the UK, timelines, and personal statement advice. Students in Treatment 1 will attend an in-person workshop providing generic information as in the control group, plus some national and school-specific statistics on applications and universities. Students in Treatment 2 will attend a workshop containing all the same content as Treatment 1. However, students in Treatment 2 will additionally watch video clips from current and/or former university students from the assigned university-major pair. The speakers in the video clips will discuss their experiences applying to and attending university. Treatment 2 is then split randomly into two arms; in Treatment 2B students will be assigned to mentors at universities of their choice while students in Treatment 2A will not be assigned to mentors.

Randomization will be conducted on a computer. Within each participating school, we will receive a list of participating students, and then randomly assign students to either the control arm, treatment 1, or treatment 2. Students in treatment 2 will then be randomized into 2A and 2B following the workshop and assigned mentors.

Individual, stratified by school

No

As the randomization is at the individual level, the number of clusters at the unit of randomization level is the same as the number of individuals. However, we will be sampling particular schools, and our central estimate is that we will recruit 20 schools.

Our central estimate is that we will have a sample size of 2000 individuals, but we may get fewer or more as we are still in the process of recruiting schools to the study. We present MDEs as a function of sample size below to give a sense of how power will depend on the sample size.

We plan to randomize students to workshops in fixed proportions: ¼ of the sample in each school that we recruit will be randomized to the control group and to treatments 1, 2A, and 2B respectively, so we will have 500 students in each under our target sample size of 2000 individuals. However, we may not be able to find mentor matches for as much as a quarter of the sample, in which case we will reduce the assignment of students to arm 2B and correspondingly increase the assignment of students to 2A.

Our primary outcomes are all binary / probability outcomes, so for power calculations we use the standard formula for the variance of a proportion. The minimum detectable effect size then depends on the baseline probability of the outcome and the sample size in each arm. We have limited information about the baseline probabilities for most primary outcomes, but we expect many to fall between 0.05 and 0.2. Below we present power under baseline probabilities of 0.05, 0.1, and 0.2. Our primary comparison compares the control group pooled with arm 1 against arm 2 (pooling 2A and 2B). For this comparison, at our central sample size estimate of 2000 individuals, with individual-level randomization, and given the baseline probabilities defined above, our minimum detectable effect sizes are: Baseline 0.2: 5.0 percentage points Baseline 0.1: 3.8 percentage points Baseline 0.05: 2.7 percentage points We will also compare each of the individual arms against each other. Power for these comparisons is smaller as we are comparing the 500 students in each arm against each other, as opposed to the 1000 students in each group that result once we pool arms as pre-specified above. For these comparisons, we have power of: Baseline 0.2: 7.1 percentage points Baseline 0.1: 5.3 percentage points Baseline 0.05: 3.9 percentage points The figure in the attached supporting documents and materials illustrates power under different assumptions about sample size and the baseline outcome level.