Referrals to motivate take-up of educational opportunities

Participants in the referral experiment are selected through two channels and randomized into three groups. The first set of participants was selected through peer-to-peer referrals collected via an incentivized online survey targeting all 635 students who completed the last edition of the soft-skills course, referred to as “the seeds”, which included 166 students who were no longer enrolled in a degree at the partner university. This set of students will be randomly assigned to two groups:

- A treatment group (T1), which receives the invitation email to the course from the Office of International Relations and learns that they were referred to take the course by a peer. For students nominated by more than one seed, one seed is randomly selected and reported as the “referrer”.
- A “nominated” control group (T2), which receives the standard invitation email to the course from the Office of International Relations without being informed that they were referred to it by a peer.

Additionally, we selected a second group of participants among all students who were not nominated by the seeds. These students are assigned to the “random” control group (C), which also receives the standard invitation email to the course from the Office of International Relations. Online registration for the course will be open for one week, and students in the experimental sample will receive up to two reminder emails. Students will have seven weeks to complete the course.

2025-03-24

2025-03-31

The primary outcomes of interest are:
- Course take-up, which is defined as completing the online registration for the course during the enrollment window;
- Course completion, both conditional and unconditional on program take-up. A student is considered to have completed the course if they completed all 9 online sessions by the course completion deadline.

We will additionally use administrative data on seeds and course participants, as well as pre-existing survey data on the seeds, to answer the following questions:
- What are the predictors of the seeds’ likelihood of making at least one referral?
- Who gets a referral: how do nominated students compare to non-nominated students?
- How does restricting the choice of the seeds to one referral only affect the selection and diversity of referred students?
- Do referrals affect course take-up and/or completion by changing students’ beliefs about program effectiveness, self-determination, motivation, and/or social obligation?

Beliefs about (expected) program effectiveness and (expected) difficulty in completing the course by the deadline (as a proxy for self-determination and motivation) are elicited during the registration survey using a 10-point Likert scale.

Additionally, we included an open-ended question asking respondents what drove their decision to register for the course. We will assess treatment effects on the likelihood of providing an answer and utilize text analysis to analyze the responses.

This study builds upon an initial phase that targeted participants from a previous experiment conducted by the PI. We invited all students who had completed the previous edition of an online soft-skill course (N = 635) to submit up to four referrals, selecting peers to be invited to the new edition of the course. We refer to these students as the “seeds” for our experiment. Seeds received performance-based incentives for their referrals: for each referred student who completes the course, they will receive a ticket in two random draws for an iPad. Seeds were informed that all the students they referred to the course would be invited to participate and that they may learn that the seed referred them to the course. Additionally, seeds were asked to indicate which peer they would refer to the course if they were only allowed to recommend one. Seeds were given two weeks to complete the referral survey.

The first part of our experimental sample consists of all the students identified through peer-to-peer referrals submitted by the seeds. This set of students is randomly assigned to two groups:
- A treatment group (T1), which receives the invitation email to the course from the Office of International Relations and learns that they were referred to take the course by a peer. For students nominated by more than one seed, one seed is randomly selected and reported as the “referrer”;
- A “nominated” control group (T2), which receives the standard invitation email to the course from the Office of International Relations without being informed that they were referred to it by a peer.

Randomization is stratified based on the following characteristics: gender, GPA above or below the population median, and rural versus urban background. Additionally, we selected a second group of participants among all students who were not nominated by the seeds. These students are assigned to a “random” control group (C), which also receives the standard invitation email to the course from the Office of International Relations. Online registration for the course will be open for one week, and students in the experimental sample will receive up to two reminder emails. Students will have seven weeks to complete the course.

All students in the experimental sample will receive an invitation to participate in the course from the University’s Office of International Relations, which includes a personalized registration link. This link will allow us to track registrations. The registration form will also collect additional socio-demographic characteristics and students' prior beliefs about the course, which will enable us to explore the mechanisms underlying any effects of the referrals. Registered students will receive weekly emails with links to the course sessions, which will allow us to track course completion.

For the analysis of the effects of referrals on course take-up and completion, we will focus on two primary hypotheses and three secondary hypotheses. We will estimate the corresponding OLS regressions and present results from one-sided tests to improve our statistical power.

Primary hypotheses:
1. Learning about a peer-to-peer referral can increase course take-up or completion through learning or psychological mechanisms (holding fixed selection) (T1 vs T2);
2. Being nominated by a seed can increase course take-up or completion (T1+T2 vs C).

Secondary hypotheses:
1. If take-up or completion is statistically different between T1 and T2, being nominated can also increase course take-up or completion through selection (T2 vs C);
2. If take-up or completion is statistically different between T1 and T2, the effect of referrals may be larger when the comparison group consists of non-nominated students (T1 vs C);
3. If the effects of referrals are driven by selection on observables, there should be no residual effect after controlling for the selection on observables (T1+T2 vs C). To explore this mechanism, we will use matching and weighting methods to assess whether any effect of referrals is robust to accounting for observable characteristics.

Not available

Randomization done by statistical software.

Individual.

No

N/A

N = 2945 (number of students invited to register for the course)

Of the N = 445 students nominated by a seed, N_T1 = 222 will be randomly assigned to the treatment group (i.e., referrals), and N_T2 = 223 to the “nominated” control group. N_C = 2500 students selected at random among those who were not nominated by any seed will be assigned to the “random” control group (C).

Using the control-group course take-up from the previous experiment (p1 = 0.26), the minimum detectable effect from the comparison of T1 to T2, using a one-sided chi-squared test, is an 11-percentage-point increase. The minimum detectable effect from the comparison of T1+T2 to C, using a one-sided chi-squared test, is a 5.8-percentage-point increase.