Effectiveness of Peer-Assisted Learning

The intervention consists of (bi-) weekly peer-assisted learning sessions taught by senior peers in law or engineering. The tutees are first year students in engineering, law, or in some other programs that have an intro to law course such as economics (though it is possible that this might still change as the student organizations decide quite late for which extra courses they will provide tutoring sessions). The tutors receive a short training sessions on how to deliver these PAL-tutoring sessions. This usually consists of both training on didactics specific to PAL as well as a refresher with respect to content for the specific course.

One treatment arm will have tutoring sessions in smaller groups than the other treatment arm. The precise number will depend on the amount of tutees and tutors wanting to participate in the program. We expect that the small groups will be 4 to 5 tutees per tutor, while the large groups will have 8 to 10 tutees per tutor. The law program usually has one tutor per group, while the engineering program has two tutors per group. The level of oversubscription will also influence the size of the control group, and perhaps if there is no oversubscription, will determine whether there is a control group.

We might update these details once the tutoring program's details have been decided by the student organizations and once the treatment arms have been decided based on the number of tutees and tutors registering, but before the tutoring program starts and thus before outcome variables are collected. In this way, no p-hacking is possible, while still allowing me to demonstrate my initial intentions.

2024-10-14

2025-06-02

Grades on course for which the first year student wanted to enrol for PAL sessions

Grades are measured using administrative data from the university records.

Sense of belonging, measured using the same six-item scale as used in Dekker et al. (2023) and Meeuwissen et al. (2010)
Self-perceived understanding of course (asked in the post-test in a single question on a scale of 0 to 10)
Course-specific exam anxiety (asked in the post-test in a single question on a scale of 0 to 10)
Individual study time for the course (asked in the post-test in terms of average hours per week)

We might also consider long-term positive effects and spillovers to other courses conditional on finding significant and sizeable effects on the main course. We assume that long-term effects and spillovers will only occur if the tutoring is effect for the main course. Long term effects might however be published in a separate paper.

We might update these outcomes updated once the tutoring program's details have been decided by the student organizations and once the treatment arms have been decided based on the number of tutees and tutors registering, but before the tutoring program starts and thus before outcome variables are collected. In this way, no p-hacking is possible. For example, a small control group might imply that we test less hypotheses.

See above.

The student organizations motivate students to enroll for the tutoring sessions. They do so by going to the classes and advertising the program during their activities and on their social media. Interested students fill in a survey stating they want to participate in the tutoring program and whether they want to give the researchers their informed consent to later retrieve their student records from the university's administrative data. The same procedure is used for recruiting tutors.

After about one to two weeks (the precise date is yet to be determined by the different student organizations), enrolment possibilities closes. I then randomize the enrolled students into two or three different groups in negotiation with the student organizations. A first treatment group will consist of students being able to follow PAL sessions in a small group of 4 to 5 tutees per tutor. A second treatment group will consist of students being able to follow PAL sessions in a larger group of 8 to 10 tutees per tutor. If there is a sufficient level of oversubscription, remaining students will be allocated to the control group. Once first-year students are allocated to such groups, we randomly allocate tutors to the different treatment groups. We might also create different treatment arms by pre-test ability in the second semester, but this is conditional on having a sufficient power in the first semester.

We will update these details once the tutoring program's details have been decided by the student organizations and once the treatment arms have been decided based on the number of tutees and tutors registering, but before outcome variables in the first semester are collected. In this way, no p-hacking is possible.

Not available

Randomization will be stratified by gender, course and pre-test ability (if strata are sufficiently large).These details might change based on the actual enrolment numbers. For example the low number of female students in engineering might make stratification by gender difficult. Again, adjustments might be made and reported after baseline data has collected but before the intervention starts, such that no p-hacking is possible while still allowing me to demonstrate my initial intentions..

The unit of randomization is students, given that students in the same program can be allocated to different experimental arms.

No

The treatment is not clustered. Students are nested into different programs, but the level of randomization is the student.

1500 students. Over the two semesters, the law PAL program had about 1100 students in the past year. The engineering PAL program had about 400 students. This varies quite a lot per year, because interest in the program and the amount of PAL programs offered varies.

This will depend on the level of oversubscription, which might differ between PAL programs for the different courses. The Law PAL programs had a significant level of oversubscription each year. We hope to be able to allocate about 1/4th or 1/3rd of the enrolled students into the control group, though again this will be negotiated with the student organization once the registration is over. The engineering PAL program only has some minor oversubscription in some years. If we are not able to create a control group, we will still compare the outcome variables between the two randomized treatment groups.

Assuming 1500 students and 1/4th of the student in the control group and an equal allocation of students between the two treatment arms, we have the following MDE: Treatment 1 + 2 (n=1125) vs control (n=375) using a two-sided comparison of two independent means with alpha = 0.1 and power = 0.8 with G*Power 3.1.9.7. --> MDE = 0.1483 d Treatment 1 (n=562) vs treatment 2 (n=563) using a two-sided comparison of two independent means with alpha = 0.1 and power = 0.8 with G*Power 3.1.9.7. --> MDE = 0.1484d