Back to History Current Version

Live streaming of university lectures

Last registered on April 22, 2019

Pre-Trial

Trial Information

General Information

Title
Live streaming of university lectures
RCT ID
AEARCTR-0004107
Initial registration date
April 16, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 22, 2019, 11:33 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Geneva

Other Primary Investigator(s)

PI Affiliation
University of Geneva

Additional Trial Information

Status
Completed
Start date
2017-01-02
End date
2018-06-30
Secondary IDs
Abstract
First year bachelor students in Economics&Management at the University of Geneva (CH) are offered access to a live streaming platform allowing to attend on-line the lectures of many of their compulsory courses. Access to the platform is randomized both across students and over weeks of the term, so that the same student could attend the classes online in some weeks but not others. Furthermore, we construct a mapping of all questions in the final exams to the weeks of the term in which the material was covered. Thus, we can exploit variation both across and within students and weeks. Classroom attendance is always available to all students, allowing us to study students' choices of attendance mode.
We examine the experimental effects on in-class attendance and exam grades.
External Link(s)

Registration Citation

Citation
Laurent-Lucchetti, Jeremy and Michele Pellizzari. 2019. "Live streaming of university lectures." AEA RCT Registry. April 22. https://doi.org/10.1257/rct.4107-1.1
Former Citation
Laurent-Lucchetti, Jeremy and Michele Pellizzari. 2019. "Live streaming of university lectures." AEA RCT Registry. April 22. https://www.socialscienceregistry.org/trials/4107/history/197160
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
During two academic semesters, Spring 2017 (Feb-May) and Fall 2017 (Sep-Dec), we live-streamed all the lectures of 8 compulsory courses of the bachelor program in Economics&Management at the University of Geneva. We chose these courses as they were taking place in the largest auditorium of the school, the only one we could equip with the necessary technology. The official maximum capacity of the auditorium is 450 seats. Only regular lectures and not assistants' sessions were streamed.
The experiment involved the courses of Introduction to Macroeconomics, Probability and Statistics and Human Resource Management in the Spring semester 2017. All three classes were taught in French. Probability and Statistics was compulsory not only for students in the Economics and Management (EM) bachelor but also for those in International Relations (IR) and this is the reason why it is the most numerous of the three courses taught in this semester.
The following term (Fall semester 2017), we experimented with three additional courses: Introduction to Microeconomics, Mathematics and Introduction to Management. Introduction to microeconomics was taught in three parallel sections, two in French (one of which entirely dedicated to IR students) and one in English. Each section was taught by a different professor but all three were fully harmonized, i.e. covered exactly the same content, used the same problem sets, the same slides and lecture notes (only translated in the two languages). All students in the three sections took the exact same exam. Students could take the exam in the language of their choice. Students were assigned a section at the beginning of the term and were not allowed to switch. IR students were all assigned to their dedicated French section, whereas EM students could choose the French or English section but were asked to stick to their choice for the entire term. Mathematics was also compulsory for both EM and IR students.

All these courses are organized according to the same model, with one lecture and one TA session per week, each of them taking place in a time slot of 2 hours, with 90 minutes of actual teaching. Usually the teaching sessions (both the lectures and the TAs) start at 15-past the hour, run for a first part of 45 minutes, allow a 15 minutes break and run for another 45 minutes.

The students involved in the experiment in the two semesters are for the most part different students. In the Spring 2017 we mostly have first year students (first enrolled in September 2016) attending their second semester of compulsory courses. In the Fall 2017 most students have just enrolled (in September 2017) and are attending their first semester of compulsory courses. A few students are present in both semesters.

The streaming platform was accessible 10 minutes before the lecture started and until 10 minutes after the end. The video showed the lecturer and the projector screen. In addition it showed the screen capture of the classroom computer on an adjustable window. The stream video could be zoomed and frozen but it could not be recorded for later viewing.

Students accessed the streaming platform using their usual university credentials, i.e. the same credentials needed to check the university email, to enroll in courses and exams, etc. Hence, sharing access with other students was problematic as it would have implied also sharing access to all these other services.

At the University of Geneva terms consist of 13 weeks (plus 1 week of mid-term break). In week one, the professors of the participating courses presented the experiment in class to their students, who then had two weeks to enroll in the university's e-learning platform (they would have to do this regardless of the experiment as the e-learning platform is regularly used for sharing documents, announcements, submitting assignments, etc.). Based on the enrollment lists of each course from the e-learning platform, we first randomly assigned students to three groups. A first group of students (15% of all students) never had access to the streaming service and we label this group the never treated. Another 15% of the students were given access to the service in all the weeks of the term and we label this group the always treated. The remaining 75% of students were given access to streaming only some weeks at random and we label this group the sometimes treated. Every week, a a varying share of students in this group was given access. In the Spring semester 2017 we randomly assigned weekly access to 50% of the sometimes treated. In the Fall semester 2017 we decided to vary this share: 80% in week 3, 40% in week 4, 60% in week 5, 20% in week 6, 80% in week 7, 40% in week 8, week 9 was mid-term break, 60% in week 10, 20% in week 11, 80% in week 12, 40% in week 13, 60% in week 14.

At the end of week 2, students were notified by email about the exact sequence of weeks when they were given access to the streaming platform. Lectures started being streamed in week 3 and through the entire duration of the term. When a student had access to the streaming platform, she could watch the lectures of all the participating courses she was enrolled in for that week. Physical attendance in the classroom was always possible and students could freely decide to go to class in person even in the weeks when they had access to streaming.
Intervention (Hidden)
Intervention Start Date
2017-02-20
Intervention End Date
2017-12-22

Primary Outcomes

Primary Outcomes (end points)
Classroom attendance, Exam grades, Dropout
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
All students who enrolled in the e-learning platform for any given course were randomized into treatment. We first randomly assigned students to three groups. A first group of students (15% of all students) never had access to the streaming service and we label this group the never treated. Another 15% of the students were given access to the service in all the weeks of the term and we label this group the always treated. The remaining 75% of students were given access to streaming only some weeks at random and we label this group the sometimes treated. Every week, a a varying share of students in this group was given access. In the Spring semester 2017 we randomly assigned weekly access to 50% of the sometimes treated. In the Fall semester 2017 we decided to vary this share: 80% in week 3, 40% in week 4, 60% in week 5, 20% in week 6, 80% in week 7, 40% in week 8, week 9 was mid-term break, 60% in week 10, 20% in week 11, 80% in week 12, 40% in week 13, 60% in week 14.
We then collect question-by-question exam results and we ask professors to provide a mapping of weeks to questions, indicating in which weeks of the term the material required to answer the question correctly was covered. We can then look compare the probability of answering correctly both across treated and untreated students in any given week as well as across different weeks for the same student.
Experimental Design Details
Randomization Method
Randomization was done in office by a random number generator on a computer.
Randomization Unit
Individual student, first randomized into three treatment groups. The first group consists of a random 15% of students who are never given access to the streaming platform. Another random 15% of students are given access to the platform all the weeks of the term. The remaining 75% are given access at random week by week.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1621 students, over 11 weeks of the term and 8 exams. Some students are present for 2 terms and the number of courses varies by student.
Sample size: planned number of observations
1621 students, over 11 weeks of the term and 8 exams. Some students are present for 2 terms and the number of courses varies by student.
Sample size (or number of clusters) by treatment arms
33'561 student-course-week observations
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Outcome = probability of answering correctly to a single exam question; power 80%; significance level 95%; sample size 23'766 students-course-week observations, standard deviation 0.305 (after controlling for week, course fixed effects, student gender, nationality, birth cohort, high-school grades) -> MDE=0.011 (approx. 1.1 percentage points).
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
December 22, 2017, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
December 22, 2017, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
1621 students, over 11 weeks of the term and 8 exams. Some students are present for 2 terms and the number of courses varies by student.
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
1621 students, over 11 weeks of the term and 8 exams. Some students are present for 2 terms and the number of courses varies by student.
Final Sample Size (or Number of Clusters) by Treatment Arms
33'561 student-course-week observations
Data Publication

Data Publication

Is public data available?
No

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Using a randomized experiment in a public Swiss university, we study the impact of online live streaming of lectures on student achievement and attendance. We find that (i) students use the live streaming technology only occasionally, apparently when random events make attending in class too costly; (ii) attending lectures via live streaming lowers achievement for low-ability students and increases achievement for high-ability ones and (iii) offering live streaming reduces in-class attendance only mildly. These findings have important implications for the design of education policies.
Citation
Cacault, M, Hildebrand, C, Laurent-Lucchetti, J and Pellizzari, M. 2019. 'Distance Learning in Higher Education: Evidence from a Randomized Experiment'. London, Centre for Economic Policy Research Discussion Paper n. 13666.
Abstract
Using a randomized experiment in a public Swiss university, we study the impact of online live streaming of lectures on student achievement and attendance. We find that (i) attending lectures via live streaming lowers achievement for low-ability students and increases achievement for high-ability ones; (ii) students use the live streaming technology only occasionally, apparently when random events make attending in class too costly, and (iii) offering live streaming reduces in-class attendance only mildly. These findings have important implications for the effective design of education policies.
Citation
M Paula Cacault, Christian Hildebrand, Jérémy Laurent-Lucchetti, Michele Pellizzari, Distance Learning in Higher Education: Evidence from a Randomized Experiment, Journal of the European Economic Association, Volume 19, Issue 4, August 2021, Pages 2322–2372, https://doi.org/10.1093/jeea/jvaa060

Reports & Other Materials