The Impact of Cell Phones in Traditional Classrooms
Last registered on December 20, 2018

Pre-Trial

Trial Information
General Information
Title
The Impact of Cell Phones in Traditional Classrooms
RCT ID
AEARCTR-0003659
Initial registration date
December 15, 2018
Last updated
December 20, 2018 9:52 PM EST
Location(s)
Region
Primary Investigator
Affiliation
Lahore University of Management Sciences
Other Primary Investigator(s)
PI Affiliation
Lahore University of Management Sciences
Additional Trial Information
Status
On going
Start date
2018-09-03
End date
2019-01-11
Secondary IDs
Abstract
The rise in the use of social media applications has dramatically increased the time spent on cell phones especially among the educated youth. Whereas the widespread ownership of mobile phone devices presents unique opportunities for the design of educational experience, and has created a lot of interest in newer models of imparting education, the life of a typical university student still predominantly revolves around attending lectures and taking exams. In such a setting, use of mobile phones in class could potentially have an adverse effect by drawing student attention away from learning. This research aims to carefully measure whether having access to a cell phone during class impacts student learning outcomes.

In this study, we aim to carefully measure the impact, if any, from students’ use of cell phone in the classroom using a randomized experiment. One of the questions we want to answer carefully is the extent to which use of a cell phone in class affects learning as measured by test scores. In addition, we want to collect data on intermediate outcomes of interest: class attendance and student engagement with class. Therefore, our primary outcome of interest is test scores while the secondary outcome variables are attendance and classroom activity.
External Link(s)
Registration Citation
Citation
Hussain, Karrar and Muhammad Naseer. 2018. "The Impact of Cell Phones in Traditional Classrooms." AEA RCT Registry. December 20. https://www.socialscienceregistry.org/trials/3659/history/39311
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2018-09-24
Intervention End Date
2018-12-24
Primary Outcomes
Primary Outcomes (end points)
Primary outcome of interest is the test scores in quizzes and exams.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
The secondary outcome variables are students' classroom attendance and classroom activity.
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
This research aims to carefully measure whether having access to a cell phone during class impacts student learning outcomes and establish some potential mechanisms. One of the questions we want to answer carefully is whether using your cell phone in class affects learning (and, if so, to what extent) as measured by test scores. We may also be able to establish any spillover effects of cell phones on other students in the class. In addition, we want to collect data on intermediate outcomes of interest, namely class attendance and student attentiveness, to establish the underlying mechanism behind any such impacts. Therefore, our primary outcome of interest is test scores while the secondary outcome variables are attendance and classroom activity.
Experimental Design Details
Based on our outcome variables, we will test the following set of hypotheses in our experiment. These hypotheses are as follows: a) Access to cell phones in class reduces student learning outcomes. If true, the treatment group (students not allowed to bring cell phone inside class rooms) will have higher test scores than the control group (students allowed to bring phones inside class room). b) The duration and order of treatment matters. We will use the second randomization in the middle of the semester to examine whether taking cell phones in the first half of the semester has a differential effect as compared to taking them in the latter half of the semester. We will also test whether exposing students to the treatment throughout the semester has a greater effect than a treatment that lasts half of the semester. Since this involves comparing students in the four cross-treatment conditions, these comparisons are expected to be relatively under-powered compared to the first comparison above. Related to the above, we want to test the following hypotheses to investigate the underlying mechanisms behind cell phones’ impact on student test scores (or lack thereof): c) Removing cell phone access during the class affects attendance. The attendance rate can either increase or decrease with our treatment so the effect is ex-ante ambiguous. If students in the treatment group learn better as a result, they might value lectures and respond by attending more classes. On the other hand, attendance for treated group could also go down if students do not like their phone being taken away and find that it imposes sufficient psychic costs to start skipping classes. d) Removing cell phones improves engagement or attentiveness in class. The class engagement may be significantly higher for the treated group as compared to the control group students if removing the phone reduces the potential for distraction and hence encourages greater focus on the learning activities in class. On the other hand, the students most likely to use cell phone during the class may find other ways to distract themselves (or fellow students) in which case the attentiveness should not change (may even worsen in case of large spillover effects in the treatment group).
Randomization Method
At the start of Week 2 of the semester (after the initial course ‘shopping’ window had formally closed), we obtained the enrollment list of each participating course from the instructors. While no new students could enroll after Week 1, some students still dropped their course before the end of the experiment due to which we will test for differential attrition by treatment status. Simultaneously, students were asked to complete a short baseline survey and a subject-specific test by the end of Week 3.

Several of the 11 courses had students from different batches enrolled in them. Within each class and batch cohort, students were randomized into treatment and control groups with equal probability. Beginning Week 4, students in the Treatment group were asked to comply with the “Treatment” condition i.e. deposit their cell phones with members of the project team at the start of each lecture and collect it after class.

For the latter half of the semester, we re-randomized students into T and C groups again stratifying on class and batch indicators while ignoring the prior treatment status. This decision was based on the following considerations:

i. By creating 4 ex-ante identical experimental groups (CC, CT, TC, TT), we varied treatment intensity orthogonally to student attributes. Besides allowing us to test if the duration of treatment matters, this set up also allows us to ask if the order of treatment matters.
ii. For the simple comparison of outcomes in Treatment versus Control, the second randomization can effectively allow us to double the sample size which improves power on the main hypothesis of interest.
iii. Like the pilot study conducted during the summer, subject fatigue started to arise in the Treatment group with students in that group increasingly disinclined to give their cell phones as the semester went on. Therefore in order to maintain high compliance with the experiment, which was crucial for power given that we estimate the ITT effect, we decided to do a second randomization offering everyone in the experiment a “fair second chance”.
Randomization Unit
The experiment was carried out in Pakistan at the Lahore University of Management Sciences (LUMS) which offers undergraduate and Masters level courses in Economics and other disciplines.

With consent from the instructors, students from a total of 8 undergraduate courses (ranging from freshmen to junior level courses) were included in this study which amounted to 11 classes or sections being taught by 9 different instructors. All of these are economics courses with the exception of one course from the education school and all instructors were generally quite permissive towards cell phone use by students in their classes.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
The number of clusters are 11 participating courses (679 unique students).
Sample size: planned number of observations
Planned number of observation is comprised of 824 student-course units.
Sample size (or number of clusters) by treatment arms
50% of each cluster is selected randomly in the treatment group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The power calculations are based on a small pilot conducted in the summer term before the start of this experiment should be sufficient to detect a 0.2sd impact on student test scores.
Supporting Documents and Materials

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Office of Sponsored Programmes and Research
IRB Approval Date
2018-09-03
IRB Approval Number
20180903
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers