Information, Education and Social Networks

We randomized access to Wikipedia in four government boarding schools in Malawi. In each school, we set up an internet library with 12 internet-enabled phones. Internet access was restricted to Wikipedia and Wiktionary. Students in the treatment group used the phones under the supervision of research staff referred to as digital librarians. The digital library was open during one school year, four days per week from 4pm-8pm and on weekends from 8am-4pm. No control group students were allowed into the mobile libraries.

Introduction to the project:

In each school, two digital librarians held a presentation to introduce the project. This introduction was conducted for one form at a time. All students received the same information. The presentation included the following information:
• Not everyone will have access to the digital library.
• Only students in forms (grade levels) 2, 3 and 4 are eligible for the project.
• The selection of the participants will be random and everyone has a chance to participate in the project.
• Selected students will only have access to Wikipedia.
• Selected students will be allowed to share the information they find, and others may ask selected students to search for information for them.
• We will conduct additional surveys throughout the year, with both selected and non-selected students.
We provided examples of types of information available on Wikipedia, including school topics (photosynthesis, equations and volcanos), sports, culture and health.

Baseline and randomization:

After introducing the project to all students, we conducted baseline surveys. Our team of eight enumerators attempted to survey every student in forms 2, 3 and 4. We performed the randomization on a computer (additional details provided below) and informed the students of the results. The selected students then took part in a mandatory induction session.

Induction:

The students in the treatment group attended a mandatory induction session in the digital library. The induction was done in small groups and aimed at informing the students about the digital library, the rules and how to use the phones and access to Wikipedia. The students drew from an envelop a personal username and had to choose a password. The students were informed that they should remember their username and password and not share this information with their friends. Then, the librarians showed them how to access Wikipedia and practiced with them. Finally, we told the students that the searches were anonymous and that we (librarians, researchers and teachers) will not be able to identify individual information on who accessed each page they searched in Wikipedia. Finally, we told the students we could suspend or remove them from the digital library if they would not follow the rules.

Digital library:

Each digital library was equipped with 12 phones. These phones were shared among approximately 75 treatment students in each school. The opening hours were from 4pm-8pm on most weekdays (four days per week) and from 8am-4pm on weekends. At least one digital librarian was always present to supervise the digital library. While the librarian supervised phone use, he or she did not monitor the content browsed by students.

A student wishing to use a phone would first sign in with the librarian. If a phone was available, students would sign the attendance book and borrow a phone for use within the digital library. If all phones were in use, they would join the waitlist or come back later. If there were students waiting, phone use was restricted to 35-40 minutes. The mobile librarian recorded arrival time as well as phone use time for each student. Only selected (intervention arm) students used the mobile library; the librarians used student photos to verify a student’s identity. Every week, the lead enumerator would visit each digital library to spot check the identities of the students and verify that no student in the control group was given access to the digital library. We also conducted spot checks, comparing student signatures to the baseline survey. We did not encounter a case where a control arm student gained access to the digital library.

Students used the phones one by one (not in pairs or groups), and were not permitted to leave the digital library with a phone. Students could ask the digital librarian for technical help, but were not permitted to talk among themselves inside the digital library. Students were allowed to take notes while using the phones. We restricted access to Wikipedia, Simple English Wikipedia and Wiktionary.

Students logged into the phones using a personal and unique username and password. The username was unrelated to other student identifiers. This username allows the researchers to link anonymous browsing data to a bin of coarsened student characteristics. During the induction, each student within a bin chose their username from the same hat, to make it clear that browsing data obtained by the researchers would not be linked to a particular student.

2017-11-04

2018-06-23

Final grade in English and Biology

We will use endline grades in English and Biology, taken from administrative data. We will use term 3 grades for forms 2 and 3, and MSCE national exam grades for form 4 students (these students do not have a term 3 grade). We will standardize each grade by subtracting the mean within the school and form, and dividing by the control group standard deviation, again within the school and form.

English and Biology were chosen as primary outcomes for two reasons. First, these are core subjects, and we therefore expect to have data for most students. Second, we hypothesize that Wikipedia is a particularly good resource for these subjects, compared to other core subjects (Math, Physics and Chichewa). Wikipedia allows students to practice English comprehension. Biology students must learn a large volume of information, and are likely to benefit from information based study materials. Physics and Math, on the other hand, require students to practice skills for which information on Wikipedia might be less useful. Chichewa is a Malawian local language with little representation on Wikipedia.

Primary and secondary outcomes were chosen pre-analysis. These outcomes are identified in our pre-intervention ethics application (upon request).

We will investigate the following classes of secondary outcomes:
1. Other academic outcomes
2. Aspirations
3. Internet literacy
4. Student time use
5. General knowledge
6. Social network position: the effect of the intervention on the structure of social networks will be discussed in a separate paper

The experiment took place in four boarding secondary schools in Malawi. The experiment involved students in forms (grade levels) 2, 3 and 4.

We interviewed all students in forms 2, 3 and 4 in the four schools at baseline and collected administrative data on baseline academic performance.

We randomized at the student level, and stratified on four key variables: school, form, grades and internet experience. The randomization assigned students to one of the three following groups: treatment (arm 1), control surveyed (arm 2) or control others (arm 0). Students in arm 1 have access to the digital library. Students in arms 1 and 2 are interviewed often (baseline, time use survey, endline A and endline B). Students in arm 0 are only interviewed for the baseline survey and a short version of the endline survey (endline A).

We stratified on four key variables: school, form, grades and internet experience. The bin for grades is defined as above or below the median grade (within the school and form). We used the average of English and Biology grades (our two primary outcomes). We constructed a separate bin for students with missing grades data. Internet experience is defined as whether the student has ever used the internet. There are 51 stratification bins.

We randomized at the student level using the Stata command randtreat, seeded with the date of the randomization (2910). The randomization assigns students to one of the three following groups: treatment (arm 1), control surveyed (arm 2) or control others (arm 0).

Student (individual)

No

N/A

1508 students

301 students are in the treatment group (arm 1). 298 students are in the control surveyed group (arm 2). 909 students are in the control other group (arm 0).

A conservative calculation suggest that this sample size will allow us to detect a 0.15 SD increase in term grades with 80% statistical power. This calculation assumes a covariate-R2 of 0.6. Pilot data shows that previous term grades explain most of the variation in current term grades: R2=0.8.