Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities

In 1998, a local non-governmental organization (NGO) launched a program known as the Primary School Deworming Program (PSDP) to provide deworming medication to individuals enrolled in 75 primary schools in Busia District, a densely-settled farming region of rural western Kenya adjacent to Lake Victoria. The schools participating in the program consisted of nearly all rural primary schools in Budalangi and Funyula divisions in southern Busia district, and contained more than 30,000 pupils at the start of the study. Baseline parasitological surveys conducted by the Kenyan Ministry of Health indicated that these divisions had high rates of helminth infection at over 90%. Using modified WHO infection thresholds (Brooker et al., 2000b), roughly one-third of children in the sample had “moderate to heavy” infections with at least one helminth at the time of the baseline survey, a rate not atypical by regional standards (Brooker et al., 2000a). The 1998 Kenya DHS indicated that 85% of children in western Kenya, in the relevant age range of 8-18 years, were enrolled in school – suggesting that the sample was broadly representative of western Kenyan children as a whole at the time.

The 75 program schools were randomly divided into three groups (Groups 1, 2, and 3) of 25 schools each: the schools were stratified by geographical area (division, then zone), the zones were listed alphabetically (within each division), and then within each zone the schools were listed in increasing order of student enrolment, and every third school was assigned to a given project group. Due to the NGO’s administrative and financial constraints, the schools were phased into the program over the course of 1998-2001, and the order of phase-in was randomly determined, creating experimental treatment groups. Group 1 schools began receiving free deworming in 1998, Group 2 schools in 1999, while Group 3 schools began receiving the drugs in 2001. The project design implies that in 1998, Group 1 schools were treatment schools while Group 2 and 3 schools were the control, and in 1999 and 2000, Group 1 and 2 schools were the treatment schools and Group 3 schools the control, and so on. In 2002 all schools received free treatment. Children in Group 1 and 2 schools thus received two to three more years of deworming than Group 3 children. Deworming drugs were offered twice per year in treatment schools.

1998-03-01

1999-11-30

The following outcomes were estimated within school (since not all students received treatment due to absenteeism and some girls being over 13 years of age), across treatment and control schools, and schools which were within 3 km and 6 km of treatment schools:

1) Geohelminth and schistosomiasis infection rates
2) School attendance
3) Academic test scores

1) Geohelminth and schistosomiasis infection rates: the Ministry of Health in Kenya conducted a baseline parasitological survey between January and March 1998 in Group 1 schools. The same survey was repeated the following year in Group 1 and Group 2 schools between January and February 1999, one year after the treatment was administered to Group 1 schools and before Group 2 schools had been treated. Each child in the parasitological sample was given a plastic container and asked to provide a stool sample; samples were examined in duplicate within twenty-four hours using the Kato-Katz method.

2) School attendance: Since administrative school attendance records are often poorly kept, school participation was measured during unannounced school visits by NGO field workers. Schools received an average of 3.8 school participation check visits per year in 1998 and 1999.

3) Test Scores: In 1998 and 1999, the NGO administered English, Mathematics, and Science-Agriculture exams to pupils in grades 3 to 8. Exams were modeled on those given by the district office of the Ministry of Education, and prepared using the same procedure. The average score across all subjects was used as the test score outcome, normalized by grade.

In 1998, a local non-governmental organization (NGO) launched a program known as the Primary School Deworming Program (PSDP) to provide deworming medication to individuals enrolled in 75 primary schools in Busia District, a densely-settled farming region of rural western Kenya adjacent to Lake Victoria. The schools participating in the program consisted of nearly all rural primary schools in Budalangi and Funyula divisions in southern Busia district, and contained more than 30,000 pupils at the start of the study. The 75 program schools were randomly divided into three groups (Groups 1, 2, and 3) of 25 schools each: the schools were stratified by geographical area (division, then zone), the zones were listed alphabetically (within each division), and then within each zone the schools were listed in increasing order of student enrolment, and every third school was assigned to a given project group. Due to the NGO’s administrative and financial constraints, the schools were phased into the program over the course of 1998-2001, and the order of phase-in was randomly determined, creating experimental treatment groups. Group 1 schools began receiving free deworming in 1998, Group 2 schools in 1999, while Group 3 schools began receiving the drugs in 2001. The project design implies that in 1998, Group 1 schools were treatment schools while Group 2 and 3 schools were the control, and in 1999 and 2000, Group 1 and 2 schools were the treatment schools and Group 3 schools the control, and so on. In 2002 all schools received free treatment. Children in Group 1 and 2 schools thus received two to three more years of deworming than Group 3 children.

List randomization.

The school is the unit of randomization into the PSDP.

Yes

75 schools

Over 30,000 pupils

Group 1 - 25 schools, received treatment in both 1998 and 1999
Group 2 - 25 schools, received treatment in 1999
Group 3 - 25 schools, received treatment in 2001

WORMS: IDENTIFYING IMPACTS ON EDUCATION AND HEALTH IN THE PRESENCE OF TREATMENT EXTERNALITIES, DATA USER'S GUIDE

This document provides an overview of the data collected in an effort to monitor and evaluate the Primary School Deworming Project (PSDP), a primary school health intervention in rural Kenya. From 1998 until 2002, medical treatment for intestinal helminths (worms) was provided by a local non-governmental organization, Internationaal Christelijk Steunfonds Africa (ICS), to 75 rural Kenyan primary schools in the western district of Busia. Medical treatment was randomly “phased in” to the 75 schools between 1998 and 2001. Schools phased into the assistance project in earlier years served as “treatment schools”, and those phased in later as “comparison schools” in the evaluation.

The seventy-five schools participating in the program consisted of nearly all rural primary schools in the Budalangi and Funyula divisions of southern Busia district, and contained nearly 35,000 pupils at the start of the study. Baseline parasitological surveys conducted by the Kenyan Ministry of Health indicate that these two divisions had high rates of helminth infection, at over 90% (levels not atypical by sub-Saharan African standards, see Brooker et al. 2000). The data described here is employed in Miguel and Kremer (2004).

Miguel, Edward; Kremer, Michael; Hamory Hicks, Joan; & Nekesa, Carolyne. (2014). Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities, Data User's Guide. UC Berkeley: Center for Effective Global Action.

http://escholarship.org/uc/item/8xf1q0w1#

WORMS: IDENTIFYING IMPACTS ON EDUCATION AND HEALTH IN THE PRESENCE OF TREATMENT EXTERNALITIES, GUIDE TO REPLICATION OF MIGUEL AND KREMER (2004)

In order to facilitate replication of the analysis presented in Miguel and Kremer (2004), and to allow for further original analyses, the data and do-files employed in the published paper have been made available to interested researchers since January 2007. This document provides notes on the files which reproduce the results presented in Miguel and Kremer (2004), and discusses the fully corrected, updated version of these results. Background to the study, as well as notes on the data sets, can be found in companion documents entitled “Data User’s Guide” and “Codebooks”.

Miguel, Edward; & Kremer, Michael. (2014). Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities, Guide to Replication of Miguel and Kremer (2004). UC Berkeley: Center for Effective Global Action.

http://escholarship.org/uc/item/8db127cm

ESTIMATING DEWORMING SCHOOL PARTICIPATION IMPACTS IN KENYA: A COMMENT ON AIKEN ET AL. (2014B)

Aiken et al. (2014b) draw the conclusion that the evidence for a relationship between deworming and school attendance is “weak” based on two fundamental errors in their data analysis. First, the authors redefine treatment to include pre-treatment control periods. Second, while the original research design was based on a stepped-wedge analysis that was adequately powered, the re-analysis
authors undertake a clearly under-powered alternative analysis which ignores the time series element of the data, and then splits the cross-sectional analysis into two separate components, each of which has inadequate power. Examining the fully powered analysis, they report: “In a fully-adjusted logistic regression model making maximum use of the data available, there appeared to be strong evidence of an improvement in school attendance”. If either error is corrected, deworming significantly increases school attendance under the full range of statistical analyses considered by Aiken et al. Their analysis also underestimates the impact of deworming on school attendance by neglecting violations of the SUTVA assumption generated by transmission of worm infection to nearby schools (Miguel and Kremer 2004). We also respond to concerns raised by Aiken et al. regarding data collection processes and blinding.

Hicks, Joan Hamory, Michael Kremer, and Edward Miguel. 2014b. Estimating Deworming School Participation Impacts in Kenya: A Comment on Aiken et al. Original author response to 3ie Replication Paper 3, part 2. Washington, DC: International Initiative for Impact Evaluation (3ie)

https://www.povertyactionlab.org/sites/default/files/publications/21%20Response%20Part%202%20Dec2014.pdf

ESTIMATING DEWORMING SCHOOL PARTICIPATION IMPACTS AND EXTERNALITIES IN KENYA: A COMMENT ON AIKEN ET AL. (2014)

Aiken et al. (2014) usefully correct some errors in Miguel and Kremer (2004). Miguel and Kremer (2004) made two key claims: 1) deworming creates positive epidemiological externalities, thus causing estimates of the impact of deworming based on individual randomization to be biased downwards; and 2) deworming increases school participation. The results in Aiken et al. (2014) are consistent with these findings. In addition to direct impacts of deworming treatment on worm infections, Aiken et al. (2014) find externality effects within schools on untreated pupils, as well as externality effects across schools up to 3 km away. Similarly, with regard to school participation, both Miguel and Kremer (2004) and Aiken et al. (2014) find direct effects of deworming and externality effects within schools on untreated pupils. As Aiken et al. (2014) point out, most of the errors they identify (rounding errors or updates to the data set) lead to only small changes in estimated coefficients. The key change is that Miguel and Kremer (2004) measured externalities among schools located within 3-6 km that were among the 12 closest schools, rather than among all schools within 3-6 km, as reported; Aiken et al. (2014) report results including the full set of schools within 3-6 km. With the updated data, there is no evidence that worm infection externalities extend beyond the 12 closest schools to the full set of schools within 6 km, perhaps unsurprisingly given the local nature of disease transmission. We disagree with Aiken et al.’s (2014) conclusion that “there was no evidence of a between-school indirect effect” or an overall effect of deworming on school participation. We show that this conclusion is based on an approach that adds substantial noise to the estimation, by heavily weighting a non-significant 3-6 km externality estimate. This note addresses these and other points, and comments on the current state of deworming evidence.

Hicks, Joan Hamory, Michael Kremer, and Edward Miguel. 2014a. Estimating Deworming School Participation Impacts and Externalities in Kenya: A Comment on Aiken et al. Original author response to 3ie Replication Paper 3, Part 1. Washington, DC: International Initiative For Impact Evaluation (3ie).

https://www.povertyactionlab.org/sites/default/files/publications/21%20Response%20Part%201%20Oct2014.pdf

COMMENTARY: DEWORMING EXTERNALITIES AND SCHOOLING IMPACTS IN KENYA: A COMMENT ON AIKEN ET AL. (2015) AND DAVEY ET AL. (2015)

Hicks, Joan Hamory, Michael Kremer, and Edward Miguel. 2015. Commentary: Deworming externalities and schooling impacts in Kenya: a comment on Aiken et al. (2015) and Davey et al. (2015). Int. J. Epidemiol. (2015): 1-4.

http://ije.oxfordjournals.org/content/early/2015/07/21/ije.dyv129.full.pdf+html

WORMS: IDENTIFYING IMPACTS ON EDUCATION AND HEALTH IN THE PRESENCE OF TREATMENT EXTERNALITIES

Intestinal helminths—including hookworm, roundworm, whipworm, and schistosomiasis—infect more than one-quarter of the world’s population. Studies in which medical treatment is randomized at the individual level potentially doubly underestimate the benefits of treatment, missing externality benefits to the comparison group from reduced disease transmission, and therefore also underestimating benefits for the treatment group. We evaluate a Kenyan project in which school-based mass treatment with deworming drugs was randomly phased into schools, rather than to individuals, allowing estimation of overall program effects. The program reduced school absenteeism in treatment schools by one-quarter, and was far cheaper than alternative ways of boosting school participation. Deworming substantially improved health and school participation among untreated children in both treatment schools and neighboring schools, and these externalities are large enough to justify fully subsidizing treatment. Yet we do not find evidence that deworming improved academic test scores.

Miguel, Edward, and Michael Kremer. 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities." Econometrica 72(1): 159-217.

https://www.povertyactionlab.org/sites/default/files/publications/21%20Primary%20School%20Deworming%20in%20Kenya%20Pre-Print.pdf