Opinion, Berkeley Blogs

Cochrane’s incomplete and misleading summary of the evidence on deworming

By Temina Madon

The Cochrane Collaboration recently updated their review of the evidence on treating school-age children for soil-transmitted helminths (STH, i.e. intestinal worms which infect nearly a quarter of the world's population). While many of the methodological flaws in the 2007 review have been addressed, the new review is still incomplete and provides a misleading summary of the evidence on mass deworming. Deworm the World, Innovations for Poverty Action (IPA), the Center for Effective Global Action (CEGA) and the Abdul Latif Jameel Poverty Action Lab (J-PAL) all encourage countries with moderate to high worm loads to mass treat for STH because it is one of the most cost-effective ways of increasing school attendance. These results remain unchanged and more recent studies (see below) have only strengthened the case. We have not yet carefully reviewed Cochrane's summary of the evidence on height, weight, anemia, and cognitive outcomes (although others have commented), but the educational benefits alone are sufficient to justify mass school-based deworming.

Problems with the Cochrane review: While claiming to cover published and unpublished studies the review excludes three important randomized and quasi-randomized studies on STH which all show positive results. Bleakley (2004) shows that an early 20th century campaign to eradicate hookworm in the US south improved school attendance, literacy, and income in adulthood. Ozier (unpublished) shows that children under one year of age (and therefore too young to be treated) at the time of mass deworming in their communities in Kenya had significantly improved cognitive outcomes due to spillover effects.  Baird et al (unpublished) shows strong long term labor market outcomes of children treated for STH.

Miguel and Kremer (2004) is the only study quoted in the main results section of the Cochrane review with outcomes on student attendance (the other quoted studies are not clustered and therefore appropriately ignored). Yet the review authors down-weight the Miguel and Kremer results on student attendance for a number of reasons. For example, that there is "high attrition" in hemoglobin (Hb) data, even though there is zero attrition for the actual student attendance results!  (The reason for the smaller Hb sample is that it was only collected for a random subsample, information that is readily available from the authors).

The Cochrane authors also criticize Miguel and Kremer (2004) for a lack of baseline (pre-treatment) data on school attendance. This is a strange point for at least four reasons. First, given the experimental design there is no systematic reason to expect differences in school attendance between the treatment and control groups. Second, Miguel and Kremer (2004) show that there is baseline balance along a wide range of other characteristics in the treatment and control groups (in Table 1), many of which correlate strongly with school attendance (i.e., family socioeconomic status). But most importantly, there is in fact baseline "balance" between Group 2 and Group 3 in the high-quality school attendance data in 1998, when both were still "control" before Group 2 was phased into deworming (see Table 8 of the Miguel and Kremer 2004 paper for this data). The Cochrane authors apparently missed this in their read of the paper. Finally, the Cochrane authors strangely "slice" up the data into impact in the first year after deworming versus the second year, and then across different "comparisons" of the treatment groups. Measuring impacts in small subsamples of the data inevitably reduces the precision of the estimate, making an effect that is strongly significant in the full sample look insignificant in small slices.

Missing the point about mass deworming. In their conclusion, the review's authors state that "Screening school children for intestinal helminths and then treating those infected probably has some value…[but] there is insufficient evidence to recommend deworming drugs in targeted community programmes." This argument overlooks the fact that if a program is effective for individuals with worms, it will still affect the same individuals (presumably with the same effect) if they are reached through a mass treatment campaign. The only reasons to prefer a screening approach is if deworming drugs had negative effects on uninfected children (they do not), or if the costs of treating uninfected children in a mass campaign were greater than the costs of individually testing children to determine whether they required treatment (in fact it is much cheaper to mass treat than to diagnose and treat).

School attendance is an important goal in its own right but it is also a step towards longer term outcomes. Preliminary results from Baird et al suggest that mass treatment for STH leads to longer hours worked and higher earnings 10 years after treatment — results remarkably similar to those found in the US South. We therefore strongly endorse the WHO and Copenhagen Consensus's recommendation to mass treat children for STH.

This post is co-authored by Annie Duflo (Executive Director of IPA), Alissa Fishbane (Managing Director of Deworm the World Initiative, IPA), Rachel Glennerster (Executive Director of J-PAL Global), Michael Kremer (Gates Professor of Developing Societies in the Department of Economics at Harvard University), Temina Madon (Executive Director of CEGA), and Edward Miguel (Professor of Economics at UC Berkeley).