Archived Science and Technology News in Context

The Politics of Counting Dead Iraqis

On October 10, 2006, at the height of the American midterm campaign season, the distinguished medical journal  Lancet published an article on the internet that suggested a statistical estimate of the number of Iraqis who died as a result of the American invasion of their country in 2003.  The estimate – 655,000 dead – was stunning because even the lower bound of its confidence interval was an order of magnitude greater than the highest estimates put forward to date.  Perhaps not surprisingly given the prominence of the Iraq war as a campaign issue, the article proved an immediate sensation, maintaining a top spot in the headlines for several news cycles.

All of a sudden, everyone from local newspaper editors to the president was weighing in on the number of Iraqi dead.  In a press conference held early on the morning of the 11th, a reporter asked U.S. president George W. Bush if he felt the study’s estimate of 650,000 casualties was credible.  Bush’s response perfectly encapsulates the major substantive bones of contention that would emerge in subsequent media debates.  First, the estimate of 650,000 was simply too high to be believed, and the president reiterated his support for an estimate of 30,000 civilian deaths that he had been citing in press conferences for over a year.  Second, he stated that the study’s purportedly scientific methodology had been “pretty well discredited,” thus making it perfectly reasonable to disbelieve the estimate.  And third, he hinted that the exact number of Iraqis killed is not particularly meaningful in evaluating whether the war was, on balance, a good thing.  Whether 30,000 or twenty times this many had died, “I do know that a lot of innocent people have died, and that troubles me and it grieves me. And I applaud the Iraqis for their courage in the face of violence. I am amazed that this is a society which so wants to be free that they're willing to -- that there's a level of violence that they tolerate” [1].  Despite the loss of a significant (but not overwhelming) number of “innocent” lives, the fact that Iraqis were still fighting for their freedom suggested that they implicitly deemed that such casualties were justified. 

Thus, rather than ending a conversation by dint of scientific authority, the Lancet article served to begin one in which everyone might as well have been an expert.  Talking heads alternately embraced the statistic as further evidence that the Iraq war has been a policy disaster and denounced it as a cheap political stunt, science gone bad, or at best, an irrelevant factoid.  What went wrong?  Was this yet another apparent case of “reality” dashing itself to pieces against arrogant and ignorant ideology?  Or was it a case of spurious results, produced by a biased scientific elite, being politely disregarded by more commonsensical men of practical affairs?  Based on the editorials and blog comments that the Lancet study provoked, it is not difficult to find support for these interpretations; and it is not surprising that they set the major tropes in the media coverage as the story unfolded.

However, the literature of science and technology studies suggests several other ways to think about the aftermath of the Lancet study.  On one level, the conversation revolves around seemingly technical matters of evaluating “inferences” versus “extrapolations,” sample size and selection techniques, standard deviations and margins of error.  Yet other concepts that arise – counting and accountability, objectivity, and representation – have obvious political as well as scientific ramifications.  Literature inthe STS tradition has addressed these concepts in a number of ways.  For instance, a number of studies have stressed the critical role of quantitative evidence in ensuring objectivity and accountability in adversarial environments. Quantitative methodologies feature prominently in the selection of projects by the Army Corps of Engineers, the approval of pharmaceuticals by the Food and Drug Administration, or the adoption of regulatory policies by the Environmental Protetion Agency [2].

Quantification may be necessary for objectivity and accountability, yet it is not sufficient.  After all, George Bush could counter the “not credible” quantitative estimate of the Lancet with his own quantitative estimate, derived from unspecified media sources.  So evidently, the power of quantitative evidence lies less in the numbers per se than in the processes that create them. The use of numbers acts to constrain the expressive options of the quantifier by excluding many subjective statements – for example, judgments of aesthetics or sentiment.  This is because the generation of quantitative estimates is generally governed by mechanical processes that are specifically intended to remove any discretion or subjectivity on the part of scientific observers.  This is most evident in the use of strict regimens of “randomization” used to generate samples upon which statistical estimates are to be based.  Scientists are not permitted to exercise their judgment in the selection of samples, but must defer to mechanized random-number generators, the physical laws governing flipped coins, and so forth.  By ostensibly removing discretion and decision-making power from the scientist, the resulting estimate is perfectly determined and therefore apolitical [3].

Even so, the responses to the Iraqi death count suggest that the rule-bound processes of quantification – processes that were intended to guarantee objectivity – failed to induce public assent to quantitative estimates in this particular case.  Bush, like many other commentators, would suggest that the Lancet study’s statistical sampling methodology was “pretty well discredited,” although he did not offer much justification for the methodology behind his own estimate, or behind other official casualty statistics.  In fact, just over a month after his comments, the report of the Baker-Hamilton Iraq Study Group criticized the way in which official American casualty estimates were generated.  They noted that “there is significant underreporting of the violence in Iraq” because the criteria used to record violent incidents and casualties “acts as a filter to keep events out of reports and databases.” 

Therefore, recommendation 78 of the Study Group’s report stated: “The Director of National Intelligence and the Secretary of Defense should institute immediate changes in the collection of data about violence and the sources of violence in Iraq to provide a more accurate picture of events on the ground” [4]. So in this case, there existed multiple internally consistent, rule bound, and “objective” methodologies that gave very different numbers of Iraqi civilian dead.

If neither quantification nor rule-bound processes of number-production could induce general acceptance of an estimate of Iraqi civilian casualties, what could?  What additional concepts of counting and accountability are needed to make this debate end?  Bush’s last comments in the press conference suggest what may be at stake.  Death is an accepted fact of life in all human societies; and as Bush noted, while he is “amazed” at the level of violence Iraqis are willing to tolerate in their pursuit of “freedom,” he is impressed by their sense that the sacrifice is justified.  In this reckoning, many of the Iraqi dead were not innocent victims of an American invasion, but active participants in a bloody but hopeful process of forging a new society.

Compare this framing with that proposed by the Lancet study’s authors, for whom war is viewed through the lens of public health.  In the Lancet article from 10 October, editor Richard Horton offered this interpretation of the study’s significance:

“Globalisation has changed the terms of human engagement at many levels – in trade, aid, economic development, environmental protection, and agriculture.  Yet foreign policy is still governed by principles that had their origin in the 19th century, based, as they were, around notions of national sovereignty and economic and geographical self-interest.  Those principles need to be radically revised.  Health and wellbeing – their underpinning values, their diverse array of interventions, and their goals of healing – offer several original dimensions for a renewed foreign policy that might at least be one positive legacy of our misadventure in Iraq."[5]

Measured according to the framework of public health – i.e., by comparing prewar baseline mortality with postwar baseline mortality – the Iraq war is surely a disaster of the first order.  This is the overwhelming message of the Lancet study.”

It remains unclear which interpretation of the Iraq war will ultimately win out, and it is difficult to predict which one will be successful.  However, the implicit existence of these multiple interpretations alerts us to the true nature of the debate that has followed the publication of the Lancet article.  The technical arguments in the press over sampling methodology, cluster size, outlier elimination, and the like are important, but more often than not they obscure the fundamental issues that must be resolved.  Instead, in order to count accurately, we need to first figure out what counts, who is accountable for what, and what the numbers we produce really mean for judging the worth of the Iraq war.  Scientists and medical experts have been helpful in getting these questions on the table, but they are unlikely to provide the ultimate answers.

By Paul Erickson

References:
[1] See “Press Conference by the President,” 11 Oct. 2006 (at www.whitehouse.gov/news/releases/2006/10/20061011-5.html)

[2] See especially Theodore Porter, Trust in Numbers (Princeton: Princeton University Press, 1995); and Sheila Jasanoff, “American Exceptionalism and the Political Acknowledgement of Risk” in Edward Burger ed., Risk (Ann Arbor: University of Michigan Press, 1993).

[3] On the significance of scientific constraints in democratic political orders, see especially Yaron Ezrahi, The Descent of Icarus: Science and the Transformation of Contemporary Democracy (Cambridge, MA: Harvard University Press, 1990)

[4] Iraq Study Group Report, p. 62

[5] Richard Horton, “Iraq: Time to Signal a New Era for Health in Foreign Policy” Lancet 369 (21 October 2001), 1395-6.

Last updated: January 25, 2007