Other Mortality Estimates

The 2004 and 2006 studies published in the Lancet are the only studies designed exclusively to study trends in overall mortality in Iraq since the 2003 invasion. One of the key messages from the authors of the study is the need for further studies to be carried out, and the conclusions of both the 2004 and 2006 papers call for an assessment of excess mortality by an independent body.

While no other studies have been designed exclusively to study mortality, a number of other projects have resulted in information that provides information relevant to overall mortality. This section contains information about how the findings of these studies and projects compare to those found by the studies published in the Lancet.

1: What other estimates are there of mortality in Iraq?

The studies published in the Lancet are the only studies to date designed exclusively to study recent developments in mortality in Iraq. There also are a number of other sources -- including active surveys, collation of media reports, and official records -- that also provide information about mortality in Iraq. The following is a brief discussion of the most prominent sources.

UN Development Program Iraq Living Conditions Survey

In 2005, the United Nations Development Program (UNDP) published the Iraq Living Conditions Survey 2004 (ILCS), a wide-ranging survey

The main finding of this survey was that living conditions in Iraq as measured by key indicators deteriorated significantly in the period April 2002 to April 2004. For example, chronic malnutrition increasing from 4% to 8% and access to safe water falling from 95% to 60% in urban areas.

In addition, the survey included a question on deaths in houeholds, and it was found that 24 thousand "war-related" deaths had occurred in the period March 2003 to April 2004, with 95% confidence that the true number is in the range 18-29 thousand.

Iraq Ministry of Health

On 9 November 2006 Health Minister Ali al-Shemari indicated that about 150,000 civilians had been killed by insurgents since the war started. The estimate was based on estimates of the number of bodies brought to mortuaries and hospitals, but the precise methodology used to derive the number was not presented.

It appears that the publication of official Iraqi estimates is politically controversial. According to press reports, in September 2006 Prime Minister Nouri al-Maliki ordered the Ministry of Health not to not release further figures about casualties to the United Nations, as it has previously done. Information such as this has lead some to believe that figures put out by official sources may be unreliable.

On 17 January 2007 it was reported that "based on a compilation of numbers from the Interior and Health ministries", the Iraqi government estimated that 12,357 civilians were killed in 2006. This figure was issued in response to a much higher UN estimate of the total number of civilians violently killed during 2006.

Iraq Body Count

The Iraq Body Count is:

"an ongoing human security project which maintains and updates the world’s only independent and comprehensive public database of [English-langugage] media-reported civilian deaths in Iraq that have resulted from the 2003 military intervention by the USA and its allies. The count includes civilian deaths caused by coalition military action and by military or paramilitary responses to the coalition presence (e.g. insurgent and terrorist attacks)."

The IBC website contains more information about the project and the methodology used.

As of November 2006, the IBC project had recorded between 45 and 50 thousand media-reported civilian deaths.

Iraq Coalition Casualty Count

Iraq Coalition Casualty Count produces figures based on media reports of military and civilian deaths in Iraq. It has recorded 23478 Iraqi deaths between the beginning of 2005 and 6th November 2006. It notes that:

This is not a complete list, nor can we verify these totals. This is simply a compilation of deaths reported by news agencies. Actual totals for Iraqi deaths are much higher than the numbers recorded on this site.

US Department of Defense

The military has released rough counts of average numbers of Iraqis killed and wounded in a quarterly accounting report mandated by Congress. In the reports, Measuring Stability and Security in Iraq, daily averages of dead and wounded Iraqi civilians, soldiers and police officers rose from 26 a day in 2004 to almost 120 a day in August 2006.

UN reports on morgue counts

The most cited morgue tally is that used by the UN, reported in the Human Rights Report of the United Nations Assistance Mission for Iraq.
These figures are based on "the number of casualties compiled by the Ministry of Health from hospitals throughout the country and the Medico-Legal Institute in Baghdad."

The most recent bi-montly UNAMI report, covering the period 1 November to 31 December 2006, states that 6,376 civilians were killed and at least 6,875 wounded in acts of violence during the reporting period, corresponding to about 105 deaths per day. According to UNAMI this brought the total civilian casualty figure for the year 2006 to 34,452 dead (an average of about 94 deaths per day) and 36,685 injured. The November and December figures are slightly lower than UNAMI's figures for civilians violently killed in the preceding two months (7,054 in September and October, including 351 women and 110 children, according to "information provided by the Ministry of Health").

Brookings Institution Iraq Index

The Brookings Institutions publishes the Iraq Index (pdf), a monthly compilation of economic, public opinion, and security data. In November 2006, the index indicated that data from Iraq Body Count and the UN Assistance Mission for Iraq give a total of 62,000 civilian deaths in the period May 2003 to November 2006. The index also notes that recent morgue data suggest that an estimated 90 percent of deaths are from violence.

2: Why is the number of deaths found by the Lancet study so much larger than other estimates commonly quoted?

Summary: The authors of the MIT/Bloomberg study argue that the discrepancy between their methodology and other methods of estimating mortality is consistent with previous experience of reporting of deaths from war zones. Passive surveillance (e.g., through media or healthcare facilities) generally is incomplete and thus undercount the true number of deaths. In addition, a difference between the MIT/Bloomberg results and some other studies results because they cover different time periods.

Passive vs. active surveillance methods

Adjusting for the time period of study, the findings of the MIT/Bloomberg estimates are around twice as high as a United Nations survey, four times as high as Iraqi ministry of Health estimates, and more than ten times as high as estimates based on media reports.

Some commentators have suggested that these differences suggest that the MIT/Bloomberg findings are "too high" (and therefore somehow suspicious), or that other estimates are "too low" (and therefore somehow faulty).

The MIT/Bloomberg study authors argue instead that the discrepancies is an expected outcome from the use of different methodologies. The MIT/Bloomberg study carried out an active survey specifically designed to obtain information about deaths from those most likely to know about them (the relatives of the deceased). It therefore is likely to have more complete reporting and a higher estimate closer to the true number of deaths in the population. The authors also argue that under-reporting by healthcare facilities, media, and other "passive surveillance measurse" is consistent with previous experience from conflict situtaions:

"Our estimate of excess deaths is far higher than those reported in Iraq through passive surveillance measures. This discrepancy is not unexpected. Data from passive surveillance are rarely complete, even in stable circumstances, and are even less complete during conflict, when access is restricted and fatal events could be intentionally hidden. Aside from Bosnia, we can find no conflict situation where passive surveillance recorded more than 20% of the deaths measured by population-based methods."

The authors also argue that the magnitude of the discrepancy -- four to ten times for hospital data or media accounts -- is consistent with experience from other conflict situations:

"In several outbreaks, disease and death recorded by facility-based methods underestimated events by a factor of ten or more when compared with population-based methods. In several outbreaks, disease and death recorded by facility-based methods underestimated events by a factor of ten or more when compared with population-based estimates. Between 1960 and 1990, newspaper accounts of political deaths in Guatemala correctly reported over 50% of deaths in years of low violence but less than 5% in years of highest violence. Nevertheless, surveillance tallies are important in monitoring trends over time and in the provision of individual data, and these data track closely with our own findings."

Differences in study periods

Another reason that headline numbers reported are difference is that estimates are available for different time periods. In particular:

  • the 2004 Bloomberg study estimated excess mortality between March 2003 and September 2004 (17 months);
  • the Iraq Living Conditions Survey 2004 recorded some deaths in the March 2003-May 2004 (13 months); and
  • the 2006 MIT/Bloomberg study encompasses a longer period, from March 2003 until June 2006 (40 months).

The 2006 study notes:

Application of the mortality rates reported here to the period of the 2004 survey gives an estimate of 112 000 (69 000–155 000) excess deaths in Iraq in that period. Thus, the data presented here validates our 2004 study, which conservatively estimated an excess mortality of nearly 100 000 as of September, 2004.

We discuss this and other factors in more detail here: 2004 Lancet article and ILCS study.

3: Are the study results consistent with the findings of the 2004 UNDP Iraq Living Conditions Survey?

Summary: A previous United Nations survey around half as many additional deaths in the same study period as the MIT/Bloomberg study. The two studies measured different quantities: as the ILCS study estimated deaths classified as "war-related" by respondents, whereas the MIT/Bloomberg study estimated all excess deaths. The two surveys also employed somewhat different methodologies.

Background on the ILCS

In 2005, the United Nations Development Program and jointly published the Iraq Living Conditions Survey 2004 (ILCS). This was a wide-ranging study with an extensive questionnaire that took respondents a median time of 83 minutes to complete. The questionnaire covered housing and infrastructure; household economy; basic demography; the education, health, and labour force characteristics of the household members; and women’s reproductive history and children’s health. The main finding was that living conditions in Iraq as measured by key indicators deteriorated significantly in the period April 2002 to April 2004. For example, chronic malnutrition increasing from 4% to 8% and access to safe water falling from 95% to 60% in urban areas.

ILCS mortality findings

In addition to these topics, the list of questions included the question: "Has any person(s) who was a regular household member died or gone missing during the past 24 months?" As the survey was carried out in April 2004, the "past 24 months" refers to the period April 2002 to April 2004.

The date of the deaths were not recorded in the ILCS, and its results therefore cannot be used to establish a change in mortality. A partial comparison with estimates of the change in the rate of death nonetheless is possible by using information about the cause of death recorded. The survey asked respondents to classify reported deaths in the categories "Disease / Traffic Accident / War related death / Pregnancy or childbirth / Other". The number of war-related deaths was estimated at 24 thousand in the period March 2003 to April 2004 (12 months), with 95% confidence that the true number is in the range 18-29 thousand.

Comparison with the 2004 Bloomberg study

The ILCS study can be compared to the 2004 Bloomberg study, which found a total of 98 thousand deaths in total in the period March 2003 to September 2004 (18 months), with 95 confidence the true number was in the range 8-192 thousand. This corresponds to 65 thousand total deaths per year, or 2.7 times the number found by the ILCS for war-related deaths.

Comparison with the 2006 MIT/Bloomberg study

The 2006 Lancet study estimated an increase in crude mortality of 7.8/1,000 for the period March 2003 to June 2006. This figure corresponds to 655 thousand deaths over 40 months.

In the period March 2003 - April 2004, similar to that of the ILCS, the study reports an increase in crude mortality of 2/1,000 (Table 3). This translates around 52 thousand deaths per year. This figure for total crude mortality of all causes thus is 2.2 times larger than that found by the ILCS exclusively for "war-related" deaths.

Potential explanations for discrepancies between estimates

There are several reasons that one may not expect the ILCS results to be similar to those found by the Lancet study:

  • Different variables are measured. The ILCS number recorded "war-related" deaths whereas the Bloomberg estimates are for total deaths. It is not clear what deaths were classified by respondents to the ILCS survey as "war-related", but this is unlikely to include all deaths (and may not include all violent deaths, such as criminal murder). For this reason, the ILCS number should be expected to be lower than the total of "violent deaths" recorded in the Bloomberg studies.

  • The ILCS survey was not focussed on mortality and may not have complete reporting. Lancet study co-author Les Roberts gave the following explanation to a forum hosted by the BBC Website (full transcript available from MediaLens):

    I suspect that [the ILCS] mortality estimate was not complete. I say this because the overall non-violent mortality estimate was, I am told, very low compared to our 5.0 and 5.5/ 1000 /year estimates for the pre-war period which many critics (above) claim seems too low. Jon [Pederson, who led the survey] sent interviewers back after the survey was over to the same interviewed houses and asked just about <5 year old deaths. The same houses reported ~50% more deaths the second time around. In our surveys, we sent medical doctors who asked primarily about deaths. Thus, I think we got more complete reporting.

4: Are the study findings compatible with Iraqi Ministry of Health estimates of mortality?

Summary: The Iraqi Health Ministry recently stated that 150,000 thousand civilians had died as a result of "terrorist" violence in Iraq. The methodology used for this estimate has not been published, but the Ministry has said it is based on tallies of bodies brought to morgues. Previous experience of similar mortality estimates through passive surveillance of health facilities and morgues indicates that only a small proportion of deaths are captured through these methods, especially where conditions are unstable and administrative capacity limited. Given the likely under-reporting, it is not unexpected that the Ministry of Health number is lower than that of the MIT/Bloomberg study; one interpretation is that one-quarter of deaths are recorded centrally. Comparison is complicated by the fact that the Health Ministry estimate is of "civilian" deaths only, whereas the MIT/Bloomberg study recorded total deaths.

Recent estimates by the Ministry of Health

A statement by Health Minister Ali al-Shemari claimed in late 2006 that about 150,000 civilian Iraqis have been killed by insurgents since the March 2003 U.S.-led invasion.

Al-Shemari stated that his estimate was based on records from morgues and hospitals, which recorded around 100 bodies per day. No details have been published about the exact methodology used to derive the estimate. 150,000 corresponds to 20-25% of the figure estimated in the MIT/Bloomberg study.

(Confusingly, on 17 January 2007 it was reported that the deputy Health Minister Hakem al-Zamili issued an estimate of 12,357 civilian Iraqis killed by violence during 2006, "based on a compilation of numbers from the Interior and Health ministries". This corresponds to a much lower average daily rate than al-Shemari's earlier statement - around 30 deaths per day. This estimate was released in response to a much higher UN estimate of around 94 civilian deaths per day during 2006).

There are several potential explanations why an estimate of civilian deaths through hospital and morgue records is likely to be lower than the total number of deaths:

Civilian vs. total deaths

Al-Shemari stated that the Health Ministry estimate was of "civilian deaths". While the definition of "civilian" is unclear, it seems unlikely to include all deaths. By contrast, the MIT/Bloomberg study recorded deaths in all categories.

Incomplete reporting

The the reporting of deaths through hospitals and morgues is almost certain to be incomplete. There are several possible reasons that this may arise:

  • not all victims are be taken to morgues;
  • not all bodies in morgues are recorded;
  • not all records are be preserved;
  • not all locally held records are transmitted to central agencies; and
  • not all records held centrally are collated.

These and other difficulties are well-known to cause under-reporting of mortality, especially in unstable conditions. By way of illustration, Les Robers (a co-author of the MIT/Bloomberg study) gave an example in a BBC interview of the under-reporting of mortality through health facilities:

It is really difficult to collect death information in a war zone! In 2002, in Katana Health Zone in eastern Democratic Republic of Congo (DRC) there was a terrible meningitis outbreak, where the health zone was supported by the Belgian Government, and with perhaps the best disease surveillance network in the entire country. A survey by the NGO International Rescue Committee showed that only 7% of those meningitis deaths were recorded by the clinics and hospitals and government officials.

An indication that these difficulties are present in Iraq was given in the July-August Human Rights Report by the United Nations. This noted no deaths in the Al-Anbar Governorate in this period, even though this is known as an area with significant insurgent violence and an area with ongoing military operations. Similarly, of the 7,054 deaths noted in the Human Rights report for September and October 2006, 4,984 deaths or 71 percent were in Baghdad, even though Baghdad contains only 24 percent of the population. (By contrast, Al-Anbar was one of the Governorates with the highest death rates in the MIT/Bloomberg studies, while Baghdad was an area of medium mortality and violence.)

There also is anecdotal evidence that the systems for central collation of death statistics in Iraq never were very strong. In a companion document) to the MIT/Bloomberg study, the authors write:

Even with the death certificate system, only about one-third of deaths were captured by the government's surveillance system in the years before the current war, according to informed sources in Iraq. At a death rate of 5/1,000/year, in a population of 24 million, the government should have reported 120,000 deaths annually. In 2002, the government documented less than 40,000 from all sources. The ministry’s numbers are not likely to be more complete or accurate today.

Another consideration is that it is unclear which institutions are being used in hospital estimates. In previous cases, Bill Rammell (at that time a Foreign Office Minister) answered indicated that Iraqi Ministry of Health figures were taken from 180 hospitals. This is significantly less than the 240 public and 70 private hospitals that the Ministry of Health described as being in existence in January 2005.

In sum, the estimate quoted by Al-Shemari suggests that almost one-quarter of the deaths estimated by the MIT/Bloomberg study may have been recorded through morgues and hospitals. This proportion seems consistent with other experience of using passive surveillance through health facilities and morgues to estimate mortality.

Political considerations

Prior to Al-Shemari's statement, the Iraqi government had given no official estimates of overall deaths since 2003. One reason may be that mortality data are highly politicised, and therefore sometimes withheld. In October 2006, Nuri Kamal al-Maliki, the Iraqi prime minister, ordered the country’s medical authorities to stop providing the United Nations with figures on the number of civilians killed and wounded [link]. This is another indication that reliable numbers may be difficult to produce.

5: Are the study results consistent with the findings of the 2004 Lancet study?

Summary: the overall number of deaths in the 2004 and 2006 Lancet studies are very similar for the comparable time period, suggesting that the excess death findings are replicable. The causes of deaths differ between the two studies, but the differences not statistically significant

Overall number of excess deaths

The 2004 Lancet study found an estimate of 98,000 (8,000-192,000) excess deaths in the period March 2003 - September 2004. The authors note that the findings of the 2006 study for the same periods yields a very similar result:

Application of the mortality rates reported here to the period of the 2004 survey gives an estimate of 112 000 (69 000–155 000) excess deaths in Iraq in that period. Thus, the data presented here validates our 2004 study, which conservatively estimated an excess mortality of nearly 100 000 as of September, 2004.

The authors argue that there is a striking similarity between the 2004 and 2006 finding, suggesting that the results can be replicated.

Causes of death

One point that has been noted as a potential inconsistency between findings of the two studies is that the causes of death in the two studies differ. While the second study finds only a small proportion of excess deaths from non-violent causes, the first survey finds this as the major cause of deaths.

There are a two main reasons that this discrepancy may arise (we discuss the causes of deaths more in this FAQ):

First, while both studies found statistically significant increases in overall mortality, neither has the statistical power to make firm conclusions about most sub-groups of deaths (e.g., in a particular age group, or from a particular cause). For example, neither of the studies could conclude with statistical certainty that non-violent deaths had increased at all in Iraq since the invasion. This does not mean that studies "conclude" that non-violent deaths have not increased, but only that (formally) the studies do not have enough statistical power to reject the hypothesis that the incidence of such deaths has not increased. Findings about overall mortality nonetheless remain valid.

Second, the difference may result from variation in the causes of death over time. The 2006 study found that deaths from violence increased substantially each year since 2003, consistent with other evidence of increasing violence in Iraq. This development means that the proportion violent deaths would be higher in the 2003-2006 period than it was in the 2003-2004 period.

Third, the categorisation of deaths is likely to be much less precise than the incidence of death; household members are more likely to be certain about the fact that a family mamber has died than about the exact cause and circumstances of his or her death. The authors acknowledge this in their discussion of the 2006 study results, pointing to the possibility that "families might have misclassified information about the circumstances of death".

6: Are the study results consistent with the findings of the Iraq Body Count?

Iraq Body Count (IBC) recorded between 45 and 50 thousand media-reported civilian deaths as of November 2006. The Iraq Body Count explains the relationship between their estimate and the total number of deaths that have occurred in Iraq:

What we are attempting to provide is a credible compilation of civilian deaths that have been reported by recognized sources. Our maximum therefore refers to reported deaths - which can only be a sample of true deaths unless one assumes that every civilian death has been reported. It is likely that many if not most civilian casualties will go unreported by the media. That is the sad nature of war.

The extent of under-reporting can never be completely known, but the MIT/Bloomberg findings suggest that one-tenth of deaths are reported by English-language media sources. The Iraq Body Count has been critical of these explanation, arguing that such significant under-reporting would constitute an "abject failure" on the part of the media.

The MIT/Bloomberg study discusses this issue and suggests that this magnitude of discrepancy between deaths recalled by relatives of the deceased and deaths reported by the media is consistent with experience from other conflict situations:

Our estimate of excess deaths is far higher than those reported in Iraq through passive surveillance measures. This discrepancy is not unexpected. Data from passive surveillance are rarely complete, even in stable circumstances, and are even less complete during conflict, when access is restricted and fatal events could be intentionally hidden. Aside from Bosnia, we can find no conflict situation where passive surveillance recorded more than 20% of the deaths measured by population-based methods. In several outbreaks, disease and death recorded by facility-based methods underestimated events by a factor of ten or more when compared with population-based estimates. Between 1960 and 1990, newspaper accounts of political deaths in Guatemala correctly reported over 50% of deaths in years of low violence but less than 5% in years of highest violence.

One source of under-reporting is that journalists are not present in all of Iraq but concentrated to Baghdad. Gilbert Burnham, the principal author of the MIT/Bloomberg study said in the New York Times pointed to this as a potential explanation.

"We found deaths all over the country," he said. Baghdad was an area of medium violence in the country, he said. The provinces of Diyala and Salahuddin, north of Baghdad, and Anbar to the west, all had higher death rates than the capital.

7: Are the study results consistent with United Nations reports?

The United Nations has not carried out indpendent studies of mortality in Iraq (except for the findings of the ILCS, see this FAQ). However, the United Nations Assistance Missions for Iraq (UNAMI) compiles a Human Rights Report, and recent editions of this have included estimates of deaths from morgue counts, using data from the Medico-Legal Institute in Baghdad and (in recent months) Ministry of Health from hospitals throughout the country.

The below table summarises these estimates:

Period Number of
deaths reported
Source
Nov 2005 886 Medico-Legal Institute in Baghdad link
Dec 2005 787 Medico-Legal Institute in Baghdad link
22-26 February 2006 249 Medico-Legal Institute in Baghdad link
March 2006 1,294 Medico-Legal Institute in Baghdad link
April 2006 1,155 Medico-Legal Institute in Baghdad link
May 2006 2,669 Medico-Legal Institute in Baghdad,
Ministry of Health hospital data
link
June 2006 3,149 Medico-Legal Institute in Baghdad,
Ministry of Health hospital data
link
July 2006 3,590 Medico-Legal Institute in Baghdad,
Ministry of Health hospital data
link
August 2006 3,009 Medico-Legal Institute in Baghdad,
Ministry of Health hospital data
link
September 2006 3,345 Medico-Legal Institute in Baghdad,
Ministry of Health hospital data
link
October 2006 3,709 Medico-Legal Institute in Baghdad,
Ministry of Health hospital data
link
November 2006 3,462 Medico-Legal Institute in Baghdad,
Ministry of Health hospital data
link
December 2006 2,914 Medico-Legal Institute in Baghdad,
Ministry of Health hospital data
link

The reports recognise that this reporting system is unlikely to be complete. When the figures from the Medico-Legal Institute were first reported in Nov-Dec 2005, the report noted that "Such figures correspond to Baghdad Governorate only and they are believed to under-represent the actual number of casualties. Subsequent reports also noted the incomplete nature of a passive surveillance system. For example, the Jan-Feb 2006 report noted that figures "are not inclusive of many victims who were not taken to the Institute."

These considerations also appear in the patterns of the data. For example, of the 7,054 deaths reported for September and October 2006, 4,984 deaths or 71 percent were in Baghdad, even though Baghdad contains only 24 percent of the population. Similarly, the report for July and August 2006 reported no deaths in the Al-Anbar Governorate, even though this is one of the most violent areas of Iraq with ongoing military operations in this period. (The MIT/Bloomberg study found that Baghdad was not one of the most violent areas of Iraq, whereas Al-Anbar Governorate was.)

The figures are only a small fraction of the deaths estimated by the MIT/Bloomberg study. As discussed above, this proportion need not imply that the results are contradictory; rather, the significant under-reporting of deaths reported through health facilities or morgues in conflict situations is consistent with previous experience. See this FAQ for more discussion.