« NHS Competition. | Main | Operation Mirrorball »
May 14, 2005
Civilian Deaths in Iraq.
Something a little odd seems to be happening and I, for one, would like an explanation. Via Blithering Bunny I see a report in The Times:
THE invasion of Iraq and its aftermath caused
the deaths of 24,000 Iraqis, including many children, according to the
most detailed survey yet of postwar life in the country.
This is from a United Nations report which can be found in full here. This does seem to jar a little, be a touch at odds, with the Lancet report from last autumn. I’m aware that I made a complete mess of analysing that report (my article is here, my apologia here,) and I’m also aware that some (many?) would argue that whether 24,000 died or 100,000 did is irrelevant, there being no number of deaths that make the enterprise "worth it".
What occasions my surprise is that the report seems to be getting no airplay, certainly nothing like the attention that The Lancet report did. The two people who showed how wrong I was, Tim Lambert and Daniel Davies of Crooked Timber, seem not to have noticed the new report, certainly have not commented upon it. The title of the report is "Iraq Living Conditions Survey 2004" and a technorati search shows, as at noon BST Saturday, no mention of it (although that could be due to my own failures in searching). A Google News search gives 53 matches, very few of which are actually to do with the report. A standard Google search shows some comments upon the report but not that many and certainly nothing like The Lancet report had two days after its publication.
Now no, I don’t want to get all conspirazoid here, it may well be that the lack of MSM coverage means that people are unaware of the report as yet. Perhaps it will all happen next week as people find out about it. I do hope it will, for I would like to see the critiques of the methodology, the analysis, the hive mind of blogs chewing over this information and aiding me in understanding what has been said. From memory, the Lancet report had a 95% confidence interval of 8,000 to 198,000 deaths caused by the invasion. The Times reports the UNDP as stating:
The 370-page report said that it was 95 per cent
confident that the toll during the war and the first year of occupation
was 24,000, but could have been between 18,000 and 29,000. About 12 per
cent of those were under 18.
Well, who’s right? Is this new report an addition to the sum of human knowledge? Should we regard The Lancet report as a valid but unfortunately flawed first attempt at an answer? (After all, the new numbers are within the old confidence interval.) Is there something about different time spans that makes both of them correct?
Maybe it’s just me, maybe I’m way off base or something, actually wanting attention paid to this new report, perhaps the same amount of attention as was paid to the one that came out just before a US Presidential election. It would, of course, be way way to cynical to think that that might actually be the reason why there is not, as yet, such attention being paid.
Update: Welcome Instapudit readers. Do have a look around. It’s also worth mentioning that after I put this up and asked Tim Lambert if he had any comments yes, indeed he does. Worth reading as he claims the new report simply shows The Lancet was correct....for they only said that 33,000 had died in "war related deaths", something a little different from the 100,000 number generally thrown around (the others coming from disease, violence and accidents). Still no breakdown of how many of those deaths were in fact of troops and insurgents though.
Tim Blair also noted the same study (from the same blog I did, Blithering Bunny).
Update II. Shannon Love has further thoughts. John Quiggin seems to have misread what I wrote above.
May 14, 2005 in Military | Permalink
TrackBack
TrackBack URL for this entry:
http://www.typepad.com/t/trackback/23056/2452639
Listed below are links to weblogs that reference Civilian Deaths in Iraq.:
» New Study on Iraq War Deaths Estimates 24,000; Med from Brainster's Blog
He's right that the lefty bloggers, to the extent they will ever acknowledge this study (and I'm not holding my breath) will inevitably conclude that no amount of lives lost was worth it... [Read More]
Tracked on May 14, 2005 4:20:23 PM
» UN Estimates Number of Iraqi Deaths at 24,000 from Les Jones
Tim Worstall and Tim Blair have more. The new number is substantially below the 100,000 deaths claimed by a Lancet study that was based on telephone surveys of 808 Iraqi households in 11 provinces. The UN survey is based on surveys of 21,600 Iraqi hous... [Read More]
Tracked on May 15, 2005 12:27:22 AM
» What Was That Death Toll Again? from Just Some Poor Schmuck
The Lancet rushed to get out their estimate of 100,000 civilian deaths in Iraq before the 2004 Presidential election. Now the UN has come out with a study putting the death toll at between 18 and 29 thousand. Tim Worstall:... [Read More]
Tracked on May 15, 2005 10:12:14 AM
» Lancet/ILCS roundup from Deltoid
Jim Lindgren agrees with me that the ILCS supports the Lancet study. He also raises some concerns about some of the numbers in Lancet study: [Read More]
Tracked on May 15, 2005 7:27:27 PM
» Rooting for the Other Side from Last Chance Cafe
I'm getting bored with the UN Dispatch but will note that it links to an Instapundit post comparing the UN report on deaths in Iraq since the beginning of the invasion with a widely quoted report from the Lancet. It... [Read More]
Tracked on Jul 5, 2005 1:18:40 AM
» http://instapundit.com/archives/033162.php from Instapundit.com
A NEW LANCET STUDY ON IRAQI CASUALTIES: I'm skeptical, given their past track record. Meanwhile, Tim Blair notes the paucity of actual data: "Remember: Lancet came up with this via a survey that identified precisely 547 deaths (as reported by the New Y... [Read More]
Tracked on Oct 12, 2006 2:20:32 AM
» http://instapundit.com/archives/033162.php from Instapundit.com
A NEW LANCET STUDY ON IRAQI CASUALTIES: I'm skeptical, given their past track record. Meanwhile, Tim Blair notes the paucity of actual data: "Remember: Lancet came up with this via a survey that identified precisely 547 deaths (as reported by the New Y... [Read More]
Tracked on Oct 12, 2006 12:44:33 PM
» http://instapundit.com/archives/033162.php from Instapundit.com
A NEW LANCET STUDY ON IRAQI CASUALTIES: I'm skeptical, given their past track record. Meanwhile, Tim Blair notes the paucity of actual data: "Remember: Lancet came up with this via a survey that identified precisely 547 deaths (as reported by the New Y... [Read More]
Tracked on Oct 12, 2006 2:02:11 PM
Comments
It may yet get attention, but one mustn't forget how much effort was made by the Lancet not only to have the 100,000 study peer-reviewed in time for the election, but to make sure that it was noticed. (That's not a criticism of the methodology behind the report). The new figure is within the Lancet's bounds. From the point of view of that part of the press that took such unholy glee in the Lancet figures, the UN study tells the wrong story, and at a time when there is no bigger story to fit it within, I can't see it getting anything like the same coverage.
Posted by: James Hamilton | May 14, 2005 1:04:47 PM
The new study surveyed a much greater number of households, and did a better job getting a wide survey of all of Iraq. It's almost unquestionably a more useful survey; the confidence interval should provide some evidence that way.
Of course it should get more play, but it won't.
Posted by: John Thacker | May 14, 2005 3:04:11 PM
The Lancet number is for all excess deaths (which includes the increase in murder, accidents and disease) while the ILCS number is for deaths directly related to the war (which just includes deaths caused by the coalition and the insurgents). The time periods are also different. If you compare like with like, the ILCS closely agrees with the Lancet. More here.
Posted by: Tim Lambert | May 14, 2005 3:17:56 PM
So the new figure is on the same magnitude as the French civilian fatalities during the D-Day operation and following weeks -- about 20,000 innocent French men, women, and children killed.
Posted by: JimO | May 14, 2005 3:30:25 PM
James Hamilton has put his finger on what will surely prove to be the Mainstream rationale for spiking the UN study: "The new figure is within the Lancet's bounds." Nothing new here, move on, folks.
The principal confidence exhibited by the Lancet's interval is their supreme confidence that no one in the media is competent to grasp its meaning. An estimate of a large number whose upper bound exceeds the lower bound by two-and-a half orders of magnitude is a frank confession that the underlying data are worthless. Neat trick, to preserve one's reputation for scientific objectivity while releasing a figure whose only value is political.
Posted by: John Van Laer | May 14, 2005 3:38:54 PM
Personally, I like the idea of "excess deaths." When is Lancet going to come out with a study of the reduced "excess deaths" due to Saddam no longer murdering his people at the rate of 50,000+ per year? Or, since this appears to be "normal operating standards" under Saddam don't they count as "excess?" Maybe under Saddam an "excess death" only occurs when you die more than once.
I think we should start tracking the deficit in "excess deaths" sinc the fall of Saddam? At this point there would seem to be a minimum of some 50,000 "excess death" deficit due to the activities of American troops. Maybe some other ME dictator can take up the slack.
Posted by: JorgXMcKie | May 14, 2005 4:08:13 PM
How many innocents died in the American Revolution? Proportional to the population, that is? How many innocent farm boys died in the American Civil War? WWI? WWII?
How many people died on American highways in the same period for nothing more than the convenience of getting from point A to point B as quickly as possible?
Death is part of life. And, some things are worth fighting for. Only effete cowards elevate life above all else. If the US were governed by a malign dictatorship, I would welcome military action to depose it, regardless of the consequences. In the words of Victor Laslo:
"You might as well ask why we breathe. If we stop breathing, we will die. If we stop fighting our enemies, the world will die."
Posted by: Reid | May 14, 2005 4:30:41 PM
John Van Laer nails it. The Lancet study was garbage. Its methodology and conclusions were risible. And this new study, while using better methodology, still produces a comically large confidence interval.
The fact is that epidemiology is not suitable for this task. (Family questionnaires are still dubious in my book.) What is suitable is a systematic, in-country survey of official records. Until we get that, we should ignore the "scientists," who are merely leftist political activists trying to discredit the war effort. Junk science at its worst.
Posted by: Fresh Air | May 14, 2005 5:16:30 PM
Mortality studies are fairly simplistic -
1) Interview a sample population
2) Determine a mortality rate/1000 population
3) Multiply times total population
To determine "excess deaths" caused by an event,
Subtract previous mortality rate from current mortality rate, multiply times population.
Lancets study did not make any attempt to validate the pre-invasion mortality rate. They used Saddams "Official" number.
The UN Study re-evaluated the pre-invasion
mortality rate.
So the real question is, was the lancet study flawed, the UN study flawed, or Saddams Official Mortality rate flawed.
Posted by: Soldier's Dad | May 14, 2005 5:44:55 PM
I spent a fair amount of time on Lambert's site noting the various flaws in how the Lancet study has been interpreted.
The conclusion I came to is that the study has been almost universally misreported. What it actually says at 95% confidence (the bar for statistical significance) is that at least 8,000 more people and no more than 194,000 more people had died. The 100,000 number was better than a guess, but does not pass the bar of having statistical significance the way the 95% CI does.
One might ask, "What the hell good is a study that only gives us a lower bound of 8,000 and upper bound of 194,000?" The answer is: not much. Such a large interval tends to make the methodology more suspect.
This study appears to be more accurate. It's important to realize tho: these studies do not statistically contradict one another. The Lancet report gives such a wide range for its 95% CI that virtually any other study will fall within those bounds; as noted above, even accepting its methodology it just doesn't say much that is useful.
Posted by: TallDave | May 14, 2005 6:11:51 PM
(Above should read: Such a large interval also tends to make the methodology more suspect.)
I was going to try to post this on Lambert's blog and see what he thinks, but it seems to be down. Tim's reasonably honest; I think he would have to accept this CI as a better estimate than the 100,000 from Lancet (since the 100,000 is not a CI and the 18,000 and 29,000 numbers are, though this seems to be misreported in the original article here (they can't be 95% about the 24,000 number, only the range)).
Posted by: TallDave | May 14, 2005 6:23:38 PM
The number of deaths due to the war is irrelevant. What number would be acceptable to those who attempt to collate lives lost in the enemy camp with the objectives of the war. You cannot do this except in the very limited area of troops lost, which determines whether a particular battle can be won. The only valid number of casualties to be concerned about is our own.
Those who complain that no war is justified because there are casualties evade the fact that there are times in human events when evil must be confronted, no matter what the cost. They would lose their lives with such an evasion if there were not others who willingly faced the dangers of evil--there is Justice in reality, no matter how much one attempts to ignore it. The intellectuals who preached the philosophy of Nazism and Communism were among the first to be murdered.
Americans go to a great deal of time, effort, and expense to keep casulaties among the enemy down, not for the sake of others, but for the sake of our own people. War is horrific enough without further burdening those who must do the fighting; the nightmares borne of slaughter are to be mitigated as much as possible.
I sympathize with any person who finds himself living in the horror of a tyrany such as Saddam's. Like a cancer, however, it must be cut out if the pain is ever to end and the healing is to begin. Life isn't always pleasant; happiness must be earned.
Posted by: Dee Bates | May 14, 2005 6:48:38 PM
The number of deaths due to the war is irrelevant. What number would be acceptable to those who attempt to collate lives lost in the enemy camp with the objectives of the war. You cannot do this except in the very limited area of troops lost, which determines whether a particular battle can be won. The only valid number of casualties to be concerned about is our own.
Those who complain that no war is justified because there are casualties evade the fact that there are times in human events when evil must be confronted, no matter what the cost. They would lose their lives with such an evasion if there were not others who willingly faced the dangers of evil--there is Justice in reality, no matter how much one attempts to ignore it. The intellectuals who preached the philosophy of Nazism and Communism were among the first to be murdered.
Americans go to a great deal of time, effort, and expense to keep casulaties among the enemy down, not for the sake of others, but for the sake of our own people. War is horrific enough without further burdening those who must do the fighting; the nightmares borne of slaughter are to be mitigated as much as possible.
I sympathize with any person who finds himself living in the horror of a tyrany such as Saddam's. Like a cancer, however, it must be cut out if the pain is ever to end and the healing is to begin. Life isn't always pleasant; happiness must be earned.
Posted by: Dee Bates | May 14, 2005 6:48:38 PM
Saw Lambert's post above. He seems to be at least partly correct in that it's questionable how much overlap there is between what Lancet attempted to measure and what this attempts to measure.
Posted by: TallDave | May 14, 2005 7:00:42 PM
TallDave--
Ignore the Lancet study. It has been thoroughly debunked and isn't worth the space on your hard drive to download.
The second study, while more carefully prepared, still uses dubious methodology and, as an earlier poster pointed out, relies upon Saddam-era prewar data. If Saddam couldn't keep track of something as near and dear to him as his own weapons, why should we expect him to accurately record something he cared little about, namely the mortality rates of his people?
Simply throw the bathwater out with the bathwater; there is no baby there.
Posted by: Fresh Air | May 14, 2005 7:33:56 PM
Hi Tim and TallDave,
Tim Lambert misses the point. This study is not only unquestionnably much better, the fieldwork was done before that of the Lancet study.
Time and time again, defenders of the Lancet study have been saying that they'd love a better study, but this was the best there was, and considering the conditions on the ground they did well.
Complete bunk as it turns out. This study interviewed more households in each of Iraq's 18 governorates than the Lancet study did in total in the 11 governorates they dared to go to.
Finally, the Lancet study's sample size is so tiny you can't find a vaguely reliable estimate in it for "war-related" deaths that would be comparable to this vastly superior study.
I guess Tim Lambert chose to (largely at least) base his 33,000 on the coalition inflicted deaths ex Fallujah in the Lancet survey, a grand total of 9 (composed of 3 shooting incidents, with one death each, and 3 bombings, one with 3 dead, one with 2 dead and one with one dead).
It's laughable to claim that this number validates the Lancet study by comparison with the UN number.
Posted by: Heiko | May 14, 2005 7:38:31 PM
If I recall correctly, a lot of electronic ink was spilled defending, not just the Lancet report, but their skewing towards the 100,000 (roughly mid-range in the parameters, but still a heck of a lot, considering news coverage yielding estimates much closer to the 15-25,000 figure. I don't blame the Lancet report authors for rushing into print. I don't blame the people who used it. But, after looking at the methodology at the time, and the caveats listed, it seemed crackers to invest as much energy, faith and furor on the results as was bestowed on it.
BTW, a previous commenter talked about the 95% level of confidence as the bar for statistical signficance. It isn't. Lots of good decisions get made on studies with 90% level of confidence.
Posted by: Tom Fuller | May 14, 2005 9:21:53 PM
Tom--
You are wrong. 95% is the standard for publication in every respectable scientific journal. In fact, lowering their standard to 90% on a secondhand smoking study got the EPA in tons of trouble with a U.S. District Judge.
Posted by: Fresh Air | May 14, 2005 9:50:49 PM
Heiko:
1) Perhaps a little bit of humility is in order from you given that this new survey completely vindicates Roberts et al's estimate of infant mortality prewar, which I seem to remember was your personal obsession.
2) It is visible that the numbers aren't comparable. Tim Lambert links to the questionnaire used and it's very clear that the UNDP survey is asking a question which would only pick up a subset of the Lancet study's deaths (and that within this subset, the two studies agree very closely).
Tim adds: Tiny thing, I know, but the post above also links to the UNDP report.
Posted by: dsquared | May 14, 2005 10:19:09 PM
Key point: civilian AND military deaths!
from the report...
"The number of deaths of civilians and military personnel in Iraq in the aftermath of the 2003 invasion is another set of figures that has raised controversy. The Living Conditions Survey data indicates 24,000 deaths,"
Well well well. How many of those 24,000 do you suppose were from the furious American attack againt the Iraqi military forces? And how many of those 24,000 do you suppose were civilians killed by car bombs and other assorted mayhem by the barbaric anti-American forces?
The simple fact is the American operations in Iraq have taken more care to avoid civilian casualties than any other military operation in history. That the anti-war types continue to try and smear the American forces with false claims of "carpet bombing" and other "war crimes" is an indicator of how much the anti-war types are really anti-American.
Posted by: Brad | May 14, 2005 10:37:56 PM
I agree with Dee Bates but for different reasons. The number doesn't matter. Whether 1 innocent person died or a million, the Iraqis will decide whether it was worth it. Unlike WWII, where we were liberating people under foreign influence or the American Revolution where we, the people, revolted, in Iraq we decide what was best for the Iraqis, we gambled with their lives. I'm betting 10 years from now, they'll be thanking us but whether it was worth it is ultimately their decision regardless of the raw numbers.
Posted by: Eric Pickup | May 14, 2005 11:10:59 PM
Brad--
Well said.
Posted by: Fresh Air | May 14, 2005 11:11:45 PM
Brad, that is a good point. The Lancet study does not assign blame. Is the US really culpable for civilian casualties deliberately caused by people fighting against a democratic Iraq? Or (as I would argue) do they deserve credit for the fact Iraq is a democracy now, and more for helping to keep it that way?
I think it's important to understand the Lancet study rather than dismiss it out of hand. First off, as I said above, it does not say much that is useful, just that at 95% confidence (the bar for statistical significance) at least 8,000 more people and no more than 194,000 more people had died than the period before the war. That's if you accept their methodology; there are such wide ranges of pre-war child mortality rates published that that stat alone nearly makes the study meaningless.
Posted by: TallDave | May 15, 2005 12:44:20 AM
Eric,
Also a good point. I've pointed out on lefty blogs that 400,000 Americans died directly in battle out of a population of 40 million to free 5 million slaves in the Civil War, and no one questions the morality of the U.S. unilateral invasion of the Confederacy. Yet somehow 35,000 killed in Iraq to bring freedom and democracy to 25 million is too high a price for liberty?
Posted by: TallDave | May 15, 2005 12:48:49 AM
It is my understanding The Lancet article was pretty much discredted because of flawed methodology.
Lancet did not do the study, but was done by a group from Johns Hopkins. Lancet may have added their own touch before publishing.
Chip
PA
Posted by: Chip | May 15, 2005 7:34:46 AM
The number of births recorded in the Lancet study is too small for a reliable estimate of the infant mortality, and the claimed increase largely explained by a handful of war related incidents.
When I thought this through a few months ago (a few simple spreadsheet calculations give a good idea of how huge sampling error is when you've only got 8 deaths out of 300 or so births), I realised that the infant mortality number estimated by the Lancet wasn't worth worrying about.
As it turns out, even much, much better studies that at least have a sufficient sample size haven't yet produced a reliable number. As the UNDP report writers point out, there seems to be a systematic undercounting when considering the South of the country, and their numbers are much lower for the 90's (29 in 89-93, 32 in 94-98) than previously published work. Yet, they've actually re-interviewed people, and covered the whole country with clusters of only 10 people each, and therefore over three times the number of clusters per governorate that the Lancet study had for the whole country.
I now think that the infant mortality numbers in Iraq are extremely uncertain, anything between 20 and 120 being quite possible.
WHO and UNDP in this study provide confidence intervals, but clearly these must exclude a lot of potential systematic error and therefore do not give a good idea of the real error span.
The UNDP study is massive, considering the smaller cluster size (10 rather than 30 households per cluster), it covers over 2000 clusters, compared to 33 in the Lancet study, and just over 20000 households compared to around 900 in the Lancet study.
It is therefore the equivalent of redoing the Lancet study something like 30 times. Only that it wasn't "redoing" it, no, the fieldwork for these over 30 studies of the power of the Lancet study was done before the work reported in the Lancet.
Sorry, I find it risible to say that this massive effort "validates" the Lancet study. No, it shows what a, by comparison, poor and politically motivated effort the latter actually was.
It also shows the huge uncertainty attached to estimates of death rates (or some other indicators such as nutrition) based on such studies, even those where random sampling error isn't such a big issue as with the Lancet's tiny sample.
Posted by: Heiko Gerhauser | May 15, 2005 8:18:22 AM
Heiko, it validates it because where it measures the same things, it gets the same numbers. And if the Lancet study was so poorly done, how come the ILCS came up with the same numbers. I also like the way you have changed your arguments about the infant mortality numbers. Before, they proved there was something wrong with the study because they contradicted the UNICEF numbers, now it's just that the sample size was too small. But the ILCS with a much larger sample gets the same number.
Posted by: Tim Lambert | May 15, 2005 8:52:41 AM
You are right I have changed my view of the UN's infant mortality numbers and of what the Lancet's infant mortality estimate means.
I didn't initially consider how important the sample size was in this case.
Open a spreadsheet, put rand() into column A, and =if(A1<0.03;1;0) into column B, drag this down 240 cells and add up column B (then divide by 240 and multiply by 1000), you'll get numbers like:
25
13
21
13
42
38
46
This assumes in effect perfect sampling, and shows up the effect of random error.
The basic message is that if the Lancet did find the right value for infant mortality pre-war, and it was around 30, it did so by good luck. It could just as easily have found the number to be 46 or 13.
Also, the post-war increase is largely explained by a handful of war related incidents. Nutrition etc.. are not the main factor responsible for the claimed increase based on the 21 deaths they found. And as for the pre-war rate the amount of random fluctuation for such a small sample is again huge.
The UNDP has a similar number as the Lancet for pre-war infant mortality, and on a much more solid footings. This couldn't just be due to random sampling issues, the sample is far too big for that (do a similar exercise in a spreadsheet for 4000 births, and the vast majority of your estimates will be within +/- 10% of the true value).
But it varies likewise from previous estimates, they get very different numbers for the 1990's than other sources, which implies huge systematic error either in UNDP's recent study or in earlier figures published by WHO. Both give 95% confidence intervals, which to me makes clear that these intervals only account for a narrow subset of sources of error (ie probably largely or exclusively statistical, rather than systematic).
I don't conclude from this that the Lancet study was "right" on infant mortality, I rather conclude that we've got to regard these statistical confidence intervals with a grain of salt, and that the true number for infant mortality cannot be easily obtained even with a large study.
Bottom line is, the Lancet study is too small to provide useful figures, the subset you refer to Tim (war related deaths) is not clearly defined, making a comparison with the Lancet's subsets difficult, and at any rate, for subsets of the 100,000 figure in the Lancet, the sample size becomes even more of an issue, ie even if the Lancet should have gotten a subset right (say accidents) it would have done so merely by good luck and could just as easily have gotten it wrong by a large factor (even for perfectly representative sampling and no systematic error).
Posted by: Heiko Gerhauser | May 15, 2005 10:10:05 AM
You don't have to do any experiments with spreadsheets -- the 95% CI was reported in the study. That tells you that it was only a rough estimate, but also that it was not meaningless. And this estimate was confirmed by the ILCS.
Posted by: Tim Lambert | May 15, 2005 10:24:42 AM
Well, yes, 0-64, it just so happens that I find the spreadsheet calculation quite intuitive and helpful.
I have to disagree on how useful the estimate is, and what the UNDP figures show.
What they show to me is that systematic error is a very large source and that neither the UNDP's nor WHO's figures for infant mortality can be trusted.
On top of that, the Lancet study has a tiny sample resulting in this humongous confidence interval without even taking the issue of systematic errors into account.
Bottom line to me is the Lancet study is tiny and this new study is the equivalent of some 30 Lancet type studies. Unlike the Lancet's work, they didn't rush to publication just in time for the presidential election, in spite of having done their field work a bit earlier.
The Lancet study is far from the best we've got, all you had to do was wait for better and more thorough work to be published.
Posted by: Heiko Gerhauser | May 15, 2005 11:25:35 AM
Fair call, Tim. I've edited my post.
Posted by: John Quiggin | May 15, 2005 12:16:06 PM
To Heiko and Tim,
Any good scientist/statistitian will tell you that the sample size determines the "power" of a study/experiment. The design of the experiment gives you the confidence to draw conclusions. When your sample size is too small your confidence limits are too wide numerically. This is how the Lancet study ended with such a huge "range". This range makes the study almost useless since the 100,000 figure is just a "middle" figure from an unacceptably large "range" (sorry to use lay terms instead of more accurate statistical terms). To claim that the new study falls within the Lancet range only shows how useless the Lancet numbers were.
Here is an example:
If I survey the deaths of 25 British men and find that they died between the ages of 20 and 80, I could claim that the "middle" (I'm using this term carefully since I'm not doing the actual statistics on this) dying age was 50. However, with such a small sample size the range for this study would be huge at 95% confidence. So, I could conclude 50+/-25 as age of death, as an example. In and of itself, the uncertainty of this huge range would make my study useless because it would mean that the "actual" number could be 25 or it could be 75 but my study does not have the power to tell me. Now if I do the same study with a sample size of 20,000 and I get a "middle" number of 50 my range would be much smaller indicating a greater confidence in that "middle" number. For example, 50+/-5 would give me great confidence in concluding that the "actual" number is 50.
After all, statistics tries to provide us with confidence that the number that we arrive at in our study is closest to reality. In the Lancet study the confidence is almost non-existent judging by the range. To say that the new study falls within the range of the Lancet study only shows the Lancet weakness and lack of confidence. If the new study had come up with a number of 66,000 you could have said the same about the Lancer study. To say that any given Lancet number is the "same" as a better study only says that it is a coincidence of statistics, given the poor Lancet confidence.
That's how statistics works.
Posted by: Marv | May 15, 2005 12:55:25 PM
Isn't another real issue WHY our military refuses to report the death toll of our "enemy" that they have had all the way along? How do conservatives justify this? And how do they justify President Bush's arrogant refusal to honor the war dead as has every other President in the last century? Just curious...
Posted by: activist kaza | May 15, 2005 4:48:50 PM
Hi "activist",
the military cannot provide accurate figures for insurgents/civilians killed by them. They've got no way of verifying the number of deaths and whether a particular person was in fact an insurgent, and for rather obvious reasons, insurgents don't want to be known as such by the authorities making it rather hard to ask them.
The military does follow up cases where innocent by-standers have been killed in order to pay compensation, and the Iraqi government is behind the UNDP survey and also tallies civilians killed, and its own military/police losses.
I don't see any way to accurately count insurgent deaths at this stage. Any attempt to do so would just result in the military being accused of inflated body counts and lying about the kill ratio. So they naturally prefer to keep it rather vague.
Posted by: Heiko Gerhauser | May 15, 2005 7:15:15 PM
Just as a reminder, the original Johns Hopkins study, as reported in Lancet, concluded that the invasion had caused 98,000 additional deaths in the 17.8 months following the March 2003 invasion.
That’s usually rounded off to 100,000 extra deaths, as compared to the number of deaths that would have been expected in the same time length just before the invasion.
The Johns Hopkins (Lancet) team threw out its Fallujah figures because they looked far too high. The team’s conclusions were based on 89 deaths reported by 958 Iraqi households, and extrapolated to represent all of Iraq.
Had the research team also included the 53 deaths (52 of them violent) reported in Fallujah, they would have extrapolated an additional 200,000 extra deaths, for a total post-invasion death toll of 300,000.
But OK, stick with the 100,000 for the sake of analysis. That’s what the Johns Hopkins (Lancet) team did.
The research team found that of the 100,000 additional deaths, 60.07 percent were due to violence. That is 60,070 extra deaths from violence alone. Trim that down for the 13 months of the U.N. study, and the Lancet number is equivalent to about 43,000 war deaths.
How does 43,000 compare to the new U.N. figure of 24,000? They’re in the same universe, but one is almost double the other.
The new U.N. study interviewed 21,600 households. The old Lancet study interviewed only 958 (or 988, counting Fallujah).
I’d give the edge to the new study. But keep the studies coming.
Posted by: Free Frank Warner | May 15, 2005 9:00:30 PM
Perhaps this is too obvious a question, but is there a good reason why the death count is done indirectly, and not through things like death certificates? In other words, when US health officials want to find out how many people died in traffic accidents, they don't go around to a couple hundred households and ask them whether anybody in the family died in a traffic accident. I have read that a number of the deaths reported in the Lancet study were backed up with death certificates. Why aren't they the primary data? One hundred thousand is a large number of extra people to die; surely there are records surrounding most of them?
Posted by: Brainster | May 16, 2005 6:47:04 AM
Hi Brainster,
I am not sure in the case of general mortality, but for insurgents at least it's easy to see why they'd want to escape official scrutiny.
Posted by: Heiko Gerhauser | May 16, 2005 6:21:52 PM
