A brief analysis of the "600,000 dead" claim

First off, I apologize for my brevity here, but I wanted to share this despite my lack of time to write about it fully. Here goes: You've likely heard of the recent study claiming that upwards of 600,000 people have died as a result of the war in Iraq. While there has been substantial coverage, I suggest checking out the actual report if you are so inclined. I did, and I also asked one of my professors (Doug Rivers, who apparently analyzed this study for CBS) what his take on it was. The short of it is this: there are certainly a few problems. For one thing, they are quite disingenuous regarding the precision with which they present their results (the statistical margin of error is something like 380-940 thousand). Additionally, their methodology is not entirely transparent (I'm particularly curious as to how they went from their study of 1800-some households to a death rate/1000 measurement), and there are also some occasionally dubious techniques (some of their sample was not truly random but rather they chose every fiftieth neighborhood) as well as some unaddressed possible sources of error (incentive for both the pollers and pollees, all Iraqi, to exaggerate the death rate, or overly attribute it to the coalition). The latter issue is likely the most substantial, but as soon as you start trying to attribute "responsibility" for death you open up a philosophical Pandora's box. And there is also some strangeness with the death certificates, where they both rely on it to show their own validity yet reject it as its own independent measure. But philosophy and other quibbles aside, the bottom line seems to be that there is arguably at least some legitimacy to this study. Certainly their premise - that passive observation of death rates (just watching the headlines and adding up the numbers) is not adequate and a comparative and definitive pre-war to post-war death rate is superior - is plausible (observational counts are extremely vulnerable to issues of inadequate reporting, which is arguably severe in Iraq for most regions outside of Baghdad). Whether or not their specific methodology lead to inflated results, it seems safe to say that their general approach is interesting and most definitely suggests results far different from existing measures (Iraq Body Count, etc.). What to conclude from all of this is, of course, that complicated problems require complicated responses, and you cannot boil down this situation to "600,000" or "50,000" or anything else comparably simple. But whatever reality is, this study I would argue gives us strong reason to reconsider the more conservative passive/observational counts. My hope is that further scholars attempt similar surveys, and hopefully manage to patch up a few of the methodological issues. But even if that doesn't happen (and given funding issues and the instability in the region it probably won't), it is worthwhile to discuss the implications of this study and consider what policy prescriptions are appropriate (something they pretend to do at the end of their paper but I would say really don't). My abbreviated view: whether or not there really have been 600,000 deaths largely attributable to the Iraq war, this survey definitely suggests that it is popular Iraqi conception that there has been (and the presence of death certificates in 92% of the cases shows that certainly a great deal of death is happening, whether it is the fault of the US or not). As such, we need to take steps to disassociate the coalition with these deaths. This could mean redistribution of troops to be more outside Baghdad, perhaps even in Kuwait, a la this proposal by Jim Fearon which I discussed back on Tacitus (warning, PDF). The result will be that violence will not subside (if anything it will escalate), but it will become clear that it is the result of sectarian strife and not the evil American empire. This is a necessary step towards the long term goal of eventual true stability: people in Iraq are itching to fight a civil war, and until we let them sort it out all we will be able to do is put a mild damper on violence, not solve it. Anyway, I'll cut it off here. I suppose I'm not really being that brief, but rather my thoughts are not as refined as I would prefer. In any case, I still felt like it was worth sharing, and I hope folks out there agree. Thank you for reading.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Or it's wrong from the word go

(#541)

Iraq Body Count

I'm trying to make the best out of a bad situation. I don't need to hear crap from a bunch of hippie freaks living in denial! Screw you guys, I'm going home!

Okay, but...

(#552)

The Iraqi body count only counts deaths that are documented by western media. Since mobility is so poor for these organizations, it is obvious that although this is very valuable number, it is also a drastic undercount.

Also, I am yet to hear even an estimate of how many Iraqi soldiers died in the original invasion. The US military has estimated that the death toll in the first Gulf war was approximately 100,000. How many died this time?

"I don't want us to descend into a nation of bloggers." - Steve Jobs

Given that most of the Desert Shield and Desert Storm

(#560)

deaths came from the month long air campaign and there was no such campaign this time, probably about a tenth that number. Typically (well, as typical as any war can be), civilian deaths run at about half to three quarters the military deaths.

That is, for the period 20 March-14 April 2003 about 10K Iraqi Military and 5-10K civilians with the lower figure more likely. Most of the big fights took place outside major cities except for Baghdad proper and that fight didn't last long enough to account for many civilian deaths.

To date, the coalition killed count is 3005; Iraqi Security forces are estimated at 5,532.. Civilian deaths are estimated at 18,321 for the past year. (LINK).

Extrapolating that civilian figure as an average for the 3.5 years, that gives you about 64,000 probably low. My guess is that aboiut 100,000 would be closer. The 665,000 figure in a low intensity conflict (and it is very low intensity) is more than the UK lost, military and civilian, in five years of high intensity conflict in WW II It is a highly improbable figure.

10% of Rwanda's population

(#561)

was killed in 2 months (from memory so correct me if I am wrong). You can't compare this with WW2 because here you have a civil war. Most of the killings today are between iraqis. Within the UK during WW2 you did not have militias killing each other on a daily basis.

This place is my vacation.

What's the relevance of Rwanda?

(#563)

That was calculated genocide. If coalition troops were doing the same in Iraq, perhaps the results would approach the numbers guessed at by Lancet.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Not coalition troops

(#564)

This is not about the coalition troops, but ALL deaths in Iraq, including those caused by other iraquis.

This place is my vacation.

Is it calculated, concerted genocide?

(#654)

Rwanda was.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Not sure about your adjectives but

(#1365)

East Timor was a brutal genocide of impoverished Christians ripe for the robbing (12-25% population killed, and lots of gold and offshore oil/gas reserves known even back then) by Suharto's Indonesia, backed by US leaders (Henry K. thru Bush 1) who had a very significant hand in bringing it on, a la the well established Guatemala and Chile paradigms.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

PS: Cluster sampling survey methods were used

(#1374)

to assess the magnitude of East Timor genocide of predominately Roman Catholics by the Suharto Govmt.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Completely OT

(#1463)

We were talking about whether Rwanda can be compared to Iraq. I suggest you start your own thread, perhaps beginning with "Bush lied, chickens fried" or "No oil for gas".

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Both Rwanda and East Timor genocide deaths were assessed

(#1518)

with same survey techniques that Burnham, Roberts et al Lancet paper used.

That is the connection, in case you might have lost the thread of conversation.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

PA, the thread's all yours

(#1575)

Don't be timorous in how you use it.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

No. Henry K. and descendents backing a repressive

(#1587)

Islamic dictatorship to 'cleanse' 25% of those annoying uppity [some even said (sotto voce, with 'Cordesman' gravitas) they might even be Commie dominoes] Christian East Timorese squatters from their [our] resources , for the benefit western mining companies and Australian oil interests?

Who would have thunk it?

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

You're changing the subject again, PA

(#1594)

Now it's about Henry Kaiser and his decedents. Please seize on a subject and stick with it. I promise to listen even if it's just you doing the talking.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

[You are being coy] K is for Kissinger, of course

(#1595)

and his descendents are mostly Republican leaders, plus a few sleazy Democrats thrown in for bad measure.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Of course the number of deaths cannot be compared with WW II

(#578)

more people were killed in one day -- on several occasions -- than have been killed in Iraq since April 2003. that was exactly the point.

The militias killing each other today account for approximately 3,000 per month by most media accounts. that equates to about 100 per day -- which also tracks with most media accounts (and I include Arab TV Web sites). While everyone seems to agree that there is a current spike in that killing (also aimed at the election here but that's another story), lets accept the one hundred a day figure from 20 March 2003 to date; that's about 1,301 days, thus you have 130,100 dead from mostly sectarian or revenge killings. And that number is bound to be high as those kinds of killings really took off only this year.

Add in all the Iraqi security force deaths, 5,000 plus and about 5-10,000 form the initial attack and a like number (probably high, but I'm easy) in coalition caused deaths since then and you get approximately 155,000. As I said, that IMO is way high but it still is an awful long way from 655,000.

As I also said, not a big issue to me, let the Statisticians get hold of it and we'll see.

but add in a large number

(#600)

of deaths from general lawlessness (the major cause of death for most of the occupation according to most sources I read), add illness and we're certainly getting into their confidence interval.

It will be interesting to see it picked over.

Lancet study author Gilbert Burnham addressed the

(#769)

WWII (and other modern conflicts) war death numbers and figures comparison point that a CSPAN WJ caller also raised at 2:08:30 of the CSPAN Lancet author interview video, from Burnham's viewpoint as a demographer.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Soviet Russia

(#581)

would be a better place to look for analogies. With the exception that there's no equivalent of the Einsatzgruppen among Coalition forces (in spite of what some say), the situation on the ground is more analogous. Close-support air and artillery in urban areas, use of human shields by the defenders, large amounts of secondary violence (i.e. execution of traitors/informants, etc.), heavy fighting in populated areas, large numbers of nonuniformed combatants.

The UK by contrast never had an occupying army and had to deal only with strategic (i.e. poorly targeted) bombing.

Nearly 12 million civilian deaths in Russia, 11 million military. So much for half to three quarters...

M Aurelius was probably right.

Disagree. While many points you make are correct, you forget

(#587)

three very important ones.

- The Soviets probably came close to killing as many of their own citizens as the Nazis did. They had no compunction about bombing or shelling their own civilians to get at a few Nazi troops. Nor did the Nazis with all their faults rely often on human shields.

- The lethality and accuracy of WW II munitions meant, by default and from both sides unto themselves as well as each other, many more casualties than todays cause.

- The UK also had a number of overseas possessions that got overrun by the Japanese who did all those things you cite and more. The civilian toll was significant. I said UK military and civlian advisedly. Not to mention that the then UK subject numbers come far closer to Iraq's present day population than did the USSR...

And the half to three quarters is a general rule of thumb. It was almost as high in Korea as it was in the former USSR but it was lower most other places. We have gotten more genteel in our war making since WW II. In any event, a mean is a mean, I mean...

Your first point

(#596)

definitely applies in Iraq. The other points are allowable, but overall the Soviet front is a far better analogy than the UK itself: certain parts of the commonwealth in asia come closer.

M Aurelius was probably right.

Well, Gee, thanks for the certain parts of the Commonwealth...

(#670)

Actually, back then it was an Empire still, and the King of the United Kingdom of Great Britain and Northern Ireland and Emperor of India, George VI lost a bunch of civilians in Hong Kong, Malaya, Singapore, Sabah, Sarawak and Burma -- not to mention at Sea. That's in addition to those lost to bombs, V1s and V2s in the UK proper.

You use your analogy, I'll use mine -- and mine's population is a closer fit. :)

But then you knew that...

The fact that six years of high intensity warfare worldwide caused fewer military and civilian deaths to the UK (or British Empire if you wish) than this flaky study purports have occurred in Iraq in three and a half years is still the point.

But not in the Soviet Union, hence my point.

(#722)

I'm going by Wikipedia figures if that's good enough for guvmint work

http://en.wikipedia.org/wiki/World_War_II_casualties

And they're including colonial civilians killed in the numbers: 67,800 all told UK civilians.

But look at China (7 million civilian, 3 million military); Soviet Union (11,500,000 civilian, 10,700,000 military); Yugoslavia (514,000 civilian, 446,000 military); Indonesia (0 military; 4,000,000 civilian); Belgium (52,000 civilian, 12,100) and a pattern emerges. Large scale blitzkrieg campaigns saw big numbers of civilian deaths; slow, brutal attrition campaigns saw huge numbers. Places infantry fighting never got close to (Brit Isles; US; Japan) saw low mortality rates.

Iraq has had both blitzkrieg and long, brutal attrition for several years now. It's not beyond reason to start expecting large civilin numbers: certainly larger than official estimates.

M Aurelius was probably right.

Our memories differ...

(#599)

There was a very intense air campaign during the invasion. I remember U.S. military updates on the results of the bombing... the Republican guard ring around Baghdad is suffering a 30% attrition... 50% attrition... 70% attrition...

"I don't want us to descend into a nation of bloggers." - Steve Jobs

Not sure our memories differ. Based on what you say, it

(#760)

seems our belief in what the air Forces -- and the clueless Media -- say versus reality may be the disconnect. Do recall that attrition was runnaways as well as killed and they were all running about that time...

Do the math, pre Desert Storm, we bombed for over amonth; pre Iraq Freedom, less than four days. Consider alos the type of munitions, DS almost all dumb bommbs; OIF, mostly guided stuff.

In Desert Storm, not counting missiles, allied air forces dropped over 88,500 tons of ordnance on the battlefield (LINK) . Add the missiles and you probably have around 95,000 tons.

For OIF, somewhere at the time; I picked up the figure of ~12,000 tons for the 20 March - 14 April time frame. Can't seem to find it anywhere now but looking at this (LINK .pdf) (Scroll to page 11, the numbers look about right). then there's this (LINK) , note they bragg about dropping a million pounds of munitions -- that's hype, that's only 500 tons; less than they dropped in Afghanistan.

No way that OIF compares to DS in any way on tonnage -- or probable casualties.

I found an estimate

(#867)

Given by George Bush in a speech, but not considered the "official government estimate": 30,000 Iraqi military dead in initial campaign.

"I don't want us to descend into a nation of bloggers." - Steve Jobs

I remember that, possible. Doesn't change what I said.

(#872)

There is a lot more

(#544)

than just some legitimacy to this study. Sadly the Lancet study has become, in the words of Tim Lambert, like "flypaper for innumerates" (and no, I'm not including you on that list but most of the criticisms are based on simply not understanding how these studies are made and used).

In the end though I think Daniel Davies has it right when he says that the point estimate is not the important result from this study, rather that there are real and large 'excess deaths' and, more importantly, the rate of violent deaths is rising.

This place is my vacation.

Agreed

(#547)

The page on Iraq Body Count that folks are linking to as a critique of this study lists the following four implications if the study is valid:

-----------
* incompetence and/or fraud on a truly massive scale by Iraqi officials in hospitals and ministries, on a local, regional and national level, perfectly coordinated from the moment the occupation began;
* bizarre and self-destructive behaviour on the part of all but a small minority of 800,000 injured, mostly non-combatant, Iraqis;
* the utter failure of local or external agencies to notice and respond to a decimation of the adult male population in key urban areas;
* an abject failure of the media, Iraqi as well as international, to observe that Coalition-caused events of the scale they reported during the three-week invasion in 2003 have been occurring every month for over a year.
-----------

It then labels these as "improbable", which is dismissive to say the least. Certainly widespread failure of the Iraqi infrastructure is not only probable but expected these past few years, and is in fact explicitly discussed in the report itself. IBC tries to present it in as unfavorable a light as possible, but even their own points read to me as not being so improbable but rather quite possibly reflective of reality (the first point being the result of a destroyed infrastructure and economy and the ensuing corruption, the second being a result of sectarian violence [which I would certainly describe as "bizarre and self-destructive behavior"], the third being a mix of the previous two and the fourth just being the media doing its job as usual).

The issue is that this report has a conclusion which many people viscerally disagree with, but they cannot be bothered to study the statistics necessary to truly understand the argument. And so you end up with a variety of responses that really are just rhetoric and have nothing to do with the issues at hand.

My critique was meant to be (vaguely) academic, and I do think there is some "wiggle room" in their study that would ideally be cleaned up. However, the "wiggle room" is nowhere near severe enough to dismiss the results offhand like many people are attempting to do, and that is why I belief the ramifications are worth discussing.

I urge people to not be fixated with the number itself: as you said, the point estimate is not the point (*ba dum cha*). Rather, ask yourself if you think it is really so implausible to argue that depending on news coverage in a country as unstable as Iraq is unwise (news coverage even in a country as stable as the US is often unreliable). Then, ask yourself what the best method really would be to determine the true death rate. Also, observe, as argued in the report and as said in the parent to this comment, that the change in the rate of violent deaths is the most pertinent issue here, and even the Iraq Body Count trendline is similar to the one in this study.

Anyway, I really should get back to reading for my classes...

That's the problem

(#555)

A lot of people read the point estimate and think the study days 650k people were killed. And that's just not true. So they think the number is too high and therefore the whole study is useless.

A good overview can be found here.

This place is my vacation.

But the point estimate is the important issue

(#618)

Mr. Gabriel,

By far. This is a political maneuver and that point estimate is in there as grist for headlines.

Even if you want to take the thing at face value, there is nothing special about the methodology, that in itself is not significant or particularly valuable. Neither is any trend revealed in a worthwhile manner if their numbers are off. Their estimate is interesting because of their estimate, thats it.

What is missing in this survey is any guarantee of reliability in its execution. Anybody who has tried to poll in the third world (I have, and I have known others) can tell you that. Iraq is a third world situation magnified.

Iraq, in its present condition, is not a place where one can send people out to canvass for information reliably. You cannot trust your pollsters, you cannot trust the respondents, and both are vulnerable to pressure by political interests and are likely to be interested parties anyway. Some consistent bias in response in regions with a large incidence of mortality will throw the result way off.

This is not a failure of mathematics. Its a case of garbage in-garbage out.

When you have something like this it is essential to do a "reality check", as is often done in election polling.

Just as an example, counting death certificates is one of these, even in Iraq there is a working civil service that issues them. These are important for the issue of death benefits (yes they have these) for widows and orphans, pension payments, inheritance, etc. This sort of thing can be done unreliably, but the study results require that these be an order of magnitude off. Even in third world countries this is something that is done with fair reliability. If anything there is a tendency to inflate these, for the sake of the aforesaid death benefits, etc.

Iraq is a welfare state, most of the population depends on rations and handouts.

Other measures, such as the IBC, also should not be an order of magnitude off. The IBC and the death certificate count are not far out of sync, which should raise a red flag about the value of the Lancet paper.

And then there is the testimony of US troops in the field, mssion rates and ordnance usage rates, etc., that do not justify such a high estimate.

No

(#624)

The point estimate is not the important issue if you are looking at the scientific study. However people may use or misuse the numbers for political reasons is another debate.

There is nothing surprising about the discrepancies with the IBC and other methods. Those are passive methods and previous wars show they tend to be off by 5-10 times the actual number of dead. Even before the war the central authorities were only counting about 1/3 of those actually dying so it's perfectly logical, and consistent with other conflicts, that they would be even more off today.

When you write stuff like this:

Iraq, in its present condition, is not a place where one can send people out to canvass for information reliably. You cannot trust your pollsters, you cannot trust the respondents, and both are vulnerable to pressure by political interests and are likely to be interested parties anyway.

It's clear you are making stuff up. The US government and this administration have requested and quoted polls taken in Iraq. Yes, it's difficult and the Confidence intervals are likely to be higher, but it can be done and it's been done.

This place is my vacation.

Polls there are of little value

(#633)

Mr. Gabriel,

And I'm not making stuff up. I have actually done this stuff in the third world. The worst problem is getting a reasonably representative sample. The other problem is getting honest responses. The politics in this case makes this effort hopeless.

And this is worse than a typical poll, where one generally is looking for a proportional response from the population as a whole. In this case they were looking for a count of a relatively low-probability occurrence. It is far more sensitive to bias than an opinion poll. What would be a reasonably acceptable error in an opinion poll would be enormous one in this case.

And if you are calling me innumerate, then I call you out. Mano a mano pal, name your data.

I began my career

(#634)

as a pollster in the Third world. While polls in general are not perfect and in the third world they are more difficult than here (to some extent, since caller ID is making polls in the US difficult as well) the idea that polls are of little value makes no sense. If you wish to believe that, that's fine.

As for the data, I think our little debate on CA showed you are no statistician, to put it mildly.

Why don't you do as Aaron suggests, read the report and tell us what it is that all the experts that looked at it missed?

This place is my vacation.

Ah, the name changes, the assertion stays the same <nt>

(#642)

xxx

I'm trying to make the best out of a bad situation. I don't need to hear crap from a bunch of hippie freaks living in denial! Screw you guys, I'm going home!

I try and I try

(#644)

and they pull me back in!

This place is my vacation.

Personally, I'm waitng for my own bi-monthly tantrum <nt>

(#649)

xxx

I'm trying to make the best out of a bad situation. I don't need to hear crap from a bunch of hippie freaks living in denial! Screw you guys, I'm going home!

So are we nt

(#673)
HankP's picture

I blame it all on the Internet

I am not a statistician

(#645)

Mr. Gabriel/GT,

Not an academic anyway, but I use and analyze statistics professionally.

I am perfectly confident of my abilities here. I have produced models and analyses for use in public hearings on regulatory issues before the State of California.

And you won no points against me in the CA case, your characterization of that is absurd.

As for the study, it does not specify any worthwhile controls vis the issues I raised.

Oh no

(#653)

I won no points. You lost them yourself when you tried to use the wrong deflator to compare state income.

Luis, it's not that complicated.

1 The methodology used is widely accepted in the scientific community. It has been and is used by the US government and many other govts and international organization in EXACTLY the kind of situation you have in Iraq right now, ie a war zone. If you think this study is useless becasue of the nature of Iraq today you are sayuing this whole firled of study, which has been applied to almost all modern wars, is useless. Maybe tha is what you believe in which case there's nothing else to debate.

2 The experts that work in the field and that are statisticians looked at this study and found no methodological problems.

3 The resulting numbers are multiples of what the passive methods tell us (IBC and Iraqi govt) but this is perfectly consistent both with other wars and with prewar Iraq.

This place is my vacation.

Thanks. some good points. Onr thing of note, both this study

(#545)

and its predecessor were released in October, just before US elections -- and both before they were actually published in "Lancet."

Not that it's political or anything...

These guys LINK, no friends of the administration or the war also are skeptical.

Your points on the coalition disassociating itself from the sectarian violence are sensible but difficult to put into practice; that would be seen as "abandonment" by those who wish to make an issue of it. This is a war of media coverage as much or more as it is a shooting war. We are not nimble enough to compete well in that effort...

We will be blamed no matter what and the, IMO, very flawed Study will be used as it suits...

Whatever the motives

(#546)

in releasing the study before an election (yes, I agree it's political) it has nothing to do with the validity of the study. If they lied or the calculations are wrong or the methodology is wrong all those are valid complaints. But pointing out the political impact of the study isn't.

This place is my vacation.

Didn't say that it did have anything to do with the

(#558)

validity. Merely pointing out that the authors and Richard Horton have an agenda.

As to validity, while I think the study is quite wrong (it doesn't pass the common sense test), I'm not a statistician and think the study is too irrelevant to much of anything aside from politics -- and I include the rather futile politics of "concern" -- to pay too much attention to.

I'm unsure how one can say that pointing out the political aspects of the release of a study that even you acknowledge was political can be invalid but you're certainly entitled to your opinion...

Well that's the problem

(#559)

this isn't a issue that common sense can decide. We need statistics. Plus, what's a common sense test of how many people are killed in a war zone? Recent wars show that official counts can undercount by up to 10 times compared to what the post-war surveys show. So how could we even apply common sense to something like this?

Whatever you may think of the political views of the authors it makes no difference regarding the study itself, unless you think they lied. The math and methodology are perfectly sound and pratically all experts that have looked at it confirm this. The numbers are off from what the other poassive counts indicate but that is common in situations like this.

This place is my vacation.

You're asking for accuracy you're unlikely to get.

(#567)

Since no war ever passes any common sense test.

There is no way to answer how many people are killed in a war zone. You can't even count the bodies -- some are so shredded you don't know if you have one person or five. Some are buried under rubble and never found. Some who were believed dead appear later, some believed dead left the country. Too many variables. Not to mention that no one involved in the shooting wants to waste time counting the dead; they're too busy trying to stay alive themselves.

Do recent wars show that or do recent revisionist histories show that?

All the experts I've seen that confirm any validity of the study are fellow epidemiologists (not math gurus or stats guys) and Amnesty International -- not unbiased sources. Early days, yet, let the Stats guys study it and I suspect it will be debunked as thoroughly as the 2004 study was.

Ken

(#569)

The 2004 study was never debunked. I mean, if you believe that there's nothing to debate here. You've simply made up stuff.

It wasn't just 'fellow epidemiologists' who think this study was well done. Not only did they consult several statisticians and not only did other statisticians perform a peer-review but newspaper reports show practiclaly all the experts (including statisticians) who looked at the repiort found no problems with the methodology.

This is standard methodology used by the US government in its own data gathering and also used in many other conflicts, including Sudan and Bosnia. There is nothing starnge about this, as Aaron points out.

This place is my vacation.

Debunking is in the eye of the beholdr. We can stipulate

(#602)

-- and I believe -- that the methodology was as sound as possible, sure. The issue is can a statistical survey such as this overcome tow very large elephants in the room; the first -- do the respondents have full knowledge and are they answering totally honestly and objectively with no political effort. My suggestion is that if you believe that is possible, I urge you to visit the Middle East.

The second issue is summed up here:

This "peer reviewed study" is a piece of polemical garbage. Everybody is supposed to take away the bumper sticker summary, "Coalition kills 100,000 Iraqi civilians, half of them children," without reading the details. It tries to use crude epidemiological models like those used to study disease and applies them to the conscious infliction of violence by human beings. The result is statistical static.
(emphasis added / kw) (LINK)

I have watched unduly numerate folks foolishly attempt to apply 'metrics' to war for over sixty years. Totally, absolutely unsuccessfully, I might add...

Then there's this (LINK) which refers to this (LINK). I ordinarily refrain from quoting him due to his leftish bias but given that and what he has to say; I'd say, yeah, debunked is a good word...

As I said, no math or stats guy but I'm reasonably intelligent and have been to war or two and I know this is unlikely to be remotely accurate. As I also said, it doesn't pass the common sense test.

Polls and surveys have a place in life. They are, however, not capable of and will not ever provide definitive and final answers to much of anything. They are subject to many human foibles and should be taken with a trainload of salt.

I know Aaron is a big fan of surveys and polls; he knows I am not, now you also know that... :)

Not a "big fan" per se

(#607)

I just think that, used properly, it can be a very effective tool for explaining a variety of phenomena, social and otherwise.

The "elephants" you refer to (and there are in fact more than the ones you mentioned) are what statisticians would label as "sampling error", "data skewness", "selection bias", and other statistical artifacts. They are undoubtedly significant issues and any decent statistical study will do its utmost to acknowledge and mediate them. If you actually read this report, you will see that they do just this: I don't think they did it perfectly (hence my own criticism), but I think they did a good enough job that their results should not be dismissed as handily as you and others wish to.

The problem with surveys in the media is that they always present the point result (the specific 46.52% or whatnot) and then simply what they call the "Margin of Error", which in reality is a 95% confidence interval. That is to say, if a poll is presented as having a result of 42% with +- 4% margin of error, that *really* means that, based on the data, it can be claimed with 95% confidence that the *true* result lies somewhere between 38% and 46%. *However*, this is only considering the *statistical* (or "sampling") error: that is, the error we get by looking at some sample size n (often ~1000 observations) rather than the full population (say, 300 million people in America). There are many other sources of error, as you yourself have intuited, and a properly done survey (as one would hope most academic ones are) acknowledges and addresses them as well. Unfortunately, the media presentation of it then again boils it down to just the point estimate and the margin of error, so you really have to go back to the source to fully understand what is going on (hence my link to the article itself).

So I guess this is a long way of saying that your instincts are good, but you are basing your own opinion on incomplete data. Try diving in and reading the results: I honestly believe that it's actually pretty accessible, you don't need to be a statistician to see what they're doing. And again, if you still feel this approach is flawed, then try and come up with something better. That is the true mission of any intellectual pursuit, academic or otherwise, and who knows? Maybe we can work out an approach better than systematic surveying to measure the death rate in Iraq. Granted, a lot of smart people have spent a lot of time thinking about these sorts of problems so it's somewhat unlikely that we'll have a dramatic breakthrough, but at least we can strive to create some positive ideas, rather than merely critical ones.

These are more than "significant" issues

(#623)

Mr. Aaron,

They are the entire study.

What we have here is a case where a very small number of "false positives" can throw the whole study off by an order of magnitude. The high likelihood of that occurring "in the field" in this case, in spite of attempts to control it (there really is no good control available under these circumstances), makes the whole study questionable.

As for the truth, it will not be possible to come up with that until the whole thing has settled down and a proper census can be arranged. Iraq is basically a welfare state, like all oil states, and the attraction of government benefits makes compliance more certain.

A brief suggestion

(#629)

Just read the report: you are responding to me throughout this thread and essentially saying the same thing in all of your comments. That is, you believe that a valid survey is simply impossible given the current conditions in Iraq.

Well, read the report. See how they actually approached the problem. Don't base your opinion on your own preconceptions and on the incomplete and biased (in both ways) coverage of both the report and the situation in Iraq. Read how they tried to solve this problem, what their approach was, how they conceived of it, and then try to provide more *precise* criticisms if you want your rebuttal to have validity. And most of all, try to provide suggestions for improvement.

Unless we're talking about Schrodinger's cat, I for one do not believe that there are such strict limits on the range of possible human knowledge. I believe that we can, and in fact must, study human interaction and behavior. Yes, it is inherently uncertain and difficult to understand, but that does not make it impossible: on the contrary, it makes it all the more critical that we try.

I did read it

(#640)

Mr. Aaron,

It is quite simple. I recommend you read it through again.

The study does not specify any controls of reporting biases on the part of the data collectors, or of respondents.

Read further on the report and note the need to deal with local rebels, militias and gangs. Yes, the supply of politically sensitive data under these conditions is not likely to be reliable - and the study did mention that the teams were instructed to explain this sensitivity.

As for the "range of human knowledge" comment, can you not see how disingenuous this is ? One does not have to deny the value of all social science research in order to point out the uselessness of a specific attempt.

And then people wonder

(#641)

why scientists tend to vote for Democrats.

This place is my vacation.

maybe ...

(#656)

... you should re-read IBC's statement on the paper, and then moderate your comments better:

We would hope that, before accepting such extreme notions, serious consideration is given to the possibility that the population estimates derived from the Lancet study are flawed. The most likely source of such a flaw is some bias in the sampling methodology such that violent deaths were vastly over-represented in the sample. The precise potential nature of such bias is not clear at this point (it could, for example, involve problems in the application of a statistical method originally designed for studying the spread of disease in a population to direct and ongoing violence-related phenomena). But to dismiss the possibility of such bias out of hand is surely both irresponsible and unwise.

Nobody says

(#664)

you should dismiss it out of hand. But this technique has been used worldwide and is considered the standard in cases like this. So if you are going to claim that the study is flawed you need something to back it up.

This place is my vacation.

Well ...

(#679)

... the way I read it, IBC has quite a bit of evidence to back up the claim that the study is flawed.

That's what their reality checks *are*. Evidence.

This is nonsense

(#657)

Mr. Gabriel,

And you know it. This has nothing to do with science. You are taking a deplorable path in this argument.

I am not

(#660)

Luis,

You keep bringing up arguments that have no weight, have been already addressed or are meaningless. You admit you are not an expert on this topic yet you brush off all the real experts that say the methodology is sound. You say, with nothing to back it up other than your opinion, that polls in Iraq are useless, despite the fact that many polls have been taken there and in many other similar war zones.

It's hard to take this seriously. Clearly you don't want to face the facts here. And this attitude is why so many scientists have given up on the GOP. It's the same attitude that leads to denying evolution or global warming studies. You've decided you know more than the experts but offer nothing to back that up. Simply saying that it's not possible to do a survey in Iraq is not an argument.

This place is my vacation.

Oh for heavens sake

(#675)

Mr. Gabriel,

No weight ? My objections are the sort of thing that undermine the entire premise of their approach. And these have nothing to do with science or methodology. There is no complaint about the mathematics of cluster sampling here. The failure I believe is in the plain, non-scientific matter of political bias in collecting the data.

You refuse to acknowledge that the objections to this have nothing to do with mathematics or science. Rather, you are using science disingenuously as a rhetorical shield.

No luis

(#681)

If you are OK with the math and the sampling what is your problem then? That it's a war zone and that leads to measurement problems? But as I pointed out many times this has been done in many other war zones so why would it not be useful here? Other polls have been taken in Iraq, why would this one be useless and the rest OK?

Or are you saying all such polls are useless?

This place is my vacation.

I am not OK with the execution

(#703)

Mr. Gabriel,

Yes it is a war zone and it leads to measurement problems. It also leads to systematic measurement problems, because this is a propaganda war.

All polls taken in Iraq have to be taken with a high uncertainty, as I have pointed out before. In this case the significance of such uncertainties is magnified greatly, because it is not an opinion poll. A sampling error that would swing a poll 10% could easily swing this one 500%.

Huh?

(#706)

1 Where did you get the last sentence from? What math backs that up?

2 Also, are you saying you think just this study is useless, all polls in Iraq are useless or all such studies in other war zones are useless?

This place is my vacation.

My math

(#723)

Mr Gabriel,

A thought experiment -

Lets assume a poll of whether Iraqis think the US forces should leave Iraq. If you oversample an area, say 1/5 of your overall population, by 100% (your actual sample is 2/5 of this group vs 1/5), where response to something is 100% vs 50% elsewhere, your erroneous result will be 70% where your correct result should be 60%. The result will be off by 16.6%.

When measuring incidence under the similar conditions - if this 1/5 of the population has an average mortality due to violence of 20/1000 (the study's implication) and the remainder has a violence-mortality of 1/1000 (African average, just FWIW), a 100% oversampling of that 1/5 will effectively double the overall rate, almost 100% off.

Sampling errors are much more significant when you are trying to measure incidence rates.

How does that relate to reality in Iraq?

(#887)

What evidence do you have there was such overcounting?

You simply make stuff up and want to use that as proof against the study?

This place is my vacation.

My specific point

(#1889)

Mr. Gabriel,

Is to point out that an incidence survey like this is much more vulnerable to error or bias (deliberate or not, and by whatever means) than a poll.

My distrust is based on other grounds, namely the many objections that have been raised by others, such as a lack of destruction "on the ground", lack of corresponding casualty reports, lack of corroboration by counting methodologies, lack of correspondence with official records, and of course the notoriety of the insurgents in penetrating the news media and other such bodies.

I assume this statement was an aberration

(#666)

you now wish you could take back. If you think it true and meaningful, your credibility just took a nosedive in my eyes.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Well

(#678)

Depends on what you mean. Yes, it's a bit over the top but it points at something very real. For some reason too many on the right are simply hostile to data or science that contradicts their world view. And yes, I know many on the left feel the same way but it appears it is much more relevant on the right. There is no equivalent on the left of the anti-evolution non-science so prevalent in sectors of the right.

What I find in this Lancet debate is a lot of people saying it must be wrong but unable to show any methodological reasons why. Hence my, yes exaggerated, reaction.

This place is my vacation.

There is ...

(#683)

... a similar reaction on the left to economics. But lets not get off topic.

Yes

(#688)

we can debate that another day. But on economics both sides have that.

This place is my vacation.

It's a bipartisan fault.

(#705)

The fact that nothing trumps science is so obvious it should never even have to be stated. But I see the same thing in another guise with, e.g., enviromentalists that blindly accept everything their prophets tell them about the case for (but never against) anthropogenic global warming without having the faintest clue of the uncertainties of the underlying science, statistics and predictive models.

Maybe many people on both sides of the aisle have difficulty when there is an attempt to reduce to a statistical certainty something they feel very passionate about.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Nonsense.

(#1009)
Bernard Guerrero's picture

I consider most progressives to be innumerate idiots, and I'll happily put my technical chops up against anyone.

Not really

(#1013)

I didn’t say all Republicans are dumb or that all scientists vote for Democrats.

But can you point out anything on the left equivalent to the anti-evolution screeds of the right?

For some reason the GOP has become the party of anti-science.

This place is my vacation.

OK

(#1016)

So we can attribute any kook fringe thing that a tiny fraction of democrats believe and declare: "The Democrats have become the party of [insert dumb idea]"

How Coulteresque!

.

“I serve as a blank screen on which people of vastly different political stripes project their own views.”

Tiny fraction of Republicans

(#1019)

If only ...

"Something I think most liberals don't understand is exactly how stupid many conservative leaders are." - Matt Yglesias

Ha!

(#1023)

A new piece o flypaper!

“I serve as a blank screen on which people of vastly different political stripes project their own views.”

Do you really

(#1024)

believe that the anti-evolution groups are a tiny fraction of the GOP? I mean Bush himself supports them.

This place is my vacation.

I really

(#1034)

believe you don't actually know who is "anti-evolution" or how many there are out there who are in fact "anti" evolution. Or for that matter how they vote.

“I serve as a blank screen on which people of vastly different political stripes project their own views.”

I don't think you should generalize to all science...

(#1070)

It is certainly true that the Republicans attract the anti-evolution types, which is definitely an anti-science perspective (in my view). You can also make a decent case for the same thing on stem cells and a reasonable case on global warming, though that is less an "anti-science" issue and more of an "anti-painful-economic-price" issue.

But you can't really extend this to scientific issues in general. The left has it's own anti-science stances on biotechnology, nuclear energy, and animal rights issues. These are crossover anti-technology issues, but it definitely bleeds into a suspicion of all the science behind the technologies in question. The fear-mongering and conspiratorial thinking on genetic engineering is just as intense as the line of hardest of hardcore creationists. (i.e., the "establishment" has conspired to hide the true facts, the employment of fringe biologists to support thier poisition, etc.)

One big difference

(#1178)
HankP's picture

is that I don't hear anyone on "the left" calling for the end of research or teaching of certain subjects. Implementation, yes, but not research.

I blame it all on the Internet

But I hear many on the Left

(#1574)

demanding that no one dare to question their sacred cows like AGW. Here's an example, and from the Boston Globe of all places.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Define "many"

(#1672)
HankP's picture

I hear the occasional story, but nothing within orders of magnitude of intelligent design. Certainly nothing that any Board of Education has voted on.

I blame it all on the Internet

Let us assume, for the sake of argument...

(#726)

...that you are correct. That necessitates assuming that almost everyone who participated in this study, both those who carried it out and those who responded, were systematically misrepresenting the results. Why? Because it can't be definitively proven otherwise (a sort of "Cartesian certainty" response to a survey which is a quite inappropriate litmus test, but if that's what you want let's see where it takes us).

In that case, the survey may not reflect the actual situation in Iraq, but it still reflects the *perceived* amount of suffering that the Iraqis have at the hands of the coalition. And if, as you say, this perception is so much larger than reality, I would say that the results are all the more relevant and the policy prescriptions need to be all the more dramatic, as no matter how optimistic you are you have to acknowledge the current trend line is towards increasing violence and that means that there will be even exponentially more blame directed towards the US, which could easily result in more terrorist activity. In any case, the results definitely verify that this whole "winning hearts and minds" campaign has been quite a bust.

So really, if what you say is true, then the situation is in a sense all the more dire. In any case, you cannot simply dismiss these results because you don't like them: the assumptions required to argue that they are not reflective of reality mean that they are just reflective of perceptions, and that's arguably just as bad.

Hardly

(#736)

Mr. Aaron,

What is almost certain (though not specified, one can reliably assume it) about the study is that the Iraqi participants are politicized, and odds are that persons of particular sectarian/political alignment were selected to conduct the surveys in specific areas. Otherwise they would be running extreme risks.

It is no accident that the highest mortality rates reported are, apparently, among Sunni Iraqis. Most Iraqi academics are Sunni.

What does this tell you about the overall situation in Iraq ? That a minority is profoundly aggrieved ? That there are segments of the population enthusiastically waging war ? This is not news. There is a litany of cases of the same sorts of persons fabricating news stories, conducting sabotage (such as the recent mass poisoning of policemen) and undermining the Iraqi government. Total war is total war, and respect for science cannot compete with that imperative.

That they would skew data to support their political opinions is a minor act in comparison to what so many of their bretheren are willing to do.

You think this is "total war"?

(#841)

Well, I'll deal with that misconception another day.

As for the rest of it, you're still not giving a substantial response. As I said to Livia, if you really believe there is such an overwhelming systematic bias you need to provide a better argument to support such a claim. I admit that the Sunni-academia anecdote you told in this comment is perhaps a start, but you're a long way from really making your case (hint: evidence would help). And while I'm guessing you may have a bit of an anti-academia bias yourself, I'd be willing to bet that the true academics in Iraq are able to detach themselves at least partially from their own political opinions, at least as far as their data analysis is concerned. You'd be surprised at the intellectual honesty a good academic is capable of. Granted, there are bad academics too, but that is the case in any walk of life. In any case, I find it hard to believe your story of incredible pernicious and systemic bias.

This is more than a "profoundly aggrieved minority": if nothing else, you cannot deny that their sample is representative enough to show that Iraqis in general see themselves as suffering quite substantially at the hands of America. And while you may not think that is news, it certainly would be if Bush and co. actually acknowledged it.

Whats the misconception ?

(#1862)

Mr. Aaron,

As for total war -

The other side is waging war without restraint and by all means available, their only constraint being the lack of resources. They are waging war directly against the other sides civilians, religious institutions, economy, civilian infrastructure, finances, you name it. That seems like it fits the definition. On another tack, you need to drop the academic insistence on terminology.

As for a systematic bias, it is my impression based on the years of reporting from Iraq. Sunni "stringers" and "handlers" used by the international news media have been caught numerous times cooperating with terrorists and insurgents in reporting the news and planting stories. The insurgency has often shown its media sophistication. It is my judgement of course, but I don't see how they would miss this opportunity to skew a study to their benefit.

The study itself mentions the need to win the acceptance of these insurgents, who are perfectly capable of realizing the value of "input" in such.

What is the significance of the outcome of a study manipulated by partisans in a war ? What is there to acknowledge ? That the other side wants to win ? That some (or many) Iraqis feel they are suffering ? This is news ?

Doesn't necessarily have to be political in nature ...

(#747)

Let's say you're a poor family, part of a tribe living in some rural Shiite area. Several relatives, close and distant have been killed. A son, a brother, some cousins, a daughter's brother in law, etc. You hear of killings occuring almost daily, and the rest of the world seems not to care. The TV stations talk about fighting the occupier, and turn a blind eye to the kidnappings and suicides attacks that you have to deal with.

One day, some professional looking people wearing nice clean suits show up with clipboards, representatives of a distant world of civilization, peace, security and science. They're there to ask you about people in your family who have died. They say they want to produce an accurate count of the dead.

Now, technically, the only one that you're legally allowed to count is the son, since he's the only guy that actually was living in the house. But to you, the deaths of the brother, the cousin, are just as important. The daughter has moved back home and is still in mourning, and she doesn't think her husbands death is unimportant either.
Maybe you don't even under stand the point of only listing the son's death. You don't understand statistics, you don't see how anyone can count accurately without listing every death.

How are you going to respond to that? Are you going to sit there and abide strictly by the rules and only have them list the son's death? Or are you going to take your opportunity, a rare opportunity for someone to hear about your suffering, and try to mention as many deaths in your family as you can, regardless of whatever silly rules the scientists have?

Nice story...

(#839)

...made my point for me, really. That, and you missed the fact that a large part of the debate is about whether or not the deaths should be attributed to the coalition (the report attributes a surprisingly large proportion of them to the coalition, and even acknowledges to a degree the dubiousness of this).

In any case, the issue is an inherently political one, that is why we are discussing it here. And as far as your anecdote, I found it doubtful that such systematic misrepresentation is truly occurring simply because you wish it to so that the results line up more with your impression of reality. But as I said, even if that is the case, then the survey is still a measure of perceptions: nowhere did I say "political perceptions", just perceptions. And yes, perceptions matter.

No ...

(#844)

... it's not because I wish it to line up with my perception of reality. Even Iraq Body Count disputes the numbers and they do not have the same political viewpoint as I.

It is a possible explanation for the huge discrepancy between the numbers available from other sources and the study.

Such a massive discrepancy just can't be waved away. There must be *some* kind of explanation, and the burden of proof should be on the authors to provide one. I find it far more plausible that a large degree of overreporting occured in the surveyed samples, than that the other sources have missed the number of deaths that is suggested.

I am sure that many deaths *have* been missed, but 600,000 is implausible. Realistically, the most likely explanation is overreporting. Maybe not in the way I've postulated, but there is certainly something in there inflating the numbers.

Nobody is waving it away

(#846)

They provide a number of possible explanations, and they do a good deal more to actually support their arguments than you are. Read their appendices and you'll see that.

If you believe the numbers are inflated, more power to you. I wouldn't be surprised if they're off by at least a bit, but I'd be willing to bet they're in the generally right magnitude of order (which does still place them substantially higher than existing counts, which almost undeniably suffer from severe undercounting due to their passive nature). Additionally, they are arguably at least an accurate reflection of perceptions, if not reality. And for those reasons alone this study is worth considering, as I have argued. Your nitpicking and rhetoric doesn't change that, and unless you can provide a substantial argument to truly support this assertion of overwhelming overreporting, the study is still of political and practical import.

yes ...

(#849)

... and the IBC response addresses those explanations, and has good arguments for why they are unlikely.

So why shouldn't we scrutinize the methodology of the Lancet study more, consider the possibility that there might be a flaw, and ask for more detail to rule out the possibility?

My abbreviated response to the oft-cited IBC arguments

(#853)

(Taken from elsewhere in this thread, but most folks seem to have missed it).

The page on Iraq Body Count that folks are linking to as a critique of this study lists the following four implications if the study is valid:

-----------
* incompetence and/or fraud on a truly massive scale by Iraqi officials in hospitals and ministries, on a local, regional and national level, perfectly coordinated from the moment the occupation began;
* bizarre and self-destructive behaviour on the part of all but a small minority of 800,000 injured, mostly non-combatant, Iraqis;
* the utter failure of local or external agencies to notice and respond to a decimation of the adult male population in key urban areas;
* an abject failure of the media, Iraqi as well as international, to observe that Coalition-caused events of the scale they reported during the three-week invasion in 2003 have been occurring every month for over a year.
-----------

It then labels these as "improbable", which is dismissive to say the least. Certainly widespread failure of the Iraqi infrastructure is not only probable but expected these past few years, and is in fact explicitly discussed in the report itself. IBC tries to present it in as unfavorable a light as possible, but even their own points read to me as not being so improbable but rather quite possibly reflective of reality (the first point being the result of a destroyed infrastructure and economy and the ensuing corruption, the second being a result of sectarian violence [which I would certainly describe as "bizarre and self-destructive behavior"], the third being a mix of the previous two and the fourth just being the media doing its job as usual).

And ...

(#926)

... the Lancet study is dismissive of the official counts and IBC. All they do is say "underreporting" and don't bother to make much of an argument for how such a massive amount of underreporting could occur.

There are certainly failures of the Iraqi infrastructure, but on such a massive scale? Without being noticed?

Is there any hard data on this point? IBC catches them in an error on the amount of underreporting by the official statistics prior to the war. Even if that increased after the war, a factor of ten is still large enough to be in want of an explanation. You can't just wave it away by saying underreporting. Claiming massive infrastructure failure is begging the question.

Ummm, then you didn't read it

(#942)

They do make an argument, and a more convincing one than most all of the critique being offered here. Simply put, IBC and "official" counts (still the rhetorical game eh?) depend on passive observational counts, which are overwhelmingly focused in Baghdad and miss a lot of what is happening in the rest of the country. It would be as if you decided to hold an election in the United States but only let people from the largest metropolises vote - you wouldn't get an accurate picture of the political opinion or makeup of the country as a whole.

And yes, there are failures in the Iraqi infrastructure, and I'd say it's quite plausible to label them as pretty "massive." It was essentially destroyed and attempted to be rebuilt from scratch. As far as the "not being noticed", well, some have noticed, and some are trying to bring attention to it (like in this report), but many (e.g. folks who hold opinions comparable to you) are dismissive and overly reliant on incomplete coverage from the media.

"IBC catches them in an error on the amount of underreporting by the official statistics prior to the war." - Explain that one please. Saying they "catch them in an error" doesn't make it so.

You're right, a factor of ten is in want of an explanation. And they offer many, the underreporting issue is just one of them. And I'm afraid that the massive infrastructure failure is not begging the question - it is a simple deductive argument based on empirical evidence and logical reasoning to support the premise that the Iraqi infrastructure has been (and still is) in dire straits and that this is a plausible explanation for both a large number of deaths and a general lack of coverage/reporting of them.

Infrastructure ...

(#995)

... is a pretty broad term. We're not talking about electricity. We're talking about the ability of offcials to count the dead.

By the study's own statement, death certificates were obtained in 92% of the cases where they were asked. If there were massive infrastructure failure of the kind required to miss 90% of the dead, then they wouldn't be providing 92% of the people reporting deaths with death certificates. Where are all these death certificates coming from if the infrastructure for counting deaths has collapsed?

You have to postulate a lot of maybes to explain how this can happen. By contrast, it's much easier to get a massive error by extrapolating if there's overreporting in your samples. Occam's Razor.

been though this before

(#998)

Even before the mess of the war up to 2/3rds of the death certificates issued were not counted by the national authorities.

This place is my vacation.

That's ...

(#1007)

... where IBC discovered the error.

Actually about 70% of the death certificates were counted. The Lancet study just got a wrong number about the number of deaths reported in Iraq in 2002. It wasn't 40,000, it was 80,000, and their extrapolation of the true number of deaths (120,000) is open to debate anyway.

So we're talking about going from a 50% undercount (maybe) to a 1000% undercount pre and post war.

Opinions can vary on their utility.

(#636)

See my comment above for a generic summary of polling. I'll respond to specifics in this one.

I'm well aware of the those and many elephants. I did skim this study but am not prepared to waste a whole lot of time on it as it is essentially immaterial. Its conclusions will change nothing on the ground and it is patently released as a political effort destroying any possible use for policy making now or in the future. You may not agree with dismissing the results, your prerogative.

The methodology of this study is, as you say, easily understood, that is not an issue. The major issue IMO is attempting through a survey and statistical sampling to produce a set of numbers that one has to acknowledge up front will be flawed -- the question thus becomes; why do it? What is to be learned or gained? Why, for example is the death rate in Iraq of Academic interest at this time?

The Historians wil probaly get better figure but attempting to get them during a conflict introduces all sorts of questions, not least that of personal agendas of respondents (and surveyors), reasons for selection of this or that Province, district, City or area and a host of other things.

The problem with surveys in the media is that the media, broadly, is pretty clueless. They present things that satisfy their agenda and suppress other things. Way of the world.

Thank you for ackonwledging that my instincts are good, I became ware of that well over half a century ago but it's always nice to add one more to the many confirmations. No, I am not basing my own conclusions on incomplete data. I am basing my conclusions on the certain knowledge that wars do damage and that the ability to assess the amount of damage is almost nonexistent during the war due to several factors and on the equally certain knowledge that I have watched many try to reduce war to a numbers game over the years and that has invariably been unsuccessful. It's mean, brutal and ugly and it's not neat, not at all...

Why is it of interest?

(#842)

Because if it was actually acknowledged and proper policy prescriptions were taken, we might actually be able to eventually help bring true progress to Iraq.

In any case, you reveal your own prejudice for not wanting to "waste a whole lot of time" on this study (and it clocks in at what, 25 pages? shouldn't take more than a half hour to read in decent detail, probably less time than you've spent in this thread alone already). As others have argued, even if you believe the motivation is suspect it does nothing to invalidate the results (to think so is an ad hominem fallacy).

I don't really follow your last paragraph, but I can say that whatever you mean by your "certain knowledge" (which doesn't seem as indubitable as you present it, but regardless), it doesn't seem to actually logically lead to your conclusion. That is, your premises and your conclusions are not connected. The sorts of data you really need are a proper reading of this report, as well as a more realistic grounding in the standards of knowledge in the social sciences (or put more shortly, a statistics course). Lacking those things you don't necessarily need to reserve judgment entirely, but it is prejudicial for you to simply dismiss the study as a "waste of time" and present your own conclusions as if they are based in "certain knowledge."

You didn't answer your own question

(#870)

or mine. Again, Why is it of interest?

Do you have any evidence that if it were acknowledged any change in policy would occur? If such a change did occur would it in fact bring true progress to Iraq?

Both assumptions are highly speculative and you fail to show how this study might affect any policy.

I read the entire study, noting such things as the three dropped clusters, the math anomaly of 50 x 40 = 200 but equating to 1,849...

I read it yesterday after I skimmed it to see if you had a point. I re-read it last night. I still disagree on the validity of any sampling process that attempts to provide 'answers' to war related questions while that war is ongoing. I read it thus twice and neither time did it take 30 minutes -- you must be a slow reader. I remain unconvinced -- totally unconvinced.

You reveal your own prejudice by defending the methodology -- and, effectively, the results -- of the study without proffering any reasonable rationale for the conduct of the study other than political; thus your accusation of and ad-hom on my part is awfully hollow and would thus seem to apply equally to you.

Certain knowledge based on experience and stated as a generality -- to wit, "...wars do damage and that the ability to assess the amount of damage is almost nonexistent during the war due to several factors..." may not seem indubitable to you but I suggest that the first part is indeed indubitable and the second logically follows that first. I further suggest the Study authors acknowledge that problem.

Further, "...and on the equally certain knowledge that I have watched many try to reduce war to a numbers game over the years and that has invariably been unsuccessful..." That is an opinion; thus it can not be stated as an indubitable fact -- but it is an opinion based on long observation and you may refute it by presenting any evidence you wish...

"...It's mean, brutal and ugly and it's not neat, not at all..." That also is an opinion; you may not share it; many do.

My premise and my conclusions are connected. The premise is that war is exceedingly complex, humans are exceedingly complex and that attempts to reduce either of the two to a numerate display is fraught with difficulty. To attempt to do that in the midst of a war compounds the difficulty and is bound to produce results that are suspect. This study proves my premise.

It is perhaps prejudicial of me to dismiss the study as a waste of time; though I sincerely believe it is that. It is not at all prejudicial of me to present my conclusions as based on certain knowledge because they are. In the absence of certain knowledge, I'm prepared to default to the books. Given some knowledge acquired and reinforced over many years, I'm unlikely to defer to books which were, after all, written by people. All people, thee, me and the authors of said books have biases and prejudices. If one has knowledge of an event or type of activity, one can filter for bias -- lacking that one has to rely on the books with concomitant possibility of error.

I repeat; Why is it of interest?

You are in some company

(#893)

You, George Bush, James 'the Big-Oil & OPEC fixer' Baker, perennial MIC flack Anthony Cordesman, andGen. "Alzheimers" Casey are out there all on message and sharing talking points in 'crisis management' mode to discredit the study.

Of course that 'highly enlightened' bunch of warmongers and their frontmen, are all much smarter than the academic/scientific demographers from Johns Hopkins Bloomberg School of Public Health, the Lancet [in the very top rank of rigorous and respected medical/scientific journals world-wide] editors and peer-reviewers, and MIT--the sponsoring entity-who would have not prominently associated itself with the papers findings if they smelled something seriously flawed with the methods, results or conclusions.

These Bushies are responsible for, and are parties to to the war-dependent killing fields of Bush's Iraq war, so naturally they would downplay its magnitude, and try to "bury the bodies" as quietly and secretively as possible. Right next to Saddam's infamous "400,000 genocided dead" (widely touted by Bushie warmongers in run-up without hard proof according to a quite extensive post invasion 19 month 2003-2004 search-survey conducted inside Iraq carried out and reported by HRW).

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Said the guy

(#917)

http://www.news.com.au/dailytelegraph/story/0,22049,20588297-5006003,00.html

busily defending North Korea just last week.

Will crush dissent for food

Obviously ...

(#921)

... my killing fields comment must have struck home.

Coming from Mister 'let them (NK) eat straw'

(#956)

That is some mischaracterizartion of what I wrote.

And if you hated evil former US ally Saddam, you will really hate the 'New and Improved' Baker-Bush Plan for Iraq, to be launched officially right after election (shades of Rape of Fallujah 11-04) which involves Strongman dictator(s) ruling Junta probably packed with the usual goonish Allawi/Chalabi/Hakim et al US puppet-suspects, who are ordered to begin brutally strangling Iraq democratic govmt in the baby crib.

How Saddam-like of US

In August 17, The New York Times carried an anonymous quote suggesting that the Bush administration had all but given up on democracy in Iraq, and was casting about for "alternatives."

Baker's Panel Rules Out democracy

'Iraqis' call for five-man junta

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Read your own links ...

(#1010)

... obviously a compulsive skimmer of headlines only.

Other Iraqis dismissed the idea that a unilateral change in the leadership would be desirable or even possible. “The only person who can undertake a coup in Iraq now is General George Casey (the US commander) and I don’t think the Americans are inclined to go in that direction,” said Ahmed Chalabi, head of a rival political party.

Okay. Scratch whitecollar conman Chalabi as the New Saddam

(#1146)

[He looks awful with a moustache anyway]

That still leaves former Mukhbarat goon/CIA operative Allawi, Hakim and his Baddass Badr Boys, and probably one or two less well known Sunnis and Kurdish compliant strongmen-in-waiting to round out the Baker-Bush New Junta Plan for Iraq.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Missing the point ...

(#1226)

... our supposed former puppet-dictator-to-be doesn't even think that we want to go in that direction. Ergo, all this talk about a dictatorship being imposed is largely paranoid fantasy. Just like Chalabi being installed as dictator was a paranoid fantasy in the first place.

Please read my links

(#1292)

before replying. Then you may comprehend my points a bit more. Then again, maybe not.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Someone want to explain

(#1341)

how PA is any less offensive or disruptive than hunter was?

Will crush dissent for food

No, No One Wants to Explain

(#1342)

Is this your first Posting Rules bleat at the new site? Heck. It might be the first from any commenter. Well done!

(Dang. There goes the consecutive-days-without-snark streak. It was bound to end!)

“Two clichés make us laugh but a hundred clichés move us, because we sense dimly that the clichés are talking among themselves, celebrating a reunion." - Umberto Eco

Just modify your criteria

(#1359)

If you make it consecutive-days-without-meaningful-snark streak you're still going strong. Around 4+ years that I know of.

Will crush dissent for food

Because we conservatives are smart enough to ignore

(#1350)

him. Ken, who like is like an old Tomcat playing with a mouse takes another approach.

The lefties couldn't do the same with Hunter's bait. They allowed him to disrupt. Ignore the squash, its easy.

I'm trying to make the best out of a bad situation. I don't need to hear crap from a bunch of hippie freaks living in denial! Screw you guys, I'm going home!

Easy on the Ad Hom

(#1344)

No questioning mental (or comprehension) abilities of fellow commenters. Even me. Thanks!

“Two clichés make us laugh but a hundred clichés move us, because we sense dimly that the clichés are talking among themselves, celebrating a reunion." - Umberto Eco

I Meant it thusly:

(#1436)

Even if respondent had read [and of course comprehended] the links, in all 'maybe-hood', it would not have altered the tone or thrust of respondent's retort.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

More hints of the 'Put Saddam's Moustache on new dictator' plan

(#1746)

In today's Washington Times:

Another scenario is being discussed -- and taken seriously in Iraq -- by many of Iraq's leading political players, under which the U.S.-trained army would overthrow struggling Prime Minister Nouri al-Maliki and replace him with a strongman who would restore order while Washington looks the other way.

Falah Hassan al-Naqib, a Sunni politician who served as minister of the interior in the interim government led by Iyad Allawi until last year, told The Washington Times he has met repeatedly with American and Iraqi generals to discuss alternative courses of action.

"All of them have a 'Plan B,' because if the situation continues as it is, they will have to defend themselves -- not just find bodies all over," Mr. al-Naqib said this summer at his house in Baghdad.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

What other kind of 'story' would you expect from Murdoch rags?

(#971)

Right up the same alley as the 'human fetus-eating Commie Red Chinese' propaganda stories and the 'Iraqi troops throwing Kuwaiti babies out of the incubators' high-priced Hill and Knowlton propaganda.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

I repeat:

(#929)

Because if it was actually acknowledged and proper policy prescriptions were taken, we might actually be able to eventually help bring true progress to Iraq.

The first step towards progress is open and intellectually honest discussion. No, it won't magically result in policy change, but it is a necessary condition for it (that is, necessary but not sufficient). No "speculation" here, just laying out the clear first step in a long road towards improvement.

The "math anomaly" is not one of math but rather of the practical issues that you don't always get responses from everybody (even in the US), and they acknowledge that. Ditto for the three dropped clusters: yes, it would be best if those things weren't there, but they hardly invalidate the study on their own.

And I must assure you that I am not a slow reader (my coursework requires me to read over 1000 pages/week), your personal slam is unnecessary and unbecoming. I merely provided a pessimistic estimate for your benefit.

"You reveal your own prejudice by defending the methodology" - if you honestly believe that, then I doubt we have much further to discuss. Standard statistical sampling has been used in many disciplines and many settings, and I am not "prejudiced" to defend it. Quite the contrary, I reached my judgment not before learning the facts but after them, and I have studied and am still studying these concepts as we speak. "Prejudice" means you pre-judge, or reach a conclusion without research or thought. You may still disagree with me, but you *cannot* accuse me of prejudice on this issue.

Your prejudice seems to be that you are dismissing a study largely because you do not like the results. You may not find it personally convincing, but that is not the issue: the issue is whether or not it reaches reasonable scientific standards, and, well, if you studied them, you'd know it did.

The connection between your premise and conclusion is superficial, rhetorical, and spurious, not logical. Yes, wars are complicated and humans are complicated, but that does not make their study impossible. There are difficulties, but not insurmountable ones.

You are generally skeptical of social sciences, and we are really just hashing over the same issues we've discussed before. So, allow me to generalize for a moment and tell you a minor anecdote. In the course of my readings this quarter I encountered an argument that made me think of you. It is a response to the sort of general skepticism you hold, and I believe a deceptively powerful one. Here's an excerpt (R. Harrison Wagner):

"Many people would say that human behavior is too unpredictable for such explanations to be possible. However, after saying that, such people will often literally bet their lives that what they have said is not true, by driving a car at seventy miles an hour down a highway while separated from cars traveling at the same speed in the opposite direction only by a painted yellow line. And in buying the car they drive they will have bet a lot of money that wherever they go there will be people willing to supply them with oil and gasoline to keep it running, and to fix it when it breaks down. Human behavior is, in fact, very predictable, and if it were not, social organization would be impossible."

I'm sure you'll respond that the situation is not analagous to wartime or other events, but I encourage you to actually ponder your argument before making it. Is it really that different? Is human nature so variable and fragile? Or are we conditioned and conditionable in a variety of circumstances, even the most dire ones? Militaries are often the most regimented (by definition, really) groups in society, and even guerilla groups and civilians in wartime almost always settle on similar strategies and equilibria.

In sum, you are presenting a general skepticism that is, in and of itself, unfalsifiable. Arguing over it, while entertaining, is ultimately useless. If you honestly believe that human behavior is simply beyond study then so be it, though to do so forces you to dismiss whole schools of thought and numerous actual advances of human society. If you are willing to pay that cost so you can dismiss studies whose conclusions you find odious, more power to you.

Yes, you do - and repetition does not make it so.

(#950)

Please explain how such a study would in any way affect the outcome in Iraq. Seriously.

I have no problem with open and honest discussion, I'm all for it in fact. I suggest that denying perfectly legitimate discussion from others because it does not fit your views of the relevance and validity of the study is not particularly conducive to that.

I've already acknowledged the essential validity of the math. As for your slow reading, lighten up -- my "slam" is no worse than was your "30 minutes to read" slam. A sense of humor is mildly helpful. :)

I did misstate this "You reveal your own prejudice by defending the methodology" -- should have specified that the methodology was not the issue, the timing, purpose and condition under which that methodology were used is, IMO, an issue. I apologize for misstating that.

OTOH, I am not dismissing the study because I dislike the results, those results are questionable in my mind because they are at odds with everyone else's numbers and because the timing of the release makes them suspect. Nor do I consider the study to be odious -- just suspect.

As for your quote, contemplate that the examples he cites are human acting counter to their own interests...

I'm not trying to be entertaining, nor to argue that human behavior is beyond study. I am definitely saying that sampling is subject to error and that most studies of any kind have a premise -- and they will frequently bear out that premise in spite of conflicting data. Nor am I willing or desirous of dismissing whole schools of thought. I am using some hyperbole to get you to realize that such surveys, studies and polls that produce 'metrics' deserve a fairly large amount of skepticism -- skepticism is not dismissal. I have not dismissed this study though I am very, very skeptical of the numbers.

I think we're at a standoff. You're asking me to ignore 45 years of experience -- the last third of them as a moderately senior user of the results of polls, surveys and studies in the implementation of policy -- which I'm not going to do and I'm asking you to contemplate things that are beyond your experience -- specifically, the difficulties of doing anything in a combat zone and the social habits of people from and in the Middle East (different than those exposed to the west) -- which you cannot do thus this is becoming rather pointless. Nothing pejorative in that, it's just a simple statement of fact.

The only reason to conduct such a survey is to serve a political aim; it has no practical value I can see. I fully understand your stated rationale for it but I cannot agree. It is unlikely to have any practical effect though it can admittedly have a political effect. I believe that such effect will be quite minor, you seem to disagree and that's okay.

You and the authors admit the data is the best that could be obtained -- that is, it contains imperfections. The question is how much did or do such imperfections affect the result. I submit the answer is a great deal. Aside from the vagaries of war having an obvious impact, the survey was conducted in the Middle East, an area of the world where escaping responsibility, obfuscation and bargaining are a way of life and where there is an overweening desire on the part of most to tell anyone asking a question what it is thought the questioner wishes to hear; an area where a guest in the home is accorded, to western mores, excessive deference. You can, as an Academic, dismiss that. I believe that would be a big mistake.

You called upon Livia, Mr. Alegria and I to present factual data to challenge the survey. Disingenuous at best; you're asking us to prove the negative. We have no obligation to do that -- regardless of your assertion that we do -- and, as I'm sure you realize, we could not produce it if we were inclined to do so. Good academic and rhetorical ploy but of no practical merit. If the study were as well done as you and Gabriel, a former third world pollster and thus both believers in the methodology seem to think, it would not be challenged by anyone. That it is challenged by two working Engineers and a former Government employee who worked policy issues should at least give you slight pause. Regardless, to say that it is a valid survey on mathematical grounds and that it uses a methodology used by many including the US Government is totally true as nearly as I can tell.

That does not address the fact that the probability of human foible and error due to location and circumstances adversely affect the accuracy and thus the reliability of the report.

You note that the Coalition was assigned responsibility for a higher number of deaths than you expected. That comes as no surprise, as you also noted, most people answer surveys fairly honestly. I agree and would add "in accordance with their perceptions." You also note that perception is reality, true again. Put those two together with the proclivities of people in the Middle East and the possibility of skewed data -- compounded by "no-go" neighborhoods and certainly selective picking of cluster locations for a variety of reasons and the entire survey has to be considered skeptically. As you did. And as I did -- the difference is just in the degree of skepticism accorded.

Like you and as you know, I was a Political Science major as an undergraduate. Unlike you, I did not pursue an advanced degree in the subject (and congratulations to you
for doing so) I instead went to work for a great many years in an environment that required application of my studies. During that time, I was bombarded with a number of
surveys and polls and I made two discoveries. The first was that there was always a reason for the effort -- and that reason almost invariably affected the design and thus the
outcome of the effort. The second fact was that the outcome was far more often than not called into question (usually to a greater rather than a lesser degree) by later events, be those events an election or more concrete facts later ascertained. I'd recommend that you bear that in mind as you later proceed to apply your own knowledge.

No proof of the negative here...

(#961)

...your dismissal is based on the premise that there is overwhelming systemic error, and such an assertion does require a supporting argument with at least some semblance of factual evidence. It doesn't matter how much experience you base your beliefs on, you still need a supporting logical argument. Don't ignore your experience by any means, but accept that you need more than just it to understand current issues.

As for the rest of it, see my comment upthread.

We can disagree on that aspect. My dismissal is based

(#1142)

not on the fact that there was overwhelming systemic error -- I acknowledged the methodology was sound -- but on the fact that there is no way to prove or disprove selection bias caused by the conditions in Iraq AND by the fact that a survey technique that works well in the west will encounter problems in the Middle East that the surveyor designers are probably not aware of and that the surveyors themselves may be aware of and will also fall prey to themselves.

Telling people what you think they want to hear is almost unbelievably important there...

I need a supporting logical argument under two conditions:

1. I'm in in an academic or competitive environment.

2. I am trying to refute the results on a factual basis.

Neither of those two conditions applies. This is not an academic environment -- nor is Iraq or most of the rest of the world. I'm not competing for anything; you posted a diary, Istated an opinion. I am not trying to refute the study; I'm merely saying that IMO it lacks credibility due to all the factors I've stated above.

I saw your comment up thread, and will answer it here:

1) I'm not dismissing the study totally but I'm more than skeptical of the motivation for this one and I have yet to see any logical rationale for it. That does not impugn other such studies, though I do believe they should be carefully reviewed and not accepted simply because the methodology and the numbers seem correct. Cultural and selection bias are endemic in surveys of people...

2) Agreed.

3) Agreed

I'm still waiting for the policy prescriptions you think this study could address.

I'm aware I need more than my experience to understand current issues. Fortunately, I'm reasonably well equipped to understand a great deal outside that experience. Unfortunately for your comment, this study falls directly in line with my experience in all areas save polling and statistics (in both of which I have no experience) and do recall I have not questioned the report on technical grounds.

Not to mention that experience isn't everything -- but some trumps none. :)

You should not have conceded that the methodology was sound

(#1249)

As Jackson Mead noted above the survey was intentionally designed to be impossible to replicate or test. http://www.opinionjournal.com/editorial/feature.html?id=110009108

47 cluster points to do 1849 interviews, and the interviewers did not take the time to record the demographic information of the respondents.

The survey is anti-science with a thin veneer.

The Jingoist

Hey, gimme a break here, as has been pointed out, I'm

(#1323)

an illiterate, innunerate clod stuck on fly paper... :)

As for the methodology being sound, knowing little other than that polls are rarely accurate, I'll take the view that it is as accurate as any -- that does not address /how/ they applied the methodolgy. I have reservations about that as well as both the war like conditions and the cultural aspects.

And, as you may have noticed, I still haven't got an answer to "I'm still waiting for the policy prescriptions you think this study could address."

I'm sure one of the defenders will get to that eventually...

Policy Prescription: Get out NOW and let Iraqis deal with

(#1334)

the rest of foreign fighters and death squaders and fomenters of sectarian and internal conflict, which are currently able to run rampant, with free rein, inside country while it is under the watchful caring occupation and "protection" of the greatest superpower on the planet.

That's what majority of polled Iraqis want (yeah yeah, you don't believe in commie librul polls): US out of Iraq now.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Sounds like a recipe for disaster to me.

(#1351)

See I was against the war in the first place because I could see large numbers of negative consequences, many of which have now come to pass.

But I'm enough of a realist to know that "get out NOW" as a policy would have terrible consequences of its own. The almost certain collapse into full-scale civil war with intervention from Iran, Turkey, and Syria being foremost in my mind. Breaking our committment to help restore civil order and normalcy being #2.

M Aurelius was probably right.

Umm

(#1778)

"I'm still waiting for the policy prescriptions you think this study could address."

Then you're clearly not reading my comments. I noted Jim Fearon's proposal of troop redistribution, something I think is a worthy topic, and nobody has bothered to offer me a substantial response on it.

As for the rest of it: "...but on the fact that there is no way to prove or disprove selection bias caused by the conditions in Iraq AND by the fact that a survey technique that works well in the west will encounter problems in the Middle East that the surveyor designers are probably not aware of and that the surveyors themselves may be aware of and will also fall prey to themselves." First off, these sorts of techniques have been used throughout the world with general success. Second off, those are simply not practical standards for a survey as it is never possible to fully vet your data, even here in the US of A, but that is simply not adequate reason in and of itself to reject the whole thing (unless of course you don't like the results, right?). Your rejection *does* depend on overwhelming systemic error (you can't just point to the possibility of error as reason to reject it, as there is *always* the possibility of error), and it *is* your burden to prove it.

And it doesn't matter what environment you're in or what "basis" you claim your argument has: I expect everyone I discuss these sorts of matters with to provide sensible reasons for their stances, and if they do not I will call them on it.

And lastly, once again, the motivation of this study is irrelevant to its validity.

Humma humma. I answered Fearons proposals when you

(#1810)

posted them on the old board. Haven't changed my mind.

As for the rest of your diatribe; we'll just have to disagree. I have absolutely no burden to prove my opinion to you and for you to assert that is the case is either overweening arrogance, extreme naivete or both.

The results still do not pass the common sense test. :)

Lastly, once again -- the reason for the study calls into question its validity. It may or may not affect the validity but it certainly is a relevant factor in assessing its overall value and reliability.

The tecninque was not used correctly

(#2997)

49 clusters is an order of magnitude too small. Why?

No demographic information on the interviewed. Why?

There are no legitimate answers to those questions, therefore the survey was not legit.

Junk Science.

The Jingoist

Scientific studies

(#620)

are not debunked in blogs. They are debunked by other scientific studies. Kaplan is not very good and his points have been shown to be wrong, mostly becasue he doesn't understand how the data is collected and used. Slate simply doesn't have very good writers. One reason I don't read them anymore

If you really want them I can search for the links but in the meantime let me reiterate that the 2004 study was not debunked. This is not a question of eye of the beholder. That's a Crit Lit approach to science where all views are equally valid.

The 2004 used pretty standard methodology as the statisticians that looked at it agreed. Yes, the range was very wide and people focused, wrongly, on the point estimate of 98 k deaths. Back then the crucial point was the number zero, which was not within the range and which meat that there were excess deaths. Today that debate is over and no one argues that the war has propduced excess deaths. But back then many did not want to accept that.

The important thing about this new study is not the 650k point estimate. It may very well be that the numbers overstated (although there is also the chance they are undertstated). The important point is that there are large and real numbers of excess deaths and that the death rate is growing.

Since no knowledgeable analyst has claimed the methodology applied is wrong on either study what is left is to claim the numbers are too high. After all even polls have skewed results every so often. It could easily be the case here.

One way to prove this would be to do another similar study, but no one has attempted that. Another, suggested by one of the researchers is that some reporter do a spot check in a local cemetery.

This place is my vacation.

Scientific studies are not polls raken in a war torn

(#647)

nation. Not at all.

I totally agree that Kaplan is flaky as three dollar bill; I linked to him and Yglesias only to show that innumerate left leaners can be just as dismissive of flawed pols as innumerate right leaners.

I don't know of anyone that doesn't realize there were and are excess deaths. Wars always do that. Nor do I know of anyone tha ever denied that -- the argument has always been over the numbers. Neitherr do I know of anyone denying that the rate is growing (though I predict it will drop significantly after the US election).

You can defend the methodology, you can defend any portion you wish but the fact remains that there is no practical reason to do this survey nor is there a practical way to enhance the accuracy during an ongoing war. The question thus becomes why? Why was it done?

Can you tell me why? Neither this survey nor the '04 survey say why, perhaps you know?

I'd posit one more thought. The surveyors were Iraqis, the respondents were Iraqis -- if you think that there was no agenda in the selection of clusters and households, in the way the questions were posed, in the answers that were given or in the reports of the survey teams, I urge you to visit the Middle East fro an lengthy vacation sometime soon, it'll be enlightening.

Why was this study undertaken?

(#655)

Because despite all its limitations it remains the best estimate we have of what's going on. It's not perfect but it's better than nothing.

As for your final paragraph there have been many polls taken in Iraq and, while probably not as good as the ones we have in the US, they are widely accepetd as reliable by all involved.

This place is my vacation.

But pointing out the political motivation of the study is.

(#562)

But pointing out the political impact of the study isn't.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

really?

(#566)

How do the political views of the Lancet editor show anything about the methodology used by the authors?

This place is my vacation.

An easy one

(#768)

Any report summarizing the results of an experiment or study (particularly where, as here, the data collection [if that's what it was] cannot be duplicated easily, if at all) is ultimately based upon the credibility of its authors. If the authors have a political motivation (stop the war, elect Democrats, stick a finger in Bush's eye, whatever) to reach a particular conclusion, then their veracity in reporting data that supposedly supports that conclusion (and not reporting data that does not) is legitimately in question, as is any tendency to "fudge" (like skewness, a statistical term of art) results. An obvious point, I would think.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Regarding much of your comment...

(#553)

...read my response to Gabriel below.

Regarding the "abandonment" issue, I agree it would be a problem, but by my estimation a smaller problem than the growing dilemma of staying in the thick of things. This situation is a mess no matter how you slice it, but we can at least choose the lesser evils among the options available to us.

Now THERE is a conspiracy theory for you.

(#1163)

Amazing. This is the exact same tinfoil theory floated by Pentagon flack Anthony Cordesman last week during a fleeting 'hit and run' cameo appearance on Diane Rehm's WAMU program discussing the 655K Iraqi civilians killed Lancet study.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Where does this stuff come from?

(#1171)

"Pentagon flack Anthony Cordesman"?? How much do you actually know about the guy? The last thing I read by him gave the Pentagon's quarterly Iraq report an "F" for content. What grade would he had given it if Cordesman wasn't a "Pentagon flack", an F-minus? Tell me again, exactly who did you say had the tinfoil tricorn on?

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Cordesman is an MIC [i]cum[/i] Pentagon Flack

(#1176)

(or vice versa) Feel better now?

The MIC does indeed occasionally bitch about the Pentagon not doing things the preferred MIC way (that's what defense industry lobbyists are for after all). But afterwords, they kiss and make up, and then they all just pile right back into the hot-tub of Govmt gravy together.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

What do you mean "feel better now"?

(#1185)

I simply pointed out the absurdity of you calling someone who just attacked the Pentagon/DoD on their major Iraq progress report a "Pentagon flack". Instead of responding coherently, you've gone off on another tangent while asking me if I "feel better." I feel nothing but dizzy watching your gyrations.

And the name of the band is "Gov't Mule", not "Govmt gravy".

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Excuse me while I get a refill...BRB.

(#1186)

Let me ponder while I am away why you seem to try to disassociate Cordesman from the Regime--err--this Bush administration.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Suggestion: focus on "Pentagon flack"

(#1188)

and assume I think that words have relatively precise meanings.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

After refill and a little "on background" googling...

(#1191)

I can opine that Cordesman voices the views of the MIC and BigOil aligned interests of the Bush coalition. His views/spins are largely aligned with 'Fixer' Baker's.

And they have their reasons for not wanting to be up front and transparent about the true magnitude of Iraqi deaths caused by Bush's War.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Have you told him this yet?

(#1194)

That you're onto Cordesman's game, that is? He my want to change the cut of his jib once he realizes that you've winkled him out.

I wonder what Cordesman would think if he knew I thought that his critique of the DoD's Iraq Progress Reports was actually just that - a critique, rather than a document that turned into a rave if you held it upside down under a black light. But, you know, they have their reasons.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

No, but...

(#1200)

Cordesman is certainly smart/bullheaded enough that he needs/accepts no one to tell him his business.

That is my response to your first paragraph. Not clear how to proceed on your second one. maybe i should switch to MDMA next refill?

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

I believe it is obvious

(#1307)

that my second paragraph was a swipe at your non-argument that no matter what Cordesman writes in public, he actually has a secret agenda that fits your unsupported (and borderline paranoid) preconceptions about a military - industrial complex (assume that's what you meant by "MIC"). Besides being just a tad dated and cliche'd to the max, that argument does not exactly reek of coherence. You might try the Illuminati next - much more entertaining.

So please carry on with your cocktails, adding whatever enhancement fits your mood.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Brief Addendum

(#565)

To those dubious regarding the methods of this study: they essentially subdivided Iraq into regions based on population density, and then randomly chose first regions, then cities, neighborhoods, and finally specific households to visit and survey. This technique is commonly accepted and used for a variety of studies in America and other western countries, and the execution of it in this study was about as good as can be expected given the conditions in Iraq (they explicitly recognized the difficulties posed by violence and how they did have to sometimes shift their selections due to it).

My critique of the study was meant to acknowledge that it is not infallible, but no academic study is. I do believe there is a lot of worth in the study (if nothing else it gives a good idea of how Iraqis perceive of their own suffering at the hands of the coalition, which is perhaps more pertinent than whatever reality may be), however, and as I have said elsewhere in this thread I would encourage skeptics to perform a little thought experiment: if this isn't a good way to measure the death rate in Iraq, then what would be?

It is, essentially, a poll

(#583)

and thus absolutely worthless as a method for gathering hard data, traditionally census counts and tax rolls. A lot of UN member nations have attempted to base their aid packages on this technique, with varying success.

The best analysis of this study is at StrategyPage, which points out that not only has the Lancet done this (and been discredited) before--but that going by Iraq's historic death rate, over 500,000 people would have died there since 2003 anyway. Whether this assertion is true or not, and whether it includes only natural death and apolitical murder--as opposed to politically-motivated murder--rates, I cannot say.

In any case, I am one who believes that a war is won by inflicting significant casualties, so I am disappointed at the paltry numbers either way.

Polls are worthless?

(#585)

I would certainly agree that they are often misperformed and mispresented by the media, but to dismiss an entire mathematical approach as "absolutely worthless" seems, well, overly dismissive. In fact, most statisticians will tell you that traditional census counts often have more error than statistically based studies, as the difficulties in performing a complete census are immense and prone to bias.

As far as the StrategyPage analysis, I'd be interested in a link, as the report itself compares the 2003-2006 death rates with the 2002-2003 (prewar) death rates to reach its conclusion. Admittedly 2002-2003 was relatively peaceful, and there were years under Saddam where many more were killed (Kurdish suppression, etc.), but still it seems fairest to compare 2003-2006 with 2002 and not with 1992 (2002 is simply closer to 2003-2006 and will have more variables in common), unless one can provide specific evidence to support a theory of resurgence in violence under Saddam.

And as for your last point, well, certainly casualties is an issue, but it's really *costs* that matter, and they don't have to necessarily be all that significant for one side or the other to lose its willingness to fight.

Exactly

(#589)

Remember when a few years back the U.S. Census Bureau tried to use sampling instead of the traditional head count becasue it was considered more accurate?

This place is my vacation.

Polls are absolutely worthless

(#590)

for gathering empirical data, yes. Ask any scientist.

Or John Kerry.

Not sure why you posted that

(#593)

it's clearly nonsense and I'm sure you know that. Practically all modern science is based on polls and sampling. How do you think new drugs are tested, for example?

This place is my vacation.

Any social scientist worth their salt...

(#595)

...will acknowledge that they are not studying events that can be understood with the same sort of "empirical" certainty as you seem to be seeking. I suggest reading up on some philosophy of science and similar topics. Social science is different from regular "hard" science, no denying it, but that doesn't mean it is completely lacking validity. Rather, it means that we collect the best data we can and try to come up with the best conclusions we can, but acknowledge that there is always some uncertainty and always some margin for error.

As far as John Kerry (I presume you're referring to the ambiguities in polls versus the election results), that simply illustrates my point all the more.

Suggestion. Your comment; "the best conclusions we can"

(#613)

is fair and valid and I agree with all that.

The problem, I suggest is that many polls are couched and presented -- if not by the pollster, then by sympathetic souls -- as fact when they are far from it. Too many are used as political fodder. Some are in fact just that and this one, due to the numbers, will be so regarded by both sides.

Further, many will defend such methodologies as portending accurate assessments when the fine print acknowledges they are not. The floggers be they the pollster or sycophants in essence harm the credibility of the process.

Aside from causing important issues to be ignored, that does not do the Social Sciences much good...

Certainly this is being used as political fodder

(#616)

Which is precisely why I linked to the actual study rather than the news coverage, and why I have encouraged people to read the report itself rather than link to preexisting analysis. You'll find that they actually account for many of the problems you're worrying about.

Certainly there are those who defend these processes overlymuch without really understanding them, just as there are those who criticize them in the same manner. Reality, as always, lies somewhere in between the dogma and the hype.

We can agree on your last clause... NT

(#763)

The US Census is a questionnaire/poll.

(#598)

nt

M Aurelius was probably right.

He. I rest Kierkegaard's case... :) NT

(#606)

Just don't rest it on your foot....

(#611)

He did cite census counts and tax rolls as examples of 'hard data.'

M Aurelius was probably right.

No sweat, it missed. Tax rolls are hard data (with omissions

(#762)

by those slick enough to slither under); the census, sorta -- though I note that all such things involving humans are subject to fluctuations, manipulation and just dumb errors and laziness. I'll look at 'em, but I ain't bettin' the ranch on 'em...

The study measures excess deaths

(#586)

so, from your summary of it, the people at Strategypage don't seem to know what the study does. The 500,000 they mention is already part of the calculation. The calculation is of deaths over and beyond what we could have expected, hence 'excess' deaths.

Polls, as you call them, are used to measure practically every economic and social statistic we use today, from unemployment to GDP and even much of the data published by the Census.

No wonder Tim Lambert called this study flypaper for innumerates.

This place is my vacation.

It is the same method that Intl and US Govmt agencies used

(#759)

for assessing deaths in Darfur, Rwanda, Balkans, the Indonesian Tsunami, and used the data derived as the basis/justification for taking official humanitarian/diplomatic/military action following the conflict or disaster.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Who says those are accurate?

(#962)

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Did you know

(#976)

that cluster sampling tends to under, not over count?

This place is my vacation.

There's one reason I havn't seen mentioned yet...

(#603)

... in Iraq most people are in close contact with large extended families, and probably often live in the same neighborhoods as other members of their extended family. That could lead to overreporting and/or duplicate reporting of the number of deaths in a particular "family". Even without duplicates, the reporting of deaths of second or third cousins, or relations by marriage could lead to a significant overstatement of the number of deaths per 1000 people, if the study is only counting the population of the immediate neighborhood, rather than the total membership in the extended family.

That's one of the reasons

(#604)

they asked to see the death certificates, which they got in about 80% of the cases.

This place is my vacation.

92%, actually

(#612)

Though frankly I agree with Livia, regardless of the death certificates I feel they needed to do a better job at accounting for the size of the family network. They claimed to survey something like 1890 households and in the process about 12,000 individuals, which implies slightly over 6 people per household (which is plausible for a poorer country where extended family might live under the same roof). I would like to see their actual data, as well as the precise logic they employed in taking their results and then converting them to their measure of annual death rate/1000 people.

That said, as I have said elsewhere in this thread, I feel such criticisms are really just peccadilloes and that overall their study is still quite sound and worthy of discussion. Chances are something as simple as a more complete appendix and a full release of their data would answer many of these questions.

Well

(#621)

About 90% were asked for death certificates and about 90% of those had them, hence the 80% figure.

I agree with you on the rest. Once again, this is flypaper for innumerates.

This place is my vacation.

A household size of six ...

(#622)

... is not unreasonable for a nuclear family in an American city. Two parents, four kids. Not unheard of at all.

An extended family living under the same roof in a poor country is more like two grandparents, two parents (at least), several kids. I'd say 10 at least. Add another family, and 15 seems plausible.

Not unheard of...

(#625)

...but not the statistical norm (e.g. average, mean, etc.) here, in the land of "2.2 average children."

And while I do wish they did more to specifically address this issue, I don't think it is so unexplained as you make it out to be. They do address their standards for establishing who lived in a household (something like they had to have been there for the past X months) and counted current members of the household and asked them if there were household members who died recently. So your conjecture about possible household size, while not completely irrelevant, is not unaddressed either. I suggest reading the actual study: as I have said elsewhere, it is more accessible than you might think.

And re: Gabriel, my apologies for the misunderstanding, I read your comment as in they got it 80% of the time they asked for it, not 80% of the time overall. With the latter conception you are quite correct. 80% and 90% are both quite high, so it's really a quibble either way.

You'd also have to take into account ...

(#632)

... that many fewer adults live on their own in one-bedroom apartments, and even many married couples don't move out until they have children.

The concept of "household" might also be somewhat different. I.e. the eldest male in a family is "head of household" over several children and grandchildren. The ability of the people doing the reporting to ensure that only people who actually lived under the same roof are counted might also be fairly limited.

What I posted above ...

(#734)

... I havn't heard a satisfactory reply yet to my other post - that there's no mention of how they verify that the person who died actually lived in the household.

Families might have all sorts of reasons for wishing to include deaths of relatives who didn't actually live under the same roof, and there doesn't appear to be much in the way of verification that they did.


What is "satisfactory"?

(#840)

Methinks you are demanding a disproportionately high standard of truth simply because you do not like the conclusion of this study. Obviously a controversial result will receive higher methodological scrutiny, but that does not mean you should accept it to solve philosophical and epistemological concerns here. Cartesian certainty is hardly a realistic standard even for hard scientific work, much less statistics.

Or put more simply, they verified by asking. The burden of proof is on *you* if you want to argue that there is some systematic bias where the vast majority of people are grossly misreporting their household sizes and results. You can't just say it is so or wish it so because it is convenient for your world view or because it hasn't been definitively disproven. *You* have to provide a positive argument.

Believe it or not, even when people aren't American, they are actually relatively honest most of the time. Yes, there is some incentive for Iraqis to misrepresent and be particularly hard on America, but I find it quite hard to believe that that alone explains these results. If you truly believe that is the case, you're going to need to provide a more substantial argument to support it.

I have the burden of proof?

(#843)

What? Since when is the burden of proof on the reader, rather than on the authors of the paper?

Their numbers don't match up with other sources. There is an unexplained aspect of their methodology that could be contributing to overcounting - thus explaining the discrepancy. I should think the burden of proof should be on them to show that they had adequate safeguards in place to ensure that the three month residency requirement was adhered to. It is up to them to explain why this residency issue can't be the cause of the discrepancy, and unfortunately, they don't. In order to back up their own assertion that the health ministry statistics are incorrect, they need to rule out other possible flaws in their own methodology. They have not ruled out this problem to my satisfaction. There is only one line in the report on it, with no discussion on how the 3-month residency period was verified.

If you want to disprove the health ministry statistics, what is your argument? Why isnt the burden of proof on you in that case? Isn't your motive for disbelieving the health ministry basically political in nature?

Yes, you do

(#845)

Given the nature of your critique - that there exists an overwhelming systemtic bias toward overreporting - you need to provide a positive argument to support it. You aren't merely accusing the authors of a simple logical fallacy or methodological error, but arguing that their entire premise is flawed, and that requires more than rhetoric but rather actual substance and evidence.

You focus on the supposedly "unexplained aspect" of their methodology as leading to overcounting, while ignoring the equally (if not more) serious aspect of existing counts (that they are passive) which definitely leads to undercounting. And your satisfaction is not what is at stake here, but rather reasonable statistical and scientific standards are, and frankly this study largely meets them. You are asking for a level of certainty that arguably nothing meets (I've alluded to Descartes a few times already, so if you haven't yet looked up "Cartesian certainty" I suggest you do), and I'm willing to bet that the only reason you are so distinguishing with this particular study is because you really don't like the conclusion. Talk about systemic bias, eh?

I'm not sure the relevancy of your health ministry point, but if you're referring to other counts they generally address them within the report (the bottom line is that passive observation misses a lot of cases, and the shattered infrastructure means that there is no real nationwide authority to refer to and any group that claims to serve such a role doesn't actually). Once again, I would suggest reading it.

I have read it

(#848)

But I don't have access to their data, so how do you expect me to prove that their method of verifying the residency period is inadequate? I simply don't know what they did to verify it. They chose not to include this information in their report.

This is hardly an unreasonable standard of proof to expect. Simply provide the information used to ensure that only those who lived continuously in the hosuehold before the event were counted. How hard is that?

We have two conflicting sources of information. Of course, the passive methods are prone to underreporting. However, the degree of underreporting necessary for the new study to be correct begs for an explanation. Either the new study overreports (which has to be explained by a flaw in the methodology), or else the official statistics are missing hundreds of thousands of deaths. Which is more plausible?

I'm sure we can both accuse eachother of believing what we want to believe. But it's also possible for the authors of the study to address questions about their methodology and rule out a variety of objections, such as the one I have raised. Likewise, as Iraq Body Count has done, they can address the claims that the Lancet study makes, and explain why the degree of underreporting posited is implausible. I don't this it is asking for some kind of cosmically impossible degree of proof to ask for them to address those questions. If the Iraqi Health Ministry statistics and IBC's methods are open to question, than so should everyone else's.

You shouldn't need their data necessarily

(#855)

Try to provide a positive argument to support your necessary premise that there exists an overwhelming systemic bias towards overreporting.

And yes, you are demanding a bit of an unreasonable standard of proof: no study of any variety in any country can truly guarantee that the responses of the people polled are accurate. This isn't to say that they should be taken at face value, but they can't just be dismissed either, not unless you can actually provide an argument in support of your accusation of systemic bias.

Nice little linguistic game regarding the labeling of "official" statistics. Unfortunately a moniker does not convey weight or validity, and neither have inherently more plausibility. True, something should be done to reconcile them, but you can't just assume that the error belongs more in this study than in the existing statistics simply because the existing statistics, well, already existed. If anything, new studies offer fresh blood and the potential to correct existing errors, not just introduce new ones.

The authors of this study have and likely will continue to address questions: the issue is the sort of questions you are asking. They are vague and, well, almost philosophical. You are asking for a degree of certainty that even the IBC doesn't meet. I mean, do they *really* know if every news report they tally is accurate? Do they send out somebody to audit it and interrogate the journalist? Why, maybe no deaths in Iraq are happening at all!

I have ...

(#905)

... provided a positive argument.

The lack of verification of the three-month residency period is a possible source of error that would, in fact, lead to overreporting if it was not adhered to. It is therefore completely reasonable for them to provide more detail on this point if they wish to eliminate that possibility from the debate.

The survey conductors were willing to ask for death certificates to verify that the deaths were real, so why not ask for some other evidence to verify that the person was actually living in the house?

You seem to think that the respondents would have to be guilty of deliberate deception if they included others, but that's necessarily the case at all. Maybe they didn't understand the question. Maybe the people conducting the survey didn't understand the point of restricting the count to only people who lived in the house. Maybe it was taken as an affront to have to provide proof of residency and so they neglected to mention it. Maybe the pollsters deliberately omitted it because *they* wanted to inflate the numbers.

The only thing visible is the fact that there's nothing in the report to back up the claim that only people living in the house for three months prior to the dying were counted. All there is is a single line asserting that this is the case. Yet they pay considerable attention to the issue of death certificates. The fact that they're looking for hard documentation in the form of death certificates to eliminate that possible source of overreporting, yet pass over the residency issue in virtual silence, suggests to me that they may have overlooked the importance of it.

Again

(#934)

I can believe the possibility of *some* error for the reasons you mentioned. But you *do* need to provide a better argument than that if you want me to believe that there is *overwhelming systemic error* - a trend towards overreporting that can explain the numbers being more than a magnitude of order larger than you likely are willing to accept.

Your criticisms are valid, though they apply to almost every statistical study ever, as nobody is ever able to fully vet their data in the way you are demanding and ultimately you do have to just trust your respondents. Plus, as I have argued, measuring perception is arguably just as important as measuring reality, if not more so. So, your conclusions (that is, your desire to dismiss the study offhand) do not follow, again unless you are able to provide an argument for *overwhelming systemic overreporting*.

Not necessarily ...

(#1004)

A factor of 5 or even 3 would be enough to bring the numbers down to a more plausible range. In my analogy with the family that had 4 relatives die, but was only supposed to report one, that would be a factor of 4 by itself. It would be pretty easy to get a systematic bias of a factor of 3 just by omitting the requirement that the decreased live in the house for a minimum of three months before their death. These are people with large families, lots of kids, lots of brothers and sisters, lots of different households with the same parent, all of whom might have a copy of the death certificate. In a survey of ~1800 households, even if in only (say) 1/3 of them, they were counting every relative not just the ones who resided in the house, it would be pretty easy to skew the results by a factor of 3 or more. It's almost inevitable that it's going to be some significant multiple of the true data in any case where the residency issue is overlooked. Plus, they overlooked asking for death certificates in 10% of cases, so, given they put a lower emphasis on the residency thing, they could easily have neglected to verify that in 1/3 or more. Plus, if you are arguing infrastructure failure on a massive scale, then I could just as plausibly argue that that infrastructure failure extends to the ability of surveyors to get honest data.

In the case of those death certificates, they obviously didn't just "trust the respondents". They asked for hard documentation. Although I admit it might not be possible to verify residency in many cases, this just makes it more important to be as careful as possible about explaining exactly what they did to try to make sure they only counted residents of the household, and to at least put some sort of error bound on the potential for overreporting.

One should be able to reconcile vs death certificates then

(#626)

Mr. Aaron,

If 80% of reported deaths in the survey were actually documented, then that is a testimonial to the fairly high reliability of the Iraqi bureaucracy and its data.

One should be able to just count the documents, and modify it by some factor to account for the undocumented cases. It seems to me that would be a very accurate count by comparison.

Going out on a limb by trusting ones sampling and response and extrapolating from that, by comparison, is quite unnecessary.

You got it wrong

(#628)

While getting a death certificate is something that still happens in Iraq (it's needed for a bunch of things) they are not issued by a central authority and there is no relaible central count of them. Even before the war the available counts missed about 2/3rds of all issued death certificates.

This place is my vacation.

I understand that this issuance/reporting is centralized

(#650)

Mr. Gabriel,

As per the study. It also dismisses the need to examine this data because of misreporting by the previous government, not the current one, which is a curious tack to take. It also dismisses information released by the current government according to "suspicions", without bothering to specify or analyze this information. To say this is sloppy is a huge understatement.

Huh?

(#658)

I can't make any sense of your post. What are you saying?

This place is my vacation.

That they did not bother to evaluate Iraqi data

(#663)

Mr. Gabriel,

They dismiss the whole matter of Iraqi government reporting of mortality.

"However, the tabulation of data from registration
of deaths in Iraq has suffered from the chaos of the current conflict. Beyond this, there is also a
suspicion that records of death, particularly related to violent deaths, is being manipulated and only
partially being released for various political reasons."

This is all they have to say about that source of data. To say that this is vague is to give the authors too much credit.

No they don't

(#668)

They looked at the data and found that even during the prewar the national data missed most of the issued death certificates.

Remember there have been many studies like this in other places and they all show that passive counts, like the national counts, tend to be off by 5 to 10 times with the real number. I've mentioned this several times and you've never addressed it.

This place is my vacation.

Getting 80% certificates

(#696)

Mr. Gabriel,

This is not a typical war.

That is a dead giveaway that nearly all of these deaths are recorded. This is not passive data-gathering, this information is as likely to be comprehensive as any field work.

As for the reporting by the previous government, note that they bother to evaluate the reports - not the raw data, but just the reports of the previous, unaccountable uncooperative government, and don't bother with the data of the current one.

Given the high rate of documentation revealed in their study, one would expect the natural progression of their work would be to validate the current database vs their field data and produce their own reports, or at least cover this as an approach with a high likelihood of success.

Again, you lost me

(#699)

This place is my vacation.

If 80% are documented

(#711)

Mr. Gabriel,

It is not a typical "third world" war at all, and there is likely to be sufficiently good official data to come to much better estimates than that sampling approach.

Really?

(#715)

Can you point to it? Because the researchers say it's not there. The evidence is that passive methods undercount by up to 10 times. You have evidence this is false?

Becasue this is the core of the matter. You claim you now more about how this methodologyshould be aplied than the researchers but you provide nothibng to back it up.

This place is my vacation.

How can they undercount by 10 times

(#725)

Mr. Gabriel,

If 80% are documented as per the authors own results ?

The authors dismiss this by stating that the reporting is politicized. They don't prove this nor do they provide or analyze government data.

You're assuming...

(#909)

...that the central government in Iraq is a competent bureaucracy. Reality has clearly shown otherwise. The death certificates may exist, but they have been issued by a variety of independently operating sub-groups of the government throughout the country, making any overall count quite difficult.

Perhaps ...

(#919)

... but according to IBC, which looks at this data much more closely than you or I, the local and national numbers correspond with eachother more than with the 600,000 figure.

And round and round we go

(#941)

IBC uses passive observation. That has been discussed at length in the report itself as well as throughout this thread. Their data is almost assuredly quite incomplete.

This isn't about passive observation.,..

(#1011)

In the IBC response, they say that BOTH the national and local numbers would have to be off by an order of magnitude, and somehow be orchestrated to move in concert together. But the national and local numbers relatively jive, and follow the same trendlines, which wouldn't really be the case if everyone was missing 90% of a deaths due to massive infrastructure failure.

Sure ...

(#619)

... well, when my Dad died, my mother got multiple copies of the death certificate, which she distributed to close relatives - his sisters and brother, one each for the adult children who had moved out, etc.

I don't see any reason why the same thing couldn't have happened here.

Did you read the study?

(#627)

This place is my vacation.

I have ...

(#630)

... skimmed over it. I havn't found any mention of how they are addressing the issue of extended families yet, but I'll let you know if I find one.

The onyl reference I could find ...

(#638)

... is this:
Deaths were recorded only if the person dying had lived in the household continuously for three months
before the event.

But there's no mention of how this is verified, even in the appendix. I'm not sure how it could be verified, if the person in question wasn't the one paying the bills. The three months continuously before might also be interpreted as any three months, not just the three immediately before the death.

I suspect they are just going on the assurances of those living there that the person was actually living in the house in question, which is pretty open to overreporting. It wouldn't even necessarily be malicious, just a chance for a particular family to have the the deaths of all their relatives "recognized" by someone vaguely official.

Authors said 92 % of time

(#754)

they asked for death certificates, interviewees were able to produce them.

They also said that this cluster sampling survey method is a standard appoach used by governments (including US government) and international institutions for getting best estimates of casualties in disasters and conflict situations. It was used in Rwanda, Balkans, Darfur, the Indonesian Tsunami, Congo, East Timor, and other places and this survey is even taught by the one of the US gov conflict/genocide monitoring agencies to its people.

See Johns Hopkins Bloomberg School of PH study coauthor Gilbert Burnham give good thorough very understandable discussion of their Lancet paper here in this 10/12/06 CSPAN Washington Journal video segment (begins at 1:45:30 and runs for half-hour).

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Here is an interview with Lancet study co-author Les Roberts

(#774)

Co-Author of Medical Study Estimating 650,000 Iraqi Deaths Defends Research in the Face of White House Dismissal

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Damned if you do

(#891)

damned if you don’t-

“In December 1995, the Lancet published another equally disturbing document, this time a letter to the editor from Sarah Zaidi and Mary C. Smith Fawzi. They relayed the findings of a study they conducted for the U.N. Food and Agriculture Organization that estimated that 567,000 Iraqi children had died "as a consequence" of sanctions imposed on Iraq by the United Nations in August 1990.”

There just doesn’t seem to be any way to avoid hundreds of thousands of deaths in Iraq.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

This is relevant how exactly?

(#907)

No seriously, aside from trying to score rhetorical points, is there a real point to this comparison? The current war in Iraq and the 1990 sanctions are very different events in very different time periods with too many different variables, meaning your argument (whatever it is) is entirely spurious I'm afraid.

Well let's see

(#923)

both studies attribute hundreds of thousands Iraqi deaths as outgrowths of U.S. led policies, and both studies appeared in some form or another in the Lancet. I, for one, am sensing a theme here.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

And your point is?

(#936)

You seem to be insinuating bias, but that is a far cry from invalidating the report (either report, for that matter, though I'm not going to discuss the first one as I haven't particularly examined it). If you want to do that, you're gonna have to dive in and get dirty (that is, actually read it, try to understand it, etc.). If not, all you are engaging in is ad hominem.

Let's see, what did I insinuate

(#954)

I thought it was if you were to take both reports on face value then hundreds of thousands of Iraqis died without an invasion, and that hundreds of thousands died with an invasion. It seems it was a case of pick your posion here, but either way hundreds of thousands of Iraqis were going to end up dead. How that's an ad hom I'm not quite sure, but I'm sure you'll explain to me how it is, all while touting about how you actually read the report.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

See parent comments...

(#958)

"The current war in Iraq and the 1990 sanctions are very different events in very different time periods with too many different variables, meaning your argument (whatever it is) is entirely spurious I'm afraid."

(Or put more directly, are you then saying that we had a strict choice between two options: war and sanctions?)

Hundreds of thousands

(#963)

died with Saddam in power and no sanctions against him as well, so if it is spurious to try and figure which policy does the least harm to the Iraqi people, given that hundreds of thousands were going to die no matter what happened, then I guess I am guilty.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

Sigh, read the study...

(#965)

...they reach their conclusion by comparing the death rates in 2002-3 (which was under Saddam, and is the fairest year for comparison as it immediately preceded the war and thus will have the least variables different and the most in common) to the death rates in the war.

Sigh, read a history book

(#968)

to get a further idea what this benevolent dictator was capable of.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

I wouldn't qualify Saddam's history

(#973)

as a fruitless insult. He launched two wars against his neighbors as well as ran a police state that systematically liquidated the opponents of his regime (both real and imagined). When sanctions were the international weapon of choice against him his wars stopped, but hundreds of thousands Iraqis still died, at least according to a study mentioned in the Lancet. And finally when he was removed from the equation hundreds of thousands of Iraqis still died, according to a study in the Lancet. Seems to me what they should study is how anyone is left living there.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

Your sarcasm aside

(#975)

The study clearly shows that the birthrate still exceeds the deathrate, so that answers your last question pretty clearly.

As for the rest of it, I've already addressed it in the parent comments. Reread as necessary, but Cliff's notes are: your argument is spurious and appeals to emotion rather than reason. Saddam is most definitely a "bad guy", but you still need to provide a substantial response to the study if you want to make a point (that is, yes, read it).

Citing a study

(#980)

that claims hundreds of thousands of Iraqis died as a direct result of the sanctions levied against Saddam isn't what I qualify as an appeal to emotion (which was the standing pre-invasion policy), nor is citing the policy before that (no sanctions and Saddam run amok), but whatever.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

History and facts....Be careful what you ask for...

(#1172)

Be careful to distinguish BushHistory from real history.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

and real history from pumpkin history

(#1252)

no amount of PHDs in Saddam's Iraq changes my mind that the situation in Iraq did not lend itself to an American invasion in order to establish forward bases of operation within the larger Middle East.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

What price those FOB's now?

(#1491)

Too expensive!

"Something I think most liberals don't understand is exactly how stupid many conservative leaders are." - Matt Yglesias

By what metric?

(#1730)

If we compare what we have invested in this conflict to others in our history it is quite the bargin.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

Report studied Iraqi children <5 yr old 1991-1993 period of

(#981)

morbidity/deaths according to this link:Sanctions, Saddam and Silence: Child Malnutrition and Mortality in Iraq

That 1991-1993 period would have been immediately after Gulf War I During GHWBush I's "Starve the Children" administration Iraq policy, and mostly before UN Oil for Food program got going into full gear.

I'm inclined to believe it. And also not surprised that the Bushies characteristically tried to pin the blame on Clinton for it.

GHWB I and Reagen before also put NK children on a "very tight diet", to use Bolton's words, during the 1980's and early 1990's .

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

I thought they were UN sanctions

(#983)

not just GWBI.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

GHWB I made sausage with the UN to stop food to Iraqi kids

(#984)

Clinton made sausage with the UN to re-start food aid to Iraqi kids under his 1994 drafted Oil-for-Food proposal to the UN. Link.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Well hate to break it to you pumpkin

(#986)

but in May 2000 Mohamed M. Ali and Iqbal H. Shah published a study in- are you ready for this- the Lancet called "Sanctions and Childhood Mortality in Iraq" which claimed childhood mortality doubled in south-central Iraq from 1994-1999, seems like ole Bill wasn't all that interested in feeling the Iraqi children's pain, that is if you believe in Lancet studies.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

Why don't you put up a link?

(#987)

I'm just going out the door right now, but I'm sure others can have a look at it in meantime and get more of the context of the piece, and see whether your bumper sticker interpretations of it have merit.

pumpkin out.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Because I can't find a free one

(#988)

but I'll see what I can dig up.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

Here is

(#992)


a nation article that outlines it-

“Ali, a researcher at the London School of Hygiene and Tropical Medicine, and Shah, an analyst for the World Health Organization in Geneva, conducted a demographic survey for UNICEF in cooperation with the government of Iraq. In early 1999 their study surveyed 40,000 households in south-central Iraq and in the northern Kurdish zone. In south-central Iraq, child mortality rates rose from 56 per 1,000 births for the period 1984-89 to 131 per 1,000 for the period 1994-99. In the autonomous Kurdish region in the north, Ali and Shah found that child mortality rates actually fell during the same period, from 80 per 1,000 births to 72 per 1,000.”

and a little bonus coverage regarding Bill’s concern for the Iraqi children-

“The controversy dates from 1995, when researchers with a Food and Agricultural Organization (FAO) study in Iraq wrote to The Lancet, the journal of the British Medical Society, asserting that sanctions were responsible for the deaths of 567,000 Iraqi children. The New York Times picked up the story and declared "Iraq Sanctions Kill Children." CBS followed up with a segment on 60 Minutes that repeated the numbers and depicted sanctions as a murderous assault on children. This was the program in which UN ambassador (and later Secretary of State) Madeleine Albright, when asked about these numbers, coldly stated, "The price is worth it."

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

It isn't working right

(#993)

here is the address-

http://www.thenation.com/doc/20011203/cortright

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

Its a pity the authors did not list child mortality figures

(#1114)

during the Bush 1 post-Gulf War years for comparison, nor did the latter study evidently overlap any of their study periods with those of the original report you referred to, so we could get some idea how the two survey results compared for same periods?

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Well according to the Nation article

(#1251)

the first survey was so roundly criticized that the authors had to admit that their mortality numbers were greatly inflated for unknown reasons, so I think it would be wise to take all of these studies with a grain of salt.

"We should not tie the hands of law enforcement in the effort to bring these terrorists to justice"- Leon E. Panetta

Not until we get a look at your touted paper itself

(#1290)

As some affable old Republican once said: "trust, but verify".

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Heh.

(#1006)
Bernard Guerrero's picture

Unfortunately...

(#952)

...this thread has degenerated into exactly what I hoped to avoid, though what I did somewhat expect. So, let's try something here.

To those who don't accept the study for whatever reason: you are welcome to hold that opinion, but you must acknowledge a few things.

1) The premises required for dismissing this study also require the dismissal of a number of similar studies and traditional casualty counts, if not a whole school of thought that has contributed a great deal to human knowledge.

2) Those who actually dedicate their lives to the study of these issues and concepts generally accept the study as plausible, and even if you believe that is simply because of "liberal academic bias" or such, it is still an important point (plus if you believe it is all bias then that further forces you to dismiss even more schools of thought and contributions to human knowledge).

3) And if nothing else, this study *does* a decent job of reflecting the *perceptions* of the Iraqi people. That is, you might believe that there is no way that many people are actually dying as a result of the war, but the Iraqi people generally *believe* it to be the case. And for this last reason alone, this study is worth pondering seriously and discussing the possible policy prescriptions.

It is this last point which I had hoped to actually discuss in this thread: positive policy suggestions for improvement of the situation in Iraq. Of course, it is not surprising that the whole issue degenerated in what is really an irrelevant and visceral manner.

Thank you all for reading, and to those who I'm in various thread-battles with, if you want to get the last word go for it. Our repartee, while entertaining, is ultimately quite fruitless if we cannot agree on a few simple premises of epistemological (related to knowledge) standards and rational thought.

Sadly

(#977)

there was a reason I said this was like flypaper for innumerates. And why scientists tend to vote for Dems.

This place is my vacation.

You said?

(#1000)

You seem to be quite prolific in saying so.

“I serve as a blank screen on which people of vastly different political stripes project their own views.”

er

(#1005)

in my first post on this I explained that Tim Lambert had characterized the Lancet studies as flypaper for innumerates. That's what I was referring to.

This place is my vacation.

That's twice I've laughed out loud

(#1161)

over something you've said. Regarding scientists voting for Dems, I'm sure you have proof, right? After all, this isn't a issue that common sense can decide. We need statistics. (At least that's what you said in your post at 2:29 PM yesterday.)

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

generic party affiliatio

(#1168)

among people with post graduate degrees? Pretty sure we have at least 2/3, maybe more, and the last numbers I saw were early 04. Check Stan Greenberg's excellent book, or you could probably google for some numbers. I would right now but am procrastinating on some cleaning.

If someone gives me some facts

(#1182)

to back up the claim that most scientists vote/are Democrat I will look at them. Call me old-fashioned, but IMO it's usually the person making the assertion who's supposed to come up with the supporting facts, if they can. Even scientists that vote Republican know that.

Greenberg was a Clintonista and Carville's factotum. The idea that I would borrow or buy a copy of his book to test your assertion is, well, audacious.

I used to be with it, but then they changed what it was. Now what I'm with isn't it, and what's it seems scary and weird. It'll happen to you.—Abraham Simpson

Also a serious pollster

(#1254)

"Clintonista"? The guy isn't Ann Coulter, the point of the book was laying out a strategy for democrats to win through honesty about the demographics we face. He included plenty of groups that we get killed in. With footnotes and cites for every stat. I have the book somewhere but am far, far too lazy to go get it for you, you can take me at my word or not, I don't really care. I googled quick and the only stuff I found on the first page was from the late 90s so I didn't bother clicking through.

One thing that surprised me was that college grads tend to break almost evenly, maybe a couple points of democratic advantage, but then people with post graduate degrees had the huge democratic tilt that my biased self was expecting for the college degree having folks.

But think about it. Scientists tend to live up here in the Liberal Northeast, or maybe Berkeley, and have spent a lot of time in academia. You really think they lean anything other than liberal? Almost every person I know who works in science has a strong democratic lean. Anecdotal, yeah, but you can buy Greenberg's book if you want better data :) It's a very good book, I'd read a similar one if Rove's pollster (Matt something?) put one out.

PS

(#1257)

Right now 65% of everybody agrees Bush is a lousy president so I'm sure I could give you a poll saying white gun-owning males disapprove of Bush if it was taken in the last year.

Last thing

(#1258)

The numbers I refer to but don't cite cause I couldn't google them are for all post graduate degrees, not just science. I think science is probably more liberal than "all post grad degrees" but it's a matter of debate. Fewer women which pushes you conservative a little but also zero MBAs which will push you liberal. I'm still standing by my gut feeling backed up by life experience and some uncited statistics that scientists trend heavily liberal.

I mean, Bush canned the head of NASA last year for publishing research that global warming is real. You think scientists are gonna get behind that?

Most post-graduate degrees are teaching credentials

(#1489)

...

Not in the sciences

(#1534)
HankP's picture

most science professors don't even like to teach.

I blame it all on the Internet

The sentences....

(#1538)
Bernard Guerrero's picture

....you and Mr. Alegria just wrote don't have to conflict. :^)

Terrible Research Approach?

(#1207)

From today's WSJ, written by Steve Moore of Gorton Moore Int'l and he looks like a Republican. Some key snippets.

However, the key to the validity of cluster sampling is to use enough cluster points. In their 2006 report, "Mortality after the 2003 invasion of Iraq: a cross-sectional sample survey," the Johns Hopkins team says it used 47 cluster points for their sample of 1,849 interviews. This is astonishing: I wouldn't survey a junior high school, no less an entire country, using only 47 cluster points.

Neither would anyone else. For its 2004 survey of Iraq, the United Nations Development Program (UNDP) used 2,200 cluster points of 10 interviews each for a total sample of 21,688. True, interviews are expensive and not everyone has the U.N.'s bank account. However, even for a similarly sized sample, that is an extraordinarily small number of cluster points. A 2005 survey conducted by ABC News, Time magazine, the BBC, NHK and Der Spiegel used 135 cluster points with a sample size of 1,711 -- almost three times that of the Johns Hopkins team for 93% of the sample size.

What happens when you don't use enough cluster points in a survey? You get crazy results when compared to a known quantity, or a survey with more cluster points. There was a perfect example of this two years ago. The UNDP's survey, in April and May 2004, estimated between 18,000 and 29,000 Iraqi civilian deaths due to the war. This survey was conducted four months prior to another, earlier study by the Johns Hopkins team, which used 33 cluster points and estimated between 69,000 and 155,000 civilian deaths -- four to five times as high as the UNDP survey, which used 66 times the cluster points.

...Appendix A of the Johns Hopkins survey, for example, cites several other studies of mortality in war zones, and uses the citations to validate the group's use of cluster sampling. One study is by the International Rescue Committee in the Democratic Republic of Congo, which used 750 cluster points. Harvard's School of Public Health, in a 1992 survey of Iraq, used 271 cluster points. Another study in Kosovo cites the use of 50 cluster points, but this was for a population of just 1.6 million, compared to Iraq's 27 million.

When I pointed out these numbers to Dr. Roberts, he said that the appendices were written by a student and should be ignored. Which led me to wonder what other sections of the survey should be ignored.

With so few cluster points, it is highly unlikely the Johns Hopkins survey is representative of the population in Iraq. However, there is a definitive method of establishing if it is. Recording the gender, age, education and other demographic characteristics of the respondents allows a researcher to compare his survey results to a known demographic instrument, such as a census.

Dr. Roberts said that his team's surveyors did not ask demographic questions. I was so surprised to hear this that I emailed him later in the day to ask a second time if his team asked demographic questions and compared the results to the 1997 Iraqi census. Dr. Roberts replied that he had not even looked at the Iraqi census.

Sounds pretty shoddy.

I'm trying to make the best out of a bad situation. I don't need to hear crap from a bunch of hippie freaks living in denial! Screw you guys, I'm going home!

As usual with the WSJ editorial page

(#1298)

they publish really bad stuff.

http://scienceblogs.com/deltoid/2006/10/flypaper_for_innumerates_wsj_e.php#more

I'm actually surprised the WSJ published this nonsense. What's worse is that lower numbers of clusters tend to under and not overestimate the actual dead. That's because the distribution is lumpy and there are a lot more potential clusters with less than average deaths than clusters with higher than average deaths.

More here from someone who really understands the math:

http://www.stats.org/stories/the_science_ct_dead_oct17_06.htm

This place is my vacation.

Thanks, Gabe

(#2998)

Mr. Moore destroys Lambert in the comments at your first link.

The Jingoist

Too bad no links provided so we can verify.

(#1308)

Also sounds shoddy.

And Steven E Moore is hardly an objective, or disinterested, or even technically qualified person to credibly critique the Johns Hopkins/MIT sponsored Lancet Iraq war dead study.

Moore is a war-cheerleader who runs a Sacramento Republican spin agency and a shortlived now defunct TruthAboutIraq-- "the Good News about Iraq" website, and worked as paid consultant for Jerry Bremer's CPA:

TruthAboutIraq is a website created by Gorton Moore International, a Sacramento-based political consultancy aligned with the Republican Party. Steven E Moore was a paid consultant to Paul Bremer's CPA in Iraq. "Ambassador Paul Bremer’s public opinion consultant in Iraq" according to a press release.

http://www.prweb.com/releases/2004/10/prweb164029.htm

http://www.sourcewatch.org/index.php?title=Gorton_Moore_International

They set up the website for a few months in 2004 and it wasn't updated again after the US elections. So presumumably the website was paid for by the CPA (ie your government's money!) or Republican movement to 'market' the Iraq war to the US electorate.

I suppose its possible Steve Moore created the website out of a genuine belief that things in Iraq are wonderful and continually improving, and a genuine commitment to the truth. In which case you have to ask why he gave up on the website straight after the 2004 presidential elections. You also have to wonder why there isn't actually good news about Iraq on the site!

Of bad news in Iraq, I'm afraid there is plenty every day. Eg. You can find a summary of some Iraq news from both western and middle east news outlets here:
http://www.juancole.com/
Dozens or hundreds dead every day. Death squads, rampant crime. Police corruption. Sectarian murders. Kidnapping. Insufficient electricity. Deteriorating hospitals & public health.....

Never mind. Ignore today's reality and go back to your 2 year old republican website for the news (even though it doesn't actually have any news).

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Get a subscription <nt>

(#1347)

xxx

I'm trying to make the best out of a bad situation. I don't need to hear crap from a bunch of hippie freaks living in denial! Screw you guys, I'm going home!

But don't you see?

(#1352)

That's just what they want...

.

“I serve as a blank screen on which people of vastly different political stripes project their own views.”

As stated below, a shoddy editorial slant run by a shoddy

(#1383)

editorial board.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Don't you mean ...

(#1998)

... a prescription?

Okay, seriously...

(#2044)

...do you expect to persuade any thinking human being with an editorial from the WSJ? The folks who don't care about being in line with reality will go along, but anyone with a modicum of sense remembers the heated insanity of the Clinton years and knows that the WSJ will publish anything, no matter how fact-free, if it disagrees with a liberal position of any sort.

Really, the fact that the WSJ is opining against the study is a major blow in favor of its credibility.

"In the very long run, we are all dead." -- John Maynard Keynes, 1st Baron Keynes

Hate to say it, Aaron, but I think the report.....

(#1381)
Bernard Guerrero's picture

....is crap. (Do I really hate to say that? Yes, I'm more interested in statistical methods being seen as valid, it directly puts more money in my pocket.)

The non-response rate they claim (0.80%) is, frankly, unbelieveable to me. It implies that these downtrodden, dangerously exposed Iraqis are willing to talk with a surveyor openly at the drop of a hat. You can't get numbers like that anywhere else in the world. I think it calls the data collected into question, and their other work by association.

Ahh bernard

(#1419)

The fact that it's unbelievable to you only means you never polled in Iraq.

Look at what people who actuaklly took polls in Iraq have to say:

So, what could have gone wrong? The more excitable fringes of the US blogosphere have come out with some interesting stuff. Let’s look at criticisms that don’t hold water first. Firstly, the turnout is unbelievably high. The report suggests that over 98% of people contacted agreed to be interviewed. For anyone involved in market research in this country the figure just sounds stupid. Phone polls here tend to get a response rate of something like 1 in 6. However, the truth is that - incredibly - response rates this high are the norm in Iraq. Earlier this year Johnny Heald of ORB gave a paper at the ESOMAR conference about his company’s experience of polling in Iraq - they’ve done over 150 polls since the invasion, and get response rates in the region of 95%. In November 2003 they did a poll that got a response rate of 100%. That isn’t rounding up. They contacted 1067 people, and 1067 agreed to be interviewed.

Or why don't you read what a real expert in statistics has to say:

While the Lancet numbers are shocking, the study’s methodology is not. The scientific community is in agreement over the statistical methods used to collect the data and the validity of the conclusions drawn by the researchers conducting the study. When the prequel to this study appeared two years ago by the same authors (at that time, 100,000 excess deaths were reported), the Chronicle of Higher Education published a long article explaining the support within the scientific community for the methods used.

This place is my vacation.

I have polled in Iraq and elsewhere in the ME.

(#1437)

Not polled in the technical sense but have asked a lot of people in a lot of nations there a lot of questions.

Two things to remember about the Middle East. First, in any report that has numbers, go up or down by 50-80%. Which way you go depends on the question but you can be assured the given number is exaggerated in the direction that will make the repondent look good. That was told me before I went and I found it to be true in the 70s. Others I know have been there in the last four years and say it's still true...

Secondly; you will always hear in any brief encounter what the person you're talking to thinks you want to hear. They are extremely polite.

How does the methodology account for that. All the numeracy in the world will not corral people into consistent rational and honest action or into disregarding their cultural norms.

The methodolgy you numerators can argue about -- the possible excution errors due to the inevitable chaos in the environment at the time and the cultural factors that methodology fails to consider you can ignore. Good a way to make a mistake as any... :)

Ahh...the 'irrational enumerators' vs the 'unknowable unknown'

(#1448)

obfuscators. What a topsey-turvey country we have become where gold-standard scientific survey approaches can be "discredited" so blatantly by the 'Criminal in Chief'.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Ahh, Bumpkin, you shouldn't have...

(#1461)

I'm not the Criminal in Chief, just a minor cog in the puppy blending VRWC. But I appreciate the promotion even if, as so many things, it's all in your mind. What a truly marvelous place!

Thank you.

Sorry if unclear

(#1469)

did not mean you. I was referring to the CiC [aka The Decider].

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Ah, but you are commenting on Mr. Whites opinion

(#1486)

Mr. Ash,

Not the "deciders".

Deciders [plural]?

(#1511)

Where did you get that from?

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Details, details. In his mind there seem to be many

(#1541)

deciders, all malevolent, all neocon, all militarists, all right wing, all dishonest. I susepct the world is flooded with them... :)

Dear Boy, you are never unclear. Nuclear on occasion but

(#1549)

never unclear. Your ability to get directly to the stomach of an issue from the way far left and atomize things never ceases to amaze and amuse.

[Blush] Something approaching another compliment?

(#1555)

[Lightning strikes twice]

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

You must be a Liddel-Hart fan; the strategy of

(#1579)

the diametric approach...

You certainly do it well, no obversity deters you.

Well, I think you indirectly....

(#1879)
Bernard Guerrero's picture

And it is a well-known librul-biased fact that one has

(#1559)

more serotonin receptors in the gut than in the brain.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Care to explain now, Harley?

(#1466)

nt

Will crush dissent for food

My Goodness

(#1479)

You are one touchy dude; especially for an apparent New-Age 'let-them-eat-straw' Republican, who should be well-schooled in controlled controversy techniques.

Sheesh.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Come on Joe

(#1504)

Think of him as a performance artist. PA the PA.

Support the Arts!

Laugh out loud, and give the artist his due.

.

“I serve as a blank screen on which people of vastly different political stripes project their own views.”

Boy, for a cynic....

(#1496)
Bernard Guerrero's picture

....you sure are credulous! When did "Leftist" start meaning "sucker for anything who doesn't like Shrub"?

Anyways, I'm not objecting to the statistics, I'm objecting to the data. Or rather, a really odd result in the data that doesn't tie with anything I've run across professionally or in the literature, and which runs dead-against a result so common it is nearly a cliche.

Homeboy, if ORB....

(#1493)
Bernard Guerrero's picture

.....is claiming a response rate of 100%, I have to start questioning them, too.

"However, the truth is that - incredibly - response rates this high are the norm in Iraq."

Color me incredulous about the incredible. It not only defies experience everywhere else, it defies logic. If it smells like a dead fish and looks like a dead fish.....

It's beyond your experience

(#1508)

but it doesn't mean it defies the experience of others. Those who have done polls in Iraq say these response levels are common.

Besides the one I quoted others also had similar experiences.

I googled around a bit looking for information on previous Iraqi polls and their response rates. It took about two minutes. Here is the methodological statement for a poll conducted by Oxford Research International for ABC News (and others, including Time and the BBC) in November of 2005. The report says, “The survey had a contact rate of 98 percent and a cooperation rate of 84 percent for a total response rate of 82 percent.” Here is one from the International Republican Institute, done in July. The PowerPoint slides for that one say that “A total sample of 2,849 valid interviews were obtained from a total sample of 3,120 rendering a response rate of 91 percent.” And here is a report put out in 2003 by the former Coalition Provisional Authority, summarizing surveys conducted by the Office of Research and Gallup. In the former, “The overall response rate was 89 percent, ranging from 93% in Baghdad to 100% in Suleymania and Erbil.” In the latter, “Face-to-face interviews were conducted among 1,178 adults who resided in urban areas within the governorate of Baghdad … The response rate was 97 percent.” So much for Iraqi surveys with extraordinary response rates being hard to find.

This place is my vacation.

You misunderstand.

(#1878)
Bernard Guerrero's picture

Not that I'm surprised. I'm not questioning that ridiculously high response rates can be found in Iraq, I'm questioning the oddity that the only places you otherwise find such numbers tend to involve either captive audiences (i.e. school children) or less that reliable gatherers (like the statistical arms of the government of the PRC). Having some idea of how easy it is for survey/polling teams to fake data and the myriad reasons they have for doing so (many having little to do with politics directly) I find the numbers suspicious. You're telling me to accept it at face value because it has happened before, but that's like asking me to believe that every time some tin-pot (Lukashenko, Mugabe and Fidel come to mind) wins an election with a 97% turnout and a margin of 90%, I should buy it because it's been observed previously. It doesn't make sense in any normal context, and particularly not in a war zone.

What?

(#1891)

If you want to claim they faked the data at least be clear about it. Otherwise you make no sense. Iraq is not the only country in the region that has such high response rates. In latin America you can see a similar phenomenon. They are not as used (and abused) as Americans are by pollsters and telemarketers and most people like others asking their opinion. When I worked (years ago) as a pollster in Latin America very high response rates were common.

This place is my vacation.

Show me those numbers with regards to this sort of .....

(#1901)
Bernard Guerrero's picture

....survey and I might be a bit more credulous about the Iraqi results. As it stands, I think I'm being quite clear: the Iraqi response rates are so high as to at least raise the question of fraud on the part of the data collectors.

Already did

(#1905)

Here and here.

Both have links to people who did polls in Iraq and they all agree on the very high response rates. Note that other countries in the region also have high rates.

This place is my vacation.

I don't think you did, Gabe.

(#2285)
Bernard Guerrero's picture

Quoting Iraqi response rates when I'm questioning Iraqi response rates is kind of silly, no? I'm claiming systemaitc error, possibly cause by, say, interviewers being scared out of their wits or such.

What I'm asking for are similar response rates other than in Iraq. You claimed to have worked a couple yourself in LatAm. Gimmee the numbers.....

I remember reading

(#2299)

and will try to find again, a post that showed that response rates in the region were similar to Iraq. From personal experience I recall that in Latin America response rates were much higher than in the US but can't say how high.

You are saying all polls from Iraq are simply useless. That's quite a claim.

This place is my vacation.

Not useless, suspicious.

(#2371)
Bernard Guerrero's picture

I'm open to being convinced that the response numbers are reasonable and the polls are legit. I'm just not willing to buy it yet. Frankly, I wasn't aware that the numbers reported for all of these Iraqi polls were so high, but having it brought to my attention has raised doubts in my mind about the entire enterprise. It flies in the face of personal past experience, my knowledge of the literature and, to my mind, logic. These are supposed to be folks moving out of their neighborhoods in droves in self-motivated ethnic cleansing, scared to go to the corner store or a bank for fear of kidnapping or a deadly encounter with a roadblock, never sure who works for which nefarious power. That they'd willingly open up nearly to a man to talk to strangers about politically dangerous subjects strikes me as unreasonable.

Ahh...the 'reluctant republican' scenario is recycled

(#1420)

Those Bush Regime flacks have just plain run out of new alibis to cover their dirty deeds.

“It is only those who have neither fired a shot nor heard the shrieks and groans of the wounded who cry aloud for blood, more vengeance, more desolation. War is hell.”--William Tecumseh Sherman

Yes, Pumpkin......

(#1502)
Bernard Guerrero's picture

.....my VRWC handbook only has the V6.43 of the Alibi Table on page 27. And my "flack" check bounced. [Rolls eyes at Pumpkin's callow belief system]