Jonathan Simon stomps Manjoo, Mark Blumenthal

The following is a quote from Farhad Manjoo’s recent attack on RFK Jr.’s article:

You can find the exit poll data that Freeman relies on in his first report on the election, located here: In Iowa, this exit polling data showed Kerry ahead by 50 percent to Bush’s 48 percent; in Nevada, Kerry was ahead 49 to 48; in New Mexico, he led 50 to 47; and in Ohio, he was at 52 to 48. As the pollster Mark Blumenthal has pointed out, the margin of error in these states varied from 5 to 7 percentage points. In none of these states does Kerry’s lead even come close to that level.



MOE (at 95%C) is roughly calculated as 1/sqrtN, where N is the number of ballots sampled. A 5% MOE would characterize a sample size of 400, and a 7% MOE would characterize a sample size of 200. The actual exit poll samples in the battleground states Manjoo mentions ranged from 1951 to 2502. Associated MOEs were about 2%.

If Blumenthal is using the cluster effect to increase the MOEs by 300%, he should know better. The cluster effect might be generously relied upon to increase the MOE by 50% to about 3%, and in fact probably significantly less because of the precision with which the raw data was demographically stratified. If not satisfied with standard statistical analysis, we can reference Edison/Mitofsky’s own methodology statement for the actual MOEs.

Where Blumenthal comes up with MOEs of 5% to 7%, and how Manjoo can quote such absolute nonsense, is truly a mystery worthy of the “mystery pollster.” But it does suggest a level of intellectual dishonesty belied by the disarmingly temperate and “balanced” tone adopted by such critics. The muddy-waters strategy has served well their cause, and harmed democracy irreparably.


0 replies on “Jonathan Simon stomps Manjoo, Mark Blumenthal”

Thanks to Jonathan for carrying the fight. As he suggests, if Mitofsky thought that the MOE should have been much larger due to clustering uncompensated by adjustment for stratification, why didn’t he report it accordingly?

However, I want to interject a note of caution on this. The true MOE is, by the nature of things, unknown.

Example: Suppose there are two identical precincts in a state. We have been studying the elections in the state for 100 years and both precincts have equally accurately predicted the outcome. However, last year, a community catering to the elderly is created in one of those precincts. So, people move from one precinct into the other. Now, neither precinct predicts accurately.

“Ah”, you say. “We can take care of that situation by consideration of stratification.” But stratification is only as good as the categories used for it. Sometimes the categories aren’t as evident as age.

The bottom line is I think we have to take Mitofsky’s admissions or indirect admissions of incompetence at face value. He knew about differential response years and years earlier, but made no serious effort to research or correct it– even after the fiasco of 2000. We have to accept the possibility that he screwed up the exit polls so badly that he does not dare to permit close examination of the results.

Of course, that means that it would be a national disgrace if this man were to ever again be paid by any media company to run an exit poll.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.