Slim and none, and slim just wrote a Times op-ed

Following a Twitter challenge yesterday, Nate Silver  — whose work I generally very much admire — posted an article in the Times today that strikes me as unusually pedantic:

But I do know what an analyst should not do: he should not use terms like “never” and “no chance” when applied to Mr. Cain’s chances of winning the nomination, as many analysts have.

There is simply no precedent for a candidate like Mr. Cain, one with such strong polling but such weak fundamentals. We do have some basic sense that both categories are important. This evidence is probably persuasive enough to say that Mr. Cain’s chances are much less than implied by his polling alone. They may, in fact, be fairly slim.

But slim (say, positing Mr. Cain’s odds at 50-to-1 against) is much different than none (infinity-to-1 against). We don’t know enough about the way these factors interact, and we can’t be sure enough that the way they’ve interacted in the past will continue on into the future, to say that Mr. Cain has no chance or effectively no chance.

I think there are a few things to say about this. First, Nate is obviously right from a technical standpoint. Cain doesn’t actually have a zero percent chance of winning the nomination. But that’s also true of both Nate and myself, despite the fact that neither of us is old enough to be president, or currently running for the office. So it doesn’t tell you much. More generally, there’s almost no situation in the study of behavioral politics (or social science, for that matter) in which we could makes such a claim. Forever is, as they say, quite a long time. If that is Nate’s point — that people saying that Cain has no chance should actually be saying he has “less than a 1% chance” — well, fine. But in that case it’s just semantics, or maybe a criticism about imprecise writing.

But that’s not what bothers me about the article. The real problem is that Nate seems to more or less agree with the people who think Cain has no chance. He concedes that Cain’s chances might be “slim” and then suggests that “slim” might mean slightly less than two percent. In effect, Nate is doing exactly what he claims the analysts shouldn’t be doing: disregarding the polling numbers and putting the vast preponderance of the explanatory weight on the fundamentals, or their intuition. How else can you get the polling front-runner down to 2%? But if it’s “arrogant to say that the man leading in the polls two months before Iowa has no chance,” then it’s probably pretty arrogant to make him a 50 to 1 longshot.

Nate also wants people to put their money where their mouth is. He offers a silly one-sided bet — will you quit your job if Cain wins? — in an attempt to prove that Cain doesn’t have “no chance.” This goes right back to the semantics. But we can harness the market without betting our livelihood at infinity-to-1 odds. For instance, Cain is currently trading at 7.4% to win the nomination on Intrade.  That’s far below his polling numbers, but I have a hunch it’s still well above Nate’s estimate of the true probability of him winning.

So I’ll throw down a counter-challenge to Nate. I’ll admit that I’ve been guilty of saying Cain has “no chance” of winning the nomination when I actually meant he has less than a 1% chance. I also think that the public opinion polling showing Cain in the lead is basically worthless (not because it’s wrong, just because it has little predictive value relative to the fundamentals). So here’s the bet: if Cain doesn’t win the nomination, Nate, you buy me a happy hour beer next time I’m in New York or you’re in DC. If he does win, I’ll treat you to a very fancy dinner (say, $400/person) next time I’m in the city or, if you prefer, donate the money in your name to the charity of your choice.

That’s roughly 100 to 1 against. If Cain has anything more than “no chance” of winning the nomination, it should be a very attractive bet.

Update: Jonathan Bernstein posted a very nice piece that is both a more complete and much cooler-headed analysis/critique of the situation, and which I highly recommend reading.

Update II: Hans Noel has a nice post up over at the Monkey Cage regarding all of this, and he’s definitely an expert that everyone should listen to. Also highly recommended.

Share

11 thoughts on “Slim and none, and slim just wrote a Times op-ed

  1. Pingback: Polls and who really gives a damn about opinions « sublimemonkey

  2. Pingback: Can We Clarify When Polls Matter? | Con Games

  3. Pingback: Link roundup « Negative Interest

  4. Pingback: Great Pumpkin 'Outliers'

  5. Pingback: FiveThirtyEight: Reads & Reactions | Il Neurone USA

  6. btc

    I think Nate’s point was more along the lines that Cain’s position is so unique that previous predictive tools may well not apply. We’re starting to get into “what are the error bars on your error bars?” territory.

    Reply
  7. Bill

    From the article you are talking about:

    “A toy model I have that accounts for both polling and nonpolling factors, including things like Mr. Cain’s lack of traditional experience or credentials, would put his chances at more like 10 or 15 percent.”

    He does go on to talk about the high level of uncertainty do to how unusual the situation is, but saying that Nate thinks Cain has a slim to none chance of winning is inaccurate.

    Reply
    1. Matt Post author

      That’s from Nate’s follow-up article, which came out after my critique. The original article says what I quoted in my piece, and only what I quoted, which was this:

      This evidence is probably persuasive enough to say that Mr. Cain’s chances are much less than implied by his polling alone. They may, in fact, be fairly slim…But slim (say, positing Mr. Cain’s odds at 50-to-1 against) is much different than none (infinity-to-1 against).

      Now, in that passage, Nate is not exactly saying that Cain has a 2% chance, and that why I didn’t say Nate explicitly said it: I said he said Cain’s chances might be slim, and that slim might mean 50-to-1 against.

      He has obviously since revised. But I stand by my original critique. Reread his original article and my critique, and I think you’ll agree.

      matt

      Reply
      1. Bill

        I apologize. I think my confusion came from the New York Times claiming that your critique was of his follow up article. I just took a look and realized that your quote was of a slightly earlier writing, before he mentioned the new model he was toying around with.

        I could say that Nate saying Cain’s chances could be 50 – 1 is not the same as him saying they are 50 – 1 but then I would truly be arguing semantics. Instead I will say that I’m glad Nate is the kind of guy to try to find a mathematically supported argument for why Cain’s chances are as good as polls show and I will say that your critique of his original article brings up a good point.

        Once again, sorry for my confusion.

        Reply
        1. Matt Post author

          No worries, Bill. And you’re right — Nate never says it’s 50 to 1, he merely implies that as a plausible estimation.

          Reply
  8. Pingback: Double Herman-uetic | Matt Glassman

Leave a Reply to Bill Cancel reply

Your email address will not be published. Required fields are marked *