Monday, November 7, 2016

Election Eve Update 2: Commentary on 538s Prediction

In the last few days there has been a lot of ink spilled over 538's projection which (ludicrously) gives Trump a 30% chance to win tomorrow. My model gives him markedly lower odds. I wanted to take a moment to highlight the key differences between my model and the 538 model, and explain why I think my methodological choices were better.

  • How national polls are handled
    • To quote Jed Bartlett from The West Wing: "There are times where we're 50 states and times when we're one country." National polls can be thought of as 50 individual state polls, and modeled accordingly. I wrote a very thorough post on this technique here.
    • FiveThirtyEight uses trends in the national polling to adjust state polls. The following represents my best understanding of his methodology: 
      • If, from last week to this week, the national poll average has moved 2 points in Trump's favor, FiveThirtyEight adjusts all individual state polls from last week to be 2 points more favorable to Trump . For example, a poll from a week ago in Florida showing Clinton +1 would be adjusted to Trump +1. 
      • This over-amplifies the voices in those latest national polls. At any given time those "latest national polls" which drive the trend have a few dozen or few hundred voters from any given state. FiveThirtyEight uses those voters to adjust the voice of the thousands of voters aggregated in the individual state polls.
  • How polls are weighted
    • Polls are voters. I weight polls by how many voters they include and how recently they were taken. This, plus my national poll technique, has caused a big national poll taken by NBC and SurveyMonkey to be weighted fairly heavily on my model. I've written about that more extensively here
    • FiveThirtyEight weights their polls based on age and (I think) their rating of the pollster.
  • Level of uncertainty
    • I've calibrated the level of national uncertainty based on prior elections, and I've calibrated the individual state uncertainty based on an unbiased poll taken on Election Day. 
    • I'm not certain how 538 has calibrated for uncertainty, but I think they are overestimating it, and here are a few examples of states from FiveThirtyEight's model that don't pass the sniff test
      • Michigan: Trump has led in 3 out of 86 polls, yet 538 gives him a 21% chance to win
      • Wisconsin: Trump has led in 3 out of 80 polls (and not in a single poll since September 8th) yet 538 gives him a 16% chance to win
      • Minnesota: Trump hasn't led in a single poll in 538's database, and Clinton has led many polls by double digits, yet 538 gives him a 15% chance to win
  • Two-way vs. Four-way polling
    • I use two-ways polls (Clinton/Trump), while 538 uses four-way polls (Clinton/Trump/Johnson/Stein), I've written here on why I think two-way polls is the appropriate choice. In short, it's functionally a two-person election.

No comments:

Post a Comment