College Football Math: Rankings

This week’s AP poll has Arizona has #10. If you take that to mean they should be favored over every team ranked below them, it's an absurd ranking. 

Arizona had a great game against one of the best teams in the country last week, and they’re unbeaten. Except they needed a hail mary to beat Cal at home, and their other wins came over UNLV, Nevada, and UTSA. Based on what we've seen thus far, Arizona is not the 10th best team in the country. They might become that team, they might get into the CFP and win a national championship. But odds are they won't. If #10 Arizona isn't the 10th best team in the country - it begs the question: what kind of rankings are the AP voters doing?

There are two bases on which one could rank football teams: achievement and prediction.

Achievement

• Achievement rankings say something about the past. They order teams based on what those teams have done on the field - who they've beaten and by how much
• As a fan, this feels more satisfying since teams can only control what they do on the field
• But rankings based solely on achievement are bad at predicting the future
• Achievement rankings might be:

That looks satisfying. We could quibble over the specifics of my methodology and the placement of specific teams in specific spots, but I think they work well enough. Most of the undefeated teams are near the top, Arizona included. Utah gets a nice bump for their win over UCLA, and Ole Miss/Auburn are high. UCLA is high too. They did lose to Utah at home but they've also played a tough schedule and they walloped ASU.

The rankings look nice, but unless you think Vegas oddsmakers are very wrong about a lot of games, these rankings are a disaster from a predictive standpoint. Some examples:

• They rate Stanford a 5.0 favorite over WSU (Vegas has them 17.5).
• They rate Notre Dame a 35 point favorite over UNC (Vegas: 16.5 points).
• They rate Arkansas an 12.5 point favorite over Alabama (Vegas: 9 point underdogs)  

Overall, the correlation between lines generated by these “Achievement” rankings and lines generated by bookies is 0.66. That's a terrible correlation.

It looks to me like the AP voters are ranking teams on Achievement, with some level of recency bias. That’s fine, but if that’s what they’re trying to do, we should stop thinking of teams as necessarily better at football than teams ranked below them.

Predictive

If the AP voters are trying to do predictive rankings, they’re doing it wrong. Football is full of craziness, good days, bad days, and the predictive power of any single football game isn’t as strong as the AP voters seem to think.

• Predictive rankings attempt to say something about the future. Each team would be an underdog against every higher-rated team, and a favorite against every lower-rated team.
• This is what my models rankings are - an attempt to identify which teams are the best teams
• This week’s predictive rankings look like:

That certainly looks less satisfying; at least it does to me. Namely, it seems wrong that Alabama is first and that Oregon/Stanford/UCLA/Oklahoma are all in the top 10 despite losing this week. But these ranking jive much better with this week’s odds:

• They rate Stanford a 19.3 favorite over WSU (Vegas has them 17.5).
• They rate Notre Dame a 17.2 point favorite over UNC (Vegas: 16.5 points).
• They rate Arkansas an 8.8 point underdog to Alabama (Vegas: 9 point underdogs)  

Overall, the correlation between lines generated by these “Predictive” rankings and lines generated by bookies is 0.95. That's a very good correlation.

All this extra look into the model's ratings means I'm a couple of days behind on the regular steps of updating. I'm running a simulation now and should have the dashboards and Watchability ready to go tomorrow.