On his show on Monday, Bill Polian made some comments about statistical analysis.
I think it was (CBS broadcaster and former New York Giants quarterback) Phil Simms on a broadcast I heard who said, ‘Statistics are virtually meaningless in football unless you can match them up with videotape and say this is what this statistic specifically means against this defense in this situation against this offense in this situation. Standalone statistics such as exist in baseball – sabermetrics, if you will – have no validity in the NFL. Zero. I was glad to hear Phil say it publicly. It’s not said enough. There is a growing – albeit small, but noisy – cadre of people who try to convince you that standalone statistics are really meaningful. They’re not. As (Colts linebackers coach) Mike Murphy is fond of saying, ‘There are two statistics that count on defense: points surrendered and turnovers.’ That’s all that counts. Everything else is meaningless. There are some we use where you do match it up with a front or a coverage. We do some pretty sophisticated analysis of it, but you have to match it with fronts and coverages. You have to know what you’re looking at. You have to understand why a particular statistic has validity. I’ll give you an example. We’re now talking about receivers being “targeted” as opposed to how many balls they catch. All that tells you is: a, that the ball was thrown his way – it doesn’t tell you why. It doesn’t tell you what kind of a ball it was – whether it was good, bad or indifferent. It tells you whether he caught it. It creates a totally meaningless picture. I guess fantasy people look at it. As I’ve said many times, I have no idea what that’s about. I’m sure it’s meaningful to them, but in terms of winning games, most of them are not meaningful.
There’s more to the quote, and you can look it up and read it. The comment touched off a firestorm over at the Footballoutsiders. The comments echoed similar dismissive comments that Polian made at the MIT Sloan Conference last year. As someone who relies heavily on the Outsiders numbers and on numbers in general, let me make some observations.
1. Polian is mostly setting up a straw man here. For instance, he attacks catch rate, and is 100% correct. Catch rate, applied in a vacuum, is a useless stat with no validity. You cannot compare two wideouts just by catch rate and know anything about them. Fortunately, I don’t think anyone does that. When considering a player’s catch rate, you have to look at his yards per reception (a higher YPC generally means longer routes, which makes a lower catch rate ok). You have to take into account the quarterback involved and his overall accuracy. You have to consider the offensive line and the number of hurries they allow. Even so, no one is claiming 100% precision. It’s one snap shot, one piece of data, not a whole picture.
Granted, all of that doesn’t happen in every article that mentions catch rate, but it’s all there as part of the background, informing and shaping how we use the stat. Polian is 100% correct in his assessment of what has to be considered to understand catch rate. But everyone already does that. If you dropped into a site like 18to88 or just read tables over at the Footballoutsiders.com, you would not get a good picture how over time the discussion and context for these stats is taking place. We can’t explain all the nuances of every stat in every article. We have to depend on our archives and on the reader participating over the long haul in the conversation. Nothing drives me more nuts than having to go back and explain basic concepts to people who just eavesdrop on one aricle (see my recent wars with idiot Pats fans who think QBs win games).
2. Stats are descriptive. All a statistic is is a description of what happened on the field. It’s a code. It represents something real that took place. People that argue that stats have no value force fans to rely on unprovable platitudes and anecdotes. Are football stats imperfect? Yes. Can we perfectly measure the impact of Dwight Freeney? I’ve been arguing that’s impossible for years. The irony is that the stat people are the ones watching the most game film, soaking in the most context. People like Outsiders are trying to find ways to quantify what they see on tape. Stats are just a form of description and commentary on the action. No one number can do it, but the more numbers you get the truer your picture of the game is. Stats are just a form of evidence. Polian loves to rip ‘pundits’ who spout opinions. He loves to rip stat guys for meaningless numbers. At some point, he should just say, “No one outside the NFL knows ANYTHING about football”. I think he has, come to think of it. Frankly, I reject that philosophy. It’s true, we don’t all have access to the famous “All 22” game film (aka ‘coaches tape’ showing all 22 players at once from a high angle), but we SHOULD! Maybe if Polian would do more to give fans and writers better access to usable game film (instead of a CBS tape of Peyton’s face at the line, when I want to see where the TE is lined up!), fans and writers would ask him fewer dumb questions.
3. Polian is defending his players. Polian has this ‘crazed grizzly’ mode when he has to defend a player or coach. The catch rate comment, I believe, goes back to recent criticism of Pierre Garcon. When Polian senses one of his guys is under fire, he cuts loose and blasts everything that moves. I’m ok with that. He’s still basically wrong, but I get why he’s doing it. Ironically, if you go back and read that piece, you’ll find just the kind of ‘context’ at the end, that Polian says is necessary to make stats useful. In this case, Polian is arguing that Freeney has way more value than numbers can show. Anyone watching the last three Colts games would agree.
4. I don’t care that he says stuff like this. Do I wish Polian would embrace stats? Yes. I’d love for a big ol’ snuggly hug from him. (Uh, a verbal hug. How about that?). But the man dislikes press and loathes bloggers. He’s also the greatest GM of the past 25 years. I honestly don’t care what he says or how he acts. I don’t care if he calls all fans morons. I really don’t. I JUST WANT HIM TO KEEP BUILDING A WINNING TEAM. Seriously, who cares? The truth is, I think most people are idiots too. Do I like it when he lumps me into ‘most people’? No. No one wants to be ‘most people’. However, we are on the verge of 9 consecutive playoff seasons, so if anyone has earned the right to call me an idiot, it’s Bill freaking Polian. He IS a lot smarter than I am. Compared to him, I AM an idiot. That doesn’t mean he’s right in this particular instance, but it does mean that I’m not going to get bent out of shape about it.
5. He’s attacking his strongest defenders. You know who loves Bill Polian? Stat guys. You know who loves Polian players? Stat guys. Why? Because good players show up in the stat sheet. God help me, I refuse to return to the day when Colts’ commentary was limited to “Our backup QB/RB is better than the starter! Coaches are stupid! Why don’t we play that guy more!”. It’s been the stat guy who has said all season that the line isn’t as bad as people think, that injuries are sinking the team, that the team was well constructed.
At the end of the day, stats are just a way to try and stay rational and objective about football. They can only tell us so much. If only, oh if only, there were sites out there that focused on trying to use numbers in context (instead of randomly) and people who watched a lot game film to try and interpret what was really happening.
Sigh. I guess we are all doomed to be lost in the dark.
Add The Sports Daily to your Google News Feed!
