The NHL Analytics Debate, My View

In North America, beyond church and sports, there has been a long tradition of discussing politics in shows like NBC’s Meet the Press and CBS’ Face the Nation which date back to the earliest days of network television.  This Sunday, Twitter ranters went after “the truth isn’t the truth” statement by former New York Mayor Rudy Giuliani.  For those of you worried I am about to go on some political diatribe I am not.  The statement got me thinking about the never ending debate about NHL analytics and their place in the game.

Recently, the Athletic‘s James Mirtle wrote an article discussing this debate between various crowds who cling to the ‘eye test’ versus those who believe such views are outdated or even distorted and think they should be replaced with advanced statistics which can objectively evaluate the skills of a player.  It is an approach made famous by the film Moneyball that stats can identify players who really help their respective teams have success without the biases of sentimentality, emotion and other intuitive assessment.  In concept, it sounds clean and objective.

Its nothing new.  I remember reading in an The Hockey News about former Chicago Blackhawks General Manager Mike Smith opening a statistics consulting firm where he and his staff collected data on all of the league’s players and claimed they had information as to who really makes a difference and those who are just overrated.  It seemed kind of strange when I read this 9 years ago, but analytics are simply a standard part of every NHL organization these days.

Part of Mirtle’s article discusses the examples fans on both sides of the analytics discussion give how its valuable or how it is flawed.  NHL players also have their questions about these statistics and how they are tabulated, like Devan Dubnyk who gave this quote to the Minneapolis Star Tribune‘s Chris Hine last winter in an article about the Minnesota Wild’s ability to prevent high-danger scoring chances, “I don’t think that’s a really accurate thing. Don’t get me wrong, we’ve been playing great.  I’m just talking about the actual — I would like to see how it’s calculated because so many things happen in a play, did it hit his shin pad and go 3 feet to the left outside of the high-danger area?”

I think Dubnyk brings up a fair point.  Even some of the most simple statistics the NHL compiles, like shots or hits can vary greatly from arena to arena.  The Athletic‘s Michael Russo has pointed out numerous times over the years how in some arenas merely shooting a puck towards the goal is enough to be counted a shot, while others can be tremendously stingy.  Popular analytics websites like Corsica base their statistics on the data the NHL provides, but if what Russo said is true about the relative tabulation of shots, hits, etc then doesn’t that put into question the accuracy of such stats?

If a player was on a team who’s arena staff was generous in tabulating shots, hits, etc wouldn’t that inflate their statistics over players who play for clubs whose statisticians are more strict?  Another example from Mirtle’s article was a stat similar to Major League Baseball’s WAR (Wins Above Replacement) is the hockey version called GAR (Goals Above Replacement), where according to such a statistic the Washington Capitals’ John Carlson is more valuable to their power play than Alex Ovechkin.   At first glance such a statement seems ridiculous when comparing Carlson to Ovechkin who had 17 power play goals and 229 (and counting) power play tallies over the course of his 13-year NHL career.  But if Carlson is the true driver of the Capitals’ power play last season as GAR would indicate then perhaps intuition is wrong.

Possession metrics like Corsi and Fenwick used to be the main points of discussion and frustration among fans who loathe or embrace what analytics bring to sports.  But a whole host of new data has been introduced and keeping up with the latest trending statistic can be a challenge.  Without question there will continue to more attempts to quantify and evaluate what we fans see on the ice and translate it into some kind number.  The new evaluative metric will likely have it’s supporters and detractors.  As I learned when I did my Master’s paper, quantitative studies are fairly straight-forward while qualitative studies are much more nebulous to analyze.

I’ve supported the concept of a scoring chance statistic created by former New York Rangers’ goaltender and TV studio analyst Steve Valiquette which assesses the relative quality of scoring chances.  Instead of more quantitative measures which simply count scoring chances, Valiquette’s concepts identify the components of a quality scoring chance.  This could help a team identify which players truly create high quality chances and those who simply play catch with the goalie to inflate their shot totals.  Will such stats indict a player like Ryan Suter who takes an unscreened wrist shot from the point on the power play or is he helping push the offense?

So it goes back to Giuliani’s quote, “the truth isn’t the truth.”  Statistics have value for those who perceive that data to be accurate or inaccurate.  Can you accept some data but still evaluate what you see on the ice for yourself?  I think you can.  Will that compromise be acceptable to those who hold analytics to be empirical perfection?  Perhaps not, but we see the world through our own lens and as ludicrous as it may sound, others may not see reality the same way we do.  The best outcome is to simply accept they have a different view and leave it at that, because its unlikely many minds are going to change with such deeply entrenched beliefs one way or the other.