<![CDATA[Ed. note: the following post was written by the wonderful and passionate Sens fan known as Stefan Wolejszo (@StefanWolejszo) who attended last night’s panel discussion on the ‘Growth of Data & Analytics in Business’ that the Senators hosted at the Canadian Tire Centre prior to the start of last night’s game versus the Colorado Avalanche.
With the emergence, accessibility and growth of analytics within the hockey industry, I thought it’d be a great idea if Stefan would share his experience from last night’s panel with a bigger audience who didn’t have a chance to be present for last night’s event.
Thankfully, he accepted my request and he has graciously penned an article detailing what transpired while offering his own thoughts and opinions on the topics that were discussed.
If you’re not already, give him a follow on Twitter. Do. It. Now.
—————————————————————————————————————————————-
Not all hockey fans are into analytics, but those who enjoy looking at and working with numbers are often frustrated by the lack of information coming from their favourite team. There is a black box where NHL teams do not typically share a lot of information about the factors they measure, and analysts hired by teams are under Non-Disclosure Agreements that prevent them from talking in any detail about what they do.
The panel discussion ‘Growth of Data & Analytics in Business’ that took place at the Canadian Tire Center prior to the Senators-Avalanche game on March 2nd was unique, enlightening, and a huge breath of fresh air.
Whoever came up with the idea of doing this, and allowing Tim Pattyson (Ottawa Senators Hockey Operations, Analytics, and Research) to speak and answer questions about what the Senators’ analytics department does, should be given a raise.
This is the exact type of thing that a portion of the fan base has been waiting for, and it was exceptionally well done.
Although Pattyson was obviously limited in what he could share with the public, he was articulate and appeared to be incredibly honest in his answers. Together these qualities made for an outstanding presentation that had more than a few gems for fans of hockey analytics to enjoy.
While there were also great comments from Tom Gillis and Anatoly Tulchinsky on analytics and marketing, for the purpose of this piece, I am going to limit myself to exclusively referencing Tim Pattyson’s material and discussing his comments that stemmed from the question and answer portion of the panel.
To my knowledge, there is no audio recording of the panel discussion, but hopefully the organization will release one. Any material that I reference is based exclusively from my memory and the notes that I took during the session.
For the purpose of this piece, I will introduce each topic and borrow @6thSens’ ‘thoughts in bold‘ format to include my own thoughts.
On the early days of analytics with the Senators:
Pattyson has been with the Ottawa Senators for a long while, and he notes that Ottawa was an early player in terms of collecting numbers to help understand what was happening on the ice. He recalled collecting stats for Jacques Martin. They were not called analytics at the time, and mainly focused on bits of information that Martin thought were important such as faceoffs (including which side the draw was on). This data collection was done by people in the pressbox who tracked the events using their eyes and recording with a piece of paper and a pencil.
Much of the early innovations in data collection and tracking have been lost or forgotten because much of it was proprietary information. While contemporary hockey analytics that you see in blogs and social media can be traced back to a particular point in time hockey teams have, to varying degrees, been using different types of tracking and analysis for some time.
On the relative size of the Ottawa Senators Analytics Department
The Senators employ three analysts: Tim Pattyson heads the hockey ops analytics department and is helped by a consultant while Tom Gillis heads the marketing side that is responsible for ticket sales. Pattyson pointed out that some teams have much larger departments, and in other sports you may see a team with as many as 10 analysts on board.
There is no hiding the fact that the Senators have a small analytics department. Given that the Senators have to work within a budget and has fairly limited means to hide mistakes through buyouts it is important to be able to identify quality young prospects on ELCs and make good decisions on player contracts in general. Given how much bang for the buck a team can get by bringing in a good analyst to help with drafting, for example, it boggles my mind that the team does not invest more in this area.
On what types of things the Senators analytics department tracks
Although he was clearly limited in terms of what he could say, Pattyson did list off a few things that the Senators track:
1) Scoring chances for and against: These are coded out by the video analyst with scoring chances flagged, and then someone else double-checks this and removes events if he disagrees about it being a scoring chance. Pattyson noted that most teams track scoring chances.
2) Zone entries and exits: Pattyson noted that the analytics community is often too rigidly focused on zone entries and exists with possession. Sometimes if the skaters are bagged and really need a line change the best play is to get to center and get the puck deep in the other end.
3) Puck battles: In a comment was made quickly and without elaboration, Pattyson pointed out that there are two somewhat distinct elements to puck battles: how many a given player engages in, and how often a player wins those battles. While they are somewhat related these measure two distinct aspects of player performance.
4) Shot attempts: In a nod to the hockey analytics community Pattyson included shot attempts on his list, but he pointed out a key limitation: when using shot attempt it is important to know whether the player was driving the play or simply on the ice when it happened. He made reference to player impact on such events being coded but did not flesh it out. Although Pattyson clearly viewed shot attempt data as being limited he did single out ‘With Or Without You’ (WOWY) metrics as being very useful.
Although it was great to hear it come from the proverbial horse’s mouth I don’t think anyone will be surprised by this list. Scoring chances were one of the first things teams tracked and are likely to be the go-to stat for a good number of NHL clubs. The inclusion of zone entries and exits really illustrates that teams can actually incorporate good ideas that come out of analytics if the case is convincing enough. Puck battles and WOWY are types of information that coaches could put to good use, which brings us to…..
On who picks what variables are being measured
Pattyson made it very clear that the coaching staff is the main driver in terms of selecting which variables to track. He pointed out that Boucher is all about developing bonds with his players, and he uses analytics information to help talk to players and form relationships with them. Pattyson noted that to Boucher analytics provides key pieces of helpful information and is very supportive of it in general, and he does look at how well given matchups worked the last time the Senators played a given team. However, a key limitation in the coach’s mind is that analytics are all about the past. If a player does not “have it” on a given night then you have to modify what you do to work around it regardless of what data on past performance tells you.
I absolutely love hearing about what Boucher and his coaching staff really think of analytics, and specifically their thoughts on how it should be used. From a coach’s perspective a lot of it is about communicating key messages to players and making the best decisions possible using all of the information at your disposal. While some coaches could use the numbers as a bludgeon it is clear that Boucher uses his powers for good instead of evil.
On whether the coaches ask analysts for information, or analysts contact coaches to give them information, during the course of a game
Kudos to the very sharp audience member who thought to ask this question. Pattyson’s response was a hard no. No questions are asked by coaches during games, and analysts absolutely will not contact coaches to pass along what they feel is important information during a game. All of the communication and passing along of information happens between games.
At first glance this seems problematic to me. While it makes sense to have a filter that decides whether information is important enough to pass along to the coaches, say an assistant coach or AGM in the press box, it seem logical to pass along things that seem important regardless of when you come across that information. I think the big factor here is the coaching staff may not want to have this type of “distraction” during games, and if that is their thing then it is the end of the story. But to me it seems that this part of the process could be improved.
On how much time and effort is spent evaluating the stats they use
This was also a question from an audience member. Pattyson’s response in this area was not as polished, likely because he had to be very careful about what he was revealing. He responded that players are graded along a series of criteria and if, for example, Karlsson comes shows up as their 5th best defenseman then it means the weighting of criteria is off. He also took the opportunity to talk about how the interpretation of a given stat is the most important part, and referring to a player simply being on the ice during an event versus driving that event.
Man, the audience for this panel discussion was very smart. I felt kind of bad for him with this question because if he could speak freely about what is being measured I’m sure he could talk about it in great length using perfect examples. The basic gist is the team collects a lot of different information that is weighted to come up with a bigger picture evaluation. This type of process is probably most familiar to those with a background in the social sciences.
On player assessments, analytics, and scouting
Pattyson noted that scouts are typically focused on watching events and coming up with systematic assessments of players. In a lighter moment he said “remember than moment in the movie Money Ball where the scouts are sitting around a table giving their opinions? Well scouts aren’t dumb like the ones in that movie.” Pattyson noted that the team has evaluations of every player in the NHL and AHL, although he added that the ones for his own team are far more detailed and scouts may only see a given player a limited number of times. Ideally the team should be trying to marry the analytics and eye test. He pointed out that scouting information and analytics all feed into a larger decision making process where information is presented to Pierre Dorion, but the GM or coach is the one weighing all of the factors and making the final decisions. Pattyson also noted that when the team assesses players they take other things into account such as whether a player is streaky or a slow starter. There are also things that are not visible from the outside, like whether the player is dedicated to working out and eating properly.
It should not be a shock that teams use behind-the-scenes information on players that are not publically available. In any event, one of the most challenging things to assess is the impact of an analytics hire on a given team simply because we don’t know how much weight information presented by that analyst is given when decisions are made. We do know from interview from people who have worked with teams that building relationships, particularly with the coaches and scouting staff, is a key part to being successful. At the end of the day it is always really important to remember that the big decisions are being made at a much higher pay grade.
Conclusion
I am going to be very candid and state for the record that I was not expecting much from the event. As I was driving there I was thinking that it is probably a waste of time because the analyst is not going to share anything good. I was wrong. It was a great event and Pattyson was remarkably open and willing to engage with questions. It is not an exaggeration to say there have been few, if any, moments when I have been prouder to be a Senators fan. If other teams are not doing events like this one they should re-think things and try to copy this model. A sizable portion of the fan base will thank you for it. ]]>
Add The Sports Daily to your Google News Feed!
