Are the Oilers really getting shafted by the referees?
In true blogging fashion, I’m going to equivocate and say “yes and no”.
This season I continued a project I’d begun late last year where I essentially audited the calling of infractions in a random sample of NHL games. I collected games on the Oilers, the Flames and then a control group of other NHL games featuring neither of those two teams.
If you’re read previous posts on this subject you are already likely familiar with my process, but for those who haven’t here’s an explanation of what I’ve done, or you can just skip down to Results for the meat of the article –
I use the NHL’s own rulebook when assessing whether an infraction has occurred, putting it into one of two categories, Physical or Technical (more on those in a moment), and then assigning a severity ranking as well as the time of the infraction and the jersey number of the player who committed the infraction and against whom it was committed. All infractions are marked as either Called or Missed.
Each infraction gets recorded and coded using the following criteria:
Missed (M) or Called (C), depending on whether the infraction was a called penalty or not.
Physical infractions (P) include elbowing, cross-checking, slashing, high-sticking, boarding, hits to the head, charging, kneeing and such.
Technical infractions (T) include interference, tripping, hooking, holding the stick, delay of game, and so on.
Severity (0, 1, 2, 3) is graded from 0 to 3.
• 0 is a phantom call.
• 1 is a weak call of the sort that I wouldn’t want routinely enforced because it would distract from the physicality and emotion of the game.
• 2 is a fair called, an earned penalty that fits the letter of the law and is apparent enough to warrant being enforced.
• 3 is a blatant or obvious call, an egregious infraction within sight of the officials (presumably, at least) whose not being called is an act in and of itself.
I record the kind of infraction committed; high-sticking, interference, elbowing, cross-checking, slashing, and so on.
I add the time of the event using the game clock and the jersey numbers of the players involved.
So a missed slashing infraction by Corey Perry on Leon Draisaitl might look something like this: M, 2P Sl, 10, 29, 15:54.
Altogether, the percentage of times a penalty is called compared to how often it isn’t I refer to as a Rate of Call (RoC).
I am acutely aware that the entire basis of this exercise rests on my own subjective perception of the game. All the data points flow from my own observation of events which are, to some extent, open for debate. I am often critical of this method in other areas of game tracking that involve subjective evaluation, so I am being up front about this here and now.
As a way to try and account for this I have adopted a basic approach being that I will publish each game’s raw data here for easy review. You can simply go to any game listed, forward to the game time marked, and observe the two players listed to compare my findings with your own view of the event.
In total I logged 16 Oilers games dating from October 16th to Jan 21st. I also logged 12 non-Oiler NHL games involving teams from around the NHL and to which I added 4 Flames games to bring the total control group up to an even 16.
Let’s skip straight to what I found (the images often come out a little small, so click to enlarge).
The Oilers committed 35% of the uncalled infractions in the games I logged while their opposition committed the remaining 65%. Despite this disparity, both teams received a total of 48 penalties, meaning that while their games were penalized at an even number of powerplay opportunities, the rate of call on infractions committed was significantly different, 17% of Oilers infractions were called while only 13% of their opponents’ infractions were penalized.
What that means is the opposition receives 13 penalties for every 100 infractions committed whereas the Oilers receive 17 penalties for every 100 infractions committed.
So why don’t I characterize this as an obvious example of biased NHL officiating? Because the Oilers commit far fewer infractions and despite being penalized at a higher rate, are still amongst the least-penalized teams in the league.
This is a tricky issue and one that separates our understanding of “justice” and rule enforcement as fans from the actual application of the rules by the referees within the game.
We’re going to have to look more deeply at what rules are being broken, by whom, how often, and to what extent in order to fully flesh out the peculiar landscape that is NHL officiating.
Breaking Down the Numbers – Time for Pie (charts)
Going back to the data I’ve mentioned and illustrated above, let’s break it down by infraction type and severity.
In total the type 1 infractions were rarely called though often committed. I would often record them more as a way of maintaining a minimum event-threshold level than for analysis and documentation purposes. At this time there’s no real benefit to covering them in detail, though we will discuss them within another context shortly.
So we’ll skip to the type 2 infractions. These are fair penalties, called and uncalled, such as a trip that, intentional or otherwise, cannot reasonably be overlooked, even if it remains unpenalized.
The Oilers committed a little more than half as many physical type 2 infractions (elbowing, boarding, slashing, cross-checking, etc) as did their opponents, but were called more than twice as often (5 penalties to 2) while they committed just one-third as many technical type 2 infractions (tripping, hooking, interference, and so on) as the opposition but were far more likely to be called, receiving 17 penalties of this type to only 7 to their opponents.
They trailed in committing type 3, obvious physical fouls by roughly 2/3rds and while they received fewer penalties overall in this category, they were more likely to receive a penalty for the infraction than their opponent. Contrast this with blatant technical fouls where the Oilers committed 18 uncalled infractions to the opposition’s 65. The Oilers only garnered 5 penalties for these actions to the opposition’s 21, and it is in this area alone, with such dramatic disparity between the two, where their opponents were penalized with greater frequency than the Oilers.
Above is a game-by-game chart of the total infractions committed, whether they were called or not, by the Oilers and their opposition. In the first two games they committed a greater number than their opponents, yet after it was largely the reverse, closing the cap occasionally but more often than not committing far fewer than the opposing team.
So let’s look a little further into that…
Overall the Oilers and their opposition received exactly the same number of penalties, 48, during the course of the 16 games recorded.
However, during that same time the opposition committed a total of 310 uncalled infractions to the Oilers’ 168, meaning that the Oilers were penalized on 17% of the infractions they committed while their opposition was penalized on only 13%.
That 5% difference is where we find the basis for the common refrain that referees like to “even up the calls” in a game, providing powerplay chances for both sides at a near-even amount, barring a few particular circumstances such as game misconducts or obvious technical fouls like shooting the puck over the glass.
So do the refs have it out for the Oilers, then?
No, not necessarily.
Here’s the other half of this exercise.
The Rest of the NHL
I also recorded 16 non-Oiler games, trying to catch as many teams around the league as I possibly could.
The teams included in this collection were the Penguins, Avalanche, Ducks, Canucks, Leafs, Canadiens, Senators, Sabres, Jets, Bruins, Hurricanes, Stars, Blues, Flyers, Rangers, Islanders and Kings. Some of these teams played twice in the games I collected. To bring the total number up to an even 16 games to match the Oilers’ data set I also added three Flames games against teams not listed above and randomly switched them up between Teams A and B.
Without a “home” side in this experiment I rotated the teams into one of two slots, Team A and Team B. Some teams were recorded twice and could have slotted into Team A for one game and Team B for another.
Here is what I found.
The close range in penalty calls made and uncalled infractions committed is fairly evident. Remember, I’m separating teams into A and B entirely by chance, so there is some wobble within a narrow margin, as one would expect within a league with so much parity.
There was little difference in type 1 technical infractions committed by either team.
The three categories of uncalled infractions of type 1 technical, type 2 technical and type 2 physical all run virtually in lockstep, and yet the rates of call for each vary slightly from one team to another. Not significantly, mind you, but enough to show the element of human judgment in the process of calling, or not calling, a penalty. You’ll notice that the rates steadily climb, from 2% for a weak physical call to 13% for a fair technical call, with the intermediate types falling in between.
The 50/50 split shown on the right side surprised me. Keep in mind, this is aggregate data, the sum total of everything that I’ve recorded throughout the process. In a moment, we’ll revisit uncalled infractions on a game-by-game basis that will illustrate that games varied but the end result was largely the same.
Finally we have the type 3, or blatant, infractions. The “wobble” back and forth between Team A and Team B is expected and the called/uncalled end up quite close to each other.
However, the percentages only tell us how balanced the calls are from the point of view of the referees, because remember there are several ways of seeing the game. Fans of either team are going to see the same game differently, just as a coach may see something one way and an official may see it another.
Let’s look at the raw numbers to get an idea of how balanced things look from the perspective of the people who commit the fouls, the players.
Here the numbers for uncalled infractions begin to go down once they peak in the type 2 physical category (on the left side) while the numbers for how often they are called go up (on the right side), meaning teams become increasingly likely to be penalized on plays that escalate in severity even if they commit them at a significantly declined rate and even if the infractions at a lower rate are justifiably worth a penalty.
In other words, there is a difference in threshold between what the rule book says is a penalty and what the referees choose to enforce, with them paying lip service to infractions that are worth penalizing but not egregious and then becoming more committed to calling them once they become blatant, albeit at a rate of between one-in-every-four to one-in-every-three times. Players must know this and play their style based on an understanding that this is how the rules are called.
Here you can see that, on a game-by-game basis, one team may greatly outstrip the other when it comes to infractions committed. There is no uniformity to be found within individual games, instead it arises from a longer-term view where we can smooth off the peaks and valleys. Some teams have particular habits (Anaheim and Arizona readily spring to mind) but as a general rule the referees try to keep man-advantage opportunities fair.
This is what we speak of when we say that referees like to keep the chances even for both sides. It doesn’t mean instant karma for every team every night, but over the long term it tends to even out.
Now, these are all details that some may find interesting, but the real point of all of this is to arrive at our next number, which is the average of how often an infraction is called.
Between Team A and Team B, the average rates of call were 17% and 15%, respectively, or about 16% if we draw it right down the middle.
And that rate of call for the Oilers? 17%.
The Oilers are being penalized on just over 4 out of every 25 infractions committed, or roughly a league-average rate.
They are not being picked-on by the refs.
What about the rate of call number for their opponents? That was 13%.
That seems rather low compared to our control group.
Maybe I was mis-recording infractions, seeing every slight against an Oiler as being worthy of a penalty. In that case there should be a greater number of uncalled infractions committed by Oilers’ opponents than I saw when observing non-Oilers teams facing off, right?
Where’s the Bias?
Let’s step aside and take a moment to examine the data and look for any evidence of bias.
When percentages vary like this it is a good idea to look at the raw numbers, because limiting yourself to just the percentages can be misleading.
The total number of uncalled infractions committed by either team over 16 non-Oiler games averages out to 338.5. An average of 21 per game.
The total number of uncalled infractions committed by the opposition over 16 Oilers games was 310. An average of 19 per game.
A difference of 28 uncalled infractions spread over 16 games would come out to me miscalculating by failing to count approximately 1.75 uncalled infractions per game. Given that most games run anywhere from 30 to 70 total infractions by both teams, that difference is minimal.
In other words, if I’m biased in favour of the Oilers in this regard, it accounts for me missing just under 2 uncalled infractions per game out of an average of 50 total infractions.
Therefore, I think we can say that I’m recording uncalled infractions at a regular pace and largely without evidence of bias regardless of whether or not the Oilers are playing.
For interests’ sake, in the 13 Flames’ games I recorded I logged a total of 319 uncalled infractions committed by them, roughly 25 per game. I also found that they were called on approximately 18% of their total infractions (meaning 18% of the time they committed an infraction, they were given a penalty because of it – remember, we’re working off a league-average rate of 16%). Given that the Flames are one of the most penalized teams in the league (per SportingCharts.com), finding them committing more infractions overall then having only a marginally higher rate of call would seem to follow logically and, I believe, lends itself to the argument that this data was recorded with relatively little bias and is partially why I have come to the conclusion that the Wideman Curse is to a greater degree the result of the Flames playing a less-disciplined style than a conspiracy by league officials. If you’re interested in reading my results for the Flames you can do so here.
So we’ve established that the Oilers are penalized at around 17%, that the league-average in the games I observed ranged from 17% to 15%, depending on the team, and that the Flames were penalized at a rate of roughly 18%.
We’re talking a fairly small difference between those three numbers.
But we’ve still got that 13% rate of call that the Oilers’ opponents recorded as an outlier, what about that?
At first I thought this was because I was seeing every little incident against an Oiler player as being penalty-worthy. That’s what one would assume if we were looking for bias in my work, right?
If that were the case, and given that I logged more Oilers games than I did for either the Flames or the NHL alone, we would find evidence of that by looking at the raw number of infractions. Presumably, we would find that I’d observed a higher-than-normal number of infractions committed against my beloved team and had thus tipped the scales, correct?
Let’s return to those raw numbers of infractions again. Here are the raw numbers on the amount of called and uncalled infractions within the overall project.
The consistency of the numbers from the NHL and Flames supports the position that the Oilers’ opponents are, in fact, recording around 310 uncalled infractions through those 16 games, or roughly 19 per game, as it falls well within the parameters of the other control groups.
Is it then realistic that the Oilers are committing half as many uncalled infractions as their opponents or even the majority of the league?
Returning to SportingCharts, at the time of this writing the Oilers were averaging around 2.75 penalties per game, 8th lowest in the NHL and in the same range as the Wild, Islanders, Rangers, Blue Jackets and Canucks.
Given that we’ve established that they are being penalized at around the league average rate per infraction committed (17%), and conceding that I don’t have a great amount of data on the uncalled infractions by those other teams listed above, I believe we can at least say that the results are within a reasonable range of confidence.
In other words, what I’ve got so far looks close enough with and without the Oilers in the picture that I think we’re on the right path.
Therefore, I think it is fair to say what we have established thus far is that…
…the Oilers committed fewer infractions per game but were penalized at the same rate as their opponent who, on average, committed twice as many infractions over the same time.
…the Oilers being picked on by the refs is a non-starter. That they aren’t getting the calls that arguably they ought to is where we need to look.
…the Oilers averaged 2.9 power plays per game this season, 23rd in the NHL and that this number is relatively low when contrasted with the number of infractions . (Remember those Flames for whom the refs have such a grudge because of Dennis Wideman? They averaged 3.11, placing them 10th in the league).
…if the Oilers’ opposition were penalized at the league-average rate of 17% they’d go from averaging 2.9 power play opportunities per game to roughly 3.8, well clear of the league lead which is 3.4, and which would work out to roughly 1 extra power play per game.
Conclusions to draw from this?
The Oilers were penalized at the same rate as the rest of the league, and the variance between teams in terms of how often they were called was minimal, regardless of how often infractions were committed.
There is a correlation between the severity or blatancy of an infraction and the likelihood of it being called.
A type 1 infraction (weak and preferably not called, but technically fits the rule book description) has about a 4% chance of being called but makes up 23% of the total infractions committed if we combine technical and physical infractions. If I were looking for a category to hunt for the so-called “make-up calls”, this would be it.
Type 2 physical infractions (fair penalties which meet the threshold in the rule book but are also clearly violations) such as elbowing, cross-checking, boarding, slashing and the like have about an 11% chance of being called and makes up 18% of the total infractions committed.
Type 2 technical infractions like hooking, holding, interference, holding the stick and so on have roughly a 14% chance of being called and makes up 20% of the overall infractions committed.
Type 3 physical infractions (the severity of which is obvious and blatant, these are penalties whose effect on the target player/play is easily identifiable) have approximately a 30% chance of being called but make up 19% of the infractions committed.
Type 3 technical infractions have a likelihood of being called 27% of the time and make up 20% of the total infractions committed.
That is a clear escalation of your chances of being penalized based on how severe or obvious the infraction is, but the overall ceiling seems to be at about 30%.
Referees seek to balance out power play opportunities in a game, within the limitations of that game (if one team goes completely bananas it can skew the balance against them).
Given much of the discussion during the off-season about a change of style under Chiarelli, that the Oilers were among the least-penalized teams in the league is somewhat surprising. I do wonder about whether their low number of power play opportunities might improve as they begin to find more success or as the spotlight of the league begins to shine more on Connor McDavid (who was a principal target for many of those uncalled infractions).
Conversely, we should watch to see if they begin to commit a greater number of fouls as the team continues to shift into a more physically assertive group and whether that translates into a greater number of penalty kills.
Overall, what I would suggest that fans take away from this information is that the referees have an incredibly difficult job to do – I conducted this exercise with the benefit of time, rewind, and ideal camera angles. Referees are equal parts adjudicators of the league rules and managers of the game as an entertainment product. Despite a few notable and dramatic exceptions, your team, no matter who you cheer for, is not being picked on (individual players may receive less leeway than others, but generally not an entire team). Also, any claims that referees call a particular infraction “all the time” is inherently false and, to be honest, humanly impossible.
Feeling frustration with the officiating is natural and fair at times, but approach with caution both the claims of other teams’ fans and your own that the refs have it in for someone or some team.
I’ll be posting all of the raw data below, and following up with some individual player queries (because I’m sure everyone wants to know about the “McDavid Effect” as it relates to penalties drawn and penalties not called).
Thank you for reading. If you have any questions I’m happy to answer them via twitter @codexrex.
|Missed or Called||Severity (0, 1, 2, 3)||Physical or Technical||Type||By #||Against #||By #||Against #||Time|