Speakin’ My Piece: Evaluating Scouting in Isolation

<![CDATA[[youtube http://www.youtube.com/watch?v=qdlRXIX-T9U&w=420&h=315]

One of my hobby horses is the idea that we do a poor job of distinguishing the scouting of hockey players from their development and management. These efforts are all inter-related and all combine to work toward the ultimate goal of achieving NHL success. This is a given.

However, I'm often struck by a persistent lack of nuance in evaluating scouting departments that fails to account for any and/or all of the following: round; years since draft; development path; management decisions; player preferences; injury; etc.

Any of these things can account for a player and team's failure to produce a viable NHL player. They do not, however, necessarily tell us that the scouting department has failed. And, if I'm a GM tasked with evaluating a specific employee (in this case a scout), I want to be sure I'm not confusing a general failure of a player to pan out with a specific scout's failure to define talent.

Consider this analogy: we often have to remind people to separate the aura of team success/failure with player success/failure. How many middling talents gain the veneer of greatness while playing on a Stanley Cup winner? How many good players get tarnished with the failures of their bottom of the league teams.

In an effort to sharpen the way we talk about scouting, I'd like to suggest something of a working definition followed by an example that I think is instructive.

Perhaps we can define scouting as the ability to identify and project professional talent

If a player fails due to a lack of talent to have an impact as a hockey player, that’s clearly a failure of the scout (relative to the draft round and age of the player).

However, for a variety of development, management, injury and player preference reasons (that scouts can’t control for), players often end up in professional leagues other than the NHL. Often, these are players taken in the mid-to-late rounds. As a GM, it is valuable evaluative information for me to know whether my scout has identified a player who goes on to enjoy success in another professional league.

This, however, doesn’t appear to be a view shared by others.

(click to embiggen all photos)

(from lowetide’s place yesterday)

So, allow me to elaborate my position.

1. A scout’s job is to evaluate and project talent. That is what they can reasonably control for.

2. A GM’s job (and our job as commentators on scouting) is to try and isolate for what scouts can control for when evaluating them.

3. Our definition of successfully identifying talent (esp. concerning mid-to-late round picks) ought to include alternative to NHL pro leagues.

4. A drafted player with a successful career in the KHL, SHL, etc. is a good sign that a scout has identified talent, regardless of which league the player plays for.

None of which is to say that these leagues (KHL, SHL, Liga, AHL, ECHL, etc.) ought to be treated equally, or that they should be treated as on par with the NHL.

The point is, a player with a successful career in the KHL is clearly a better talent than a player who falls out of the minor pro North American leagues. Our assessment of scouting (esp. for mid-to-late round picks) should acknowledge this.