Yet another site with Draft Analysis. Who are we to trust? We think we have an answer.
The answer is to TRUST NO ONE. So why do we bother with all of this draft analysis if it cannot be trusted? The answer is that in time it will be trusted because we will verify the accuracy and skill level of the analysts doing the work.
A meteorologist friend of mine has a quote: “If you aren’t verifying then you aren’t forecasting.” He is absolutely correct. What it means is that we have snippets of draft commentary from Wonder with his Top 5 at each position from last year’s draft, but how do we really go back and grade him on how well his forecasts turned out? Yes, you have to wait 3 years, and ideally 5 years later to truly see how he did. But this entire process (up until 2010) can arguably be summed up with two words:
MORE NOISE.
This site is dedicated to providing meaningful insight and REDUCING NOISE. So while we naively thought that by merely giving draft analysis it would be good enough and you would simply ‘know’ that there was value, we were kidding ourselves. All it is is more noise. We need to verify.
I can cherry pick all I want. I can tell you that Wonder nailed Austin Collie, Cliff Avril, Louis Murphy and Andre Smith. But he also had Vernon Gholston rated near the top in 2008, so how do we really know what value Wonder (or Pete, or any of the commenters here on the blog) is adding to our information we seek on the draft? The answer is that we do not, because it is really just a strobe… a flash of light that goes on and then off until we wait again for it to give us another fleeting flash of light. That is not good enough.
The amount of noise reduction is going to be completely dependent on the amount of verification we seek.
The draft project (and its associated competition) is reverse engineering the problem to get the solution. This weekend I happened to be in a Barnes and Noble and saw a slew of 2010 NFL Draft Periodicals. One was worse than the next. NONE OF THEM VERIFIED THEIR PREVIOUS WORK. I did see one of them look back and verify the work of the 32 teams’ General Managers!
LOL! They checked the pickers but they did not check to see how their own work was doing! That team verification was one page and the other 80 pages were about what they thought about all of these players. How appropriate and telling- if they hand you the same regurgitated reporting year after year, you spend $8 and will never know how they do. This player got a 7.8, this one got an 8.0. What does that mean? 3 years from now it will mean nothing. Does 8 mean he goes to the Pro Bowl and 7.8 means he does not?
I rarely see these magazines with that much variance on their Top 30, which is another red flag. Shouldn’t the ‘sheep’ lemmings uniformity of these predictions scare you? When was the last time you saw Kiper or McShay tell you he would not pick a consensus first rounder until Round 3 or 4? One thing you do not get here is lemming forecasting. Last season, for example, Wonder did not even have Andre Smith worth a first round pick. (He was picked 6th in the entire draft by the Bengals.) This season once again we’ll be ripping apart some of the consensus. But it means nothing without verification. We’ll be verifying the calls with this system.
What about the predictions after early rounds? To quote Pete, it is the later rounds where you earn your bacon. And there is not an ounce of reporting on how well these periodicals do with regard to the unearthing of the true sleepers. What is their success rate? At least with this verification process we will be able to tell you how well we are doing. We’ll see what the accuracy is at every level of the draft.
This weekend, Wonder and I worked on transcribing ~8 positions into the following format:
Ranking (1-200)..PlayerName..PlayerPosition..Rating(1-7)..Comment
The Rating is of course the one we’ll be verifying most closely. It is in integer format, meaning no hedging for whether a player is a Pro Bowl player (2) or just a solid starter (3) by giving a 2.5. On more than a few players I have made Wonder a little uncomfortable by forcing him to pick. That is a good sign that the process is making him completely transparent on EVERYONE. This is precisely why so many analysts do not get verified, why we will and why this will reduce the noise for all of those out there who want to know who knows what they are talking about.
I spoke with another team of drafting scouts at NFLHOUSE. They really like the idea of this draft verification process because they are asked about what separates them and they do not have the answer quantified. They have a Top 100, they actually rank the Top 250 internally and they are going to put in ratings with the 1-7 projected NFL endpoint we are using.
Reminder- the deadline for submission of your analysis is March 30th, and I have received NO entries. Are you willing to be subjected to this process? We are all right and wrong to one degree or another- but now we are going to find out.
Add The Sports Daily to your Google News Feed!
