The pitfalls of comparing quarterbacks

Our Bloguin brother Nate from the Colts blog 18to88 just wrote an excellent post comparing Ben Roethlisberger and Peyton Manning. He focus on sack rate and the money line is: “Roethlisberger has taken more sacks in his career than Manning has in his, despite the fact that Manning has thrown TWICE as many passes.” Nate also dives into their playoff and Super Bowl numbers, where Manning again has the edge. 

It’s an interesting read and I gave my two cents in the comments section over there. Nate is one of the finest bloggers I know and he certainly knows more about football than I probably ever will, but I find these types of comparisons flawed on a number of levels.  

More so than any other major sport, football is a team game with a very large number of variables effecting every game and every play. This is especially true when comparing quarterbacks. After the jump, I’ll dive into several specific issues that make it extremely difficult to statistically compare any 2 quarterbacks.

Weather
This is a big one, and a topic that I have harped on over and over again since I started blogging. Quarterbacks in the AFC North and AFC East divisions have a huge disadvantage when it comes to throwing the football. The Steelers have 11 guaranteed games each year: 8 in Pittsburgh and one each in Cleveland, Baltimore and Cincinnati.  The Saints, by comparison, play 8 games a year in the Superdome, 1 in the Georgia Dome and one each in Charlotte and Tampa Bay. Every year, Drew Brees will play over half of his games in domed stadiums and nearly all in warm weather. The same is true for Peyton Manning and the Colts. The Steelers, on the other hand, might go all year without playing indoors. 

This might seem like a marginal point, but I think there is a huge advantage for passing offenses that play in domes compared to those that might play 3-4 games a year in 30 degree weather. It doesn’t make sense for the Steelers to throw the ball 40+ times a game because of the elements. It’s not a coincidence that the AFC North teams are known for their running game and stout defenses. It’s because of the weather. 

In my opinion, this is one of the biggest reasons why Roethlisberger will never be a statistically ‘elite’ quarterback. It also makes the numbers Tom Brady puts up even more impressive to me.

Teammates
In baseball, advanced statistics are a lot more applicable because the game is a series of one-on-one matchups repeated over and over again. Even variables like ballparks, weather and opposing pitchers can be boiled down to a formula after enough data is collected. Teammates play more of a role in the NHL and NBA, but, going back to what I said above, all of those games are played on the same surface which at least makes it a little easier to compare 2 players. 

For quarterbacks, every single player on the offense is going to have an obvious impact on their numbers. Peyton Manning spent the majority of his career throwing to Marvin Harrison and Reggie Wayne. Ben Roethlisberger hasn’t had slouches for receivers, by any means, but he has never had guys with that much talent at his disposal over a long period of time. On the other hand, Roethlisberger has had an above-average running game to work with while Manning has had virtually no help in that department. Defenses can key on stopping the pass against the Colts, which makes his job more difficult.

Defense and special teams are going to play a role in determining a quarterbacks numbers too. For the Steelers, they’ve consistently had one of the league’s top defenses for the last decade. Any offensive coordinator with half a brain is not going to throw the ball all over the field no matter who is quarterback is when they is a defense that good behind him. Many of the quarterbacks who put up gaudy numbers have had suspect defenses forcing them into passing the ball a bunch of times (Schaub, Manning, Brees, etc.)

Because of these factors, Manning attempted just over 42 passes per game last season. Roethlisberger averaged just over 32 attempts. Add in the weather factor and it is obvious who will look better on the stat sheet most nights.

Play calling/offensive system
Every team operates differently. The most successful offenses in recent years have relied on short, timing-based passing (Packers, Patriots, Colts, Saints). The Steelers, wisely or not, have went the other way and relied on a running game coupled with a downfield passing attack. Roethlisberger threw the ball deep (15+ yards) more than any other starting QB in the league last season. Going into 5 or 7-step drops is going to lead to a lot more sacks versus somebody like Manning who plays in an offense that is designed for the quarterback to typically get rid of the ball quickly. 

Each team’s philosophy is a little different and will also have a large effect on the quarterback’s stats. 

****

So how can we accurately compare two quarterbacks? I definitely think statistics are useful, but only to a point. The best way, for me at least, is still the ‘eye test.’ I learn more about a quarterback from actually watching him play than from crunching numbers in a spreadsheet. After living in Indianapolis for 4 years and watching Manning play almost every Sunday, he is in a different stratosphere than Roethlisberger when it comes to throwing the football. He makes throws that nobody else can make. 

But that doesn’t make Roethlisberger much less of a quarterback at all. I’m not sure any other QB in the league would be as good of a fit for the Steelers. Manning might struggle in Bruce Arians system or playing often in bad weather. There’s no way to know for sure. 

At the end of the day, both guys are amazing quarterbacks and both might end up in the Hall of Fame. But when comparing the two, it simply isn’t fair to just look at the numbers. There are a million different factors that determine their sack rate, interceptions, Super Bowls, etc. There are too many factors at play. Watch Peyton Manning or Ben Roethlisberger play football and you will see how uniquely gifted and skilled each of them are.