Answering Someone Else’s Question

Yesterday, BBS over at Stampede Blue asked if the Colts would really have finished 6-10 if they had a different quarterback.  The thought has been echoed by many, most notably Eli Manning in a recent interview.  As always, there is no way to resolve such debates, but the numbers can be of some help.  In reality, the question can only be solved if we define the debate more clearly.  Saying “the Colts would be 6-10 with a different quarterback” requires some clarification.  There are at least three different scenarios that can be considered.

1.  The Colts are so bad, and Peyton Manning is so good that any other quarterback would win only six games with this team.

This statement is impossible to prove conclusively, but I think we can safely say it is false.  Consider the 13 win San Diego Chargers.  According to DVOA, the Colts had a better defense and a better run game by a wide margin.  The Chargers had better special teams play, but only slightly.  I don’t think anyone would argue that the Chargers receivers are dramatically better than the Colts, and even if you think Gates is better than Clark, the gap isn’t a chasm.  Phillip Rivers led the Chargers to 13  wins with essentially an inferior team.  So, with some certainty, we can agree that Manning is not the ONLY quarterback in football who could have coaxed the Colts into the playoffs.

2.  If the Colts didn’t have Peyton Manning, they would win no more than 6 games with Curtis Painter.

Frankly, I’m not sure that’s what people mean when they propose the question, but I don’t doubt the statement is true.  In fact, I don’t think they would win even one game with Curtis Painter.  However, no GM in his right mind would ever enter a season with a 6th round rookie draft pick as his #1 QB option, so if that is the comparison people are making, it’s a false one.

3.  With an average quarterback, the Colts would win only 6 games.

This is something we can examine.  For our purposes, we’ll look at the Jacksonville Jaguars and the Buffalo Bills.  The Jags played almost an identical schedule to the Colts and won 7 games, but their “expected wins” was only 5.5.  Seriously, the Jags OVERPERFORMED this year.  Wow.

According to DVOA, David Garrard was the most ‘average’ quarterback in football last year.  In essence, if you switched Garrard for Manning, how would the Colts fare?

Our second team actually won exactly 6 games last year.  The Bills are an odd case because they featured good play at most positions on the field, but had HORRIBLE quarterback play all year.

First, let’s look at the three teams defense and Special Teams.  For defenses, a negative DVOA is good, for special teams a positive number is good.

We can plainly see that the Colts were an inferior defensive team to the Bills.  Even given the fact that Indy benched starters for three games and Buffalo plays in the snow (which hurts offense), it’s important to note that the Colts defense has the advantage of playing with a good offense that generates better field position for them.  However, Indy did have a better defense than Jacksonville did.

After the defense, we look at the offense.  This is obviously pure conjecture because the quarterback should help or hurt the rest of the offense, but the raw numbers will give us a starting point.  First the running games and offensive lines

Both 6-10 teams had vastly superior running games to Indianapolis, which featured some of the worst run blocking in the league.  The big difference between the teams comes in pass protection.  The Colts had the best adjusted sack rate, and the Bills had the worse with the Jags not far behind.  We know that one of Manning’s great strengths is his quick release and ability to avoid sacks.  With Curtis Painter under center (playing with some reserves), the Colts O line had a sack rate of 10.7% which would be the worst in football over a full season.  Again, some of that is on the quarterback.  Those of us who watched the Colts offensive line all season are not at all convinced that it was good.  I’m not saying it was as bad at pass blocking as the Buffalo line, but neither do I buy that it was actually the best pass blocking line in football.  Basically, I think we can safely assume the Colts offensive line and running game was no better than either of these other teams, and quite possibly was considerably worse.

It’s difficult to evaluate receivers apart from the quarterback, obviously.  Buffalo did bring in an elite WR in Terrell Owens, but at this point in his career, who can tell if his drop in production was caused by age or the quarterback?  What we can do is try and adjust the number of points the teams scored by the VOA of the quarterbacks. Normally we use DVOA which includes an adjustment for defense, but in this case we want to know how the teams might have fared against the same schedules, so we’ll use the unadjusted VOA number. This isn’t what VOA is designed for, but it might provide some insight.  The Bills fielded two QBs who played almost the same amount and had almost equally awful VOAs. We’ll give them the DVOA for Trent Edwards who was supposed to be the starter going into the season, but in the end it doesn’t make a big difference.

Ultimately, this is a bit of a math game, and I admit I’m doing some things with the numbers that perhaps they were not designed for. Still, it gives us an idea about how much Peyton Manning means.  Without him, the 2009 Colts offense was basically the same as the 2009 Jags.  Indy has better skill players at WR and TE, so that would roughly offset the superiority of the Jags run game. The Bills clearly would have been a better team with an average quarterback.

If the Colts had gone into the season with substandard QBs like Fitzpatrick and Edwards, 6-10 would have been a good season.  With a completely average QB (which Garrard is the definition of both statistically and to the naked eye), Indy would have allowed 307 points and scored around 290.  Statistically, that leads to the Colts being between a 7 and 8 win team.

So, this doesn’t prove anything, obviously, but it’s food for thought.  Ultimately your answer to the question of how many games the Colts would have won without Manning depends on your opinion of the O line as pass blockers and if you think the Colts WRs are better than the run game is bad.  Personally, I think 7 or 8 wins with David Garrard sounds about right. With the Buffalo quarterbacks, 4-5 wins would probably be high.

UPDATE:  The more I think about it, the less I like my methodology.  Let’s alter the Adjusted Points graph to reflect the percentage of run/pass play.  Then, I’ll adjust the points down based on an average QB for just the passing downs.

Using this system the Colts fare a lot better without Manning than under the last system.  Granted, this isn’t perfect either, because we should assume the QBs do aid or suppress the run game.  How bad would the Colts run game be without the threat of Manning going deep?  It’s impossible to say.  This is just another lens through which we can view the problem  Under this system, the Colts average about 8.4 wins, so 9-7 would be more reasonable.  Again, it proves nothing, but might be a more accurate way to view the issue.