Beyond the stats: Dissecting rookie QB competitions in the preseason

It looks like Kevin Kolb and EJ Manuel will battle for the starting quarterback position at training camp and into the preseason. The first round draft pick won’t be anointed the starter and savior of the franchise without earning his coach’s trust. How well would Manuel have to play to win the job?

Since 2003, 29 rookie quarterbacks became the prominent starter for their franchise. 21 of those quarterbacks stated ten or more games in their rookie seasons. Their success levels vary quite a bit, from vaulting their teams into the playoffs like Matt Ryan and Robert Griffin III, to being late bloomers like Alex Smith, to being Blaine Gabbert, Jimmy Clausen, or Trent Edwards. You don't have to be in living in a Sports Man Cave to know that just because you win the starting gig as a rookie doesn’t necessarily mean you’ll be great…or terrible.

Being a Sports Geek, I took a step further, and preseason performance (as measured by adjusted yards per attempt or AY/A) doesn’t necessarily translate to the regular season. There’s just no correlation, as you can see in the graph below.

Some interesting notes on this distribution:

• The distribution’s correlation is 0.30, indicating that preseason AY/A isn’t very predictive of regular season AY/A.
• The average quarterback had 0.54 fewer AY/A in the regular season than in the preseason.
• Andy Dalton had the greatest improvement from preseason to regular season (2.69 AY/A).
• Mark Sanchez had the greatest decline once entering the regular season (4.90 AY/A).

After digging deeper into the data, I was able to find one interesting relationship: the preseason net touchdowns and interceptions (passing TDs – interceptions) grossed up to a full season’s worth of attempts and the quarterback’s regular season net touchdowns and interceptions (referred to by TD-INT from here).

But the only meaningful regression occurred once the data was filtered down to players who started at least thirteen games and weren’t extreme outliers (Andy Dalton and Ben Roethlisberger were better than they initially showed, while Byron Leftwich and Mark Sanchez were worse than they seemed at first). The distribution is below, with the best-fit line in black and one standard deviation above and below the mean in red.

But what does that really tell us? If the rookie wins the competition and starts at least thirteen games, there has historically been a 75% chance that his TD-INT in the preseason will relate to his regular season TD-INT in a similar way to this distribution.

But that’s pretty backwards looking. Maybe winners of quarterback competitions have something in common.

Preseason play during true quarterback competitions usually determines who wins the starting job. I counted nine quarterback controversies, whether they were media generated or not, in the past two seasons. Of those controversies (or competitions), only two starters (winner of the quarterback competition) had fewer pass attempts than the backups.

Mark Sanchez had one less pass attempt than Tim Tebow last season, while Jay Cutler attempted twenty fewer passes than Caleb Hanie in Chicago in 2011. Both eventual starters were much more efficient than their competitors, however. Sanchez and Cutler outperformed the future backup by 1.84 and 1.53 AY/A, respectively.

And that brings us to looking beyond simple pass attempts, as six of the nine quarterback competition winners had better AY/A than the future backup. Both of these findings are relatively straightforward though, since the better quarterback in preseason usually wins the stating gig if there’s a competition.

Manuel will be need to beat out Kolb in the preseason. The benchmarks he’ll have to beat are in the table below, which are Kolb’s preseason averages over the past three seasons.

The bar isn’t terribly high. Blaine Gabbert outperformed that three year average last season. Kyle Orton in 2011 blows Kolb and Gabbert out of the water, and he eventually lost his job to Tebow and is the current backup in Dallas.

We may very well see EJ as the opening day starter. That’d be neat. Keep notes of his performance throughout training camp and preseason, because we now have an idea of how good he needs to be for the EJ3 era to begin.