Research Room: Judging the star quality of the Pirates’ OF

For Pittsburgh Pirates fans who genuinely enjoy the incredible about of data and statistics involved in baseball, the Statcast era was supposed to be a golden age.

With Statcast, stat geeks like myself, would finally be able to quantify things like a fielder’s ability to range for baseballs, a baserunner’s ability to get a jump on a pitcher, and all sorts of other goings on around the field that were previously left up to the “eye-test” to judge a player.

Unfortunately, MLB keeps much of that more advanced data out of the public domain, reserving it for itself and the teams in the league.

While most of this information is proprietary, the MLB does give us a little bit of data on outfielder’s catching ability in the form of a five star rating system. Batted balls that have a catch probability of 0-25% are rated as 5-star catches or a “Top 10” type catch. 26-50% probability are 4-star catches, 51-75% are 3-stars, 76-90% 2 stars, and finally 91-95% are 1 star catches.

The catch probability is based on the time between when the ball is released from the pitcher’s hand and when it either lands or is projected to land on the field, as well as the distance between the fielder’s starting position and the point where the ball is projected to land.

Although these divisions in “stars” are somewhat arbitrary, (meaning they could just as easily been broken up into 5 equal groupings of 20% catch probabilities or had 10 different star divisions or whatever else), this information allows us to set up an interesting fielding statistic that judges a player’s ability to range for balls based on how many catches in each “star” interval they made.

Here is the Statcast “Catch Probability Leaderboard”; it gives us the player’s name with how many outs and opportunities a player has had at each of the 5 different star divisions. From this we can set up a statistic where we weigh each of these catches on their “Star Rating”, similar to how Slugging% weighs hits by the number of bases collected.

Here’s what our Star Rating (SR for short) formula looks like:

Research Room: Judging the star quality of the Pirates' OF

So what Star Rating really measures is an outfielder’s ability to catch balls hit to him while giving more weight to more difficult catches. This makes it a more informative statistic over something like Fielding% because it allows us to give more weight to fielders that can get to more difficult balls. This is similar to how Slugging% is more informative than Batting Average, because it weighs more productive hitting to a greater extent.

Comparing some Pittsburgh Pirates OFs to others

For instance in 2016 center fielders Billy Hamilton and Andrew McCutchen had similar Fielding% of about .990, Hamilton, however, was able to get to a lot of difficult balls that McCutchen was not able to, and so Hamilton had 1.87 SR and McCutchen’s was only 0.97 SR.

The idea behind subtracting out Errors is that a player that makes spectacular catches but trades them off for poor fielding otherwise should be penalized in some fashion. Originally I planned on giving Errors a weight equal to the star value of the average fielding opportunity, but the coefficient came out to 1.047 which was close enough to 1 to just assume it for simplicity’s sake.

[perfectpullquote align=”right” cite=”” link=”” color=”” class=”” size=””]While most of this information is proprietary, the MLB does give us a little bit of data on outfielder’s catching ability[/perfectpullquote]

The one thing that this metric does not account for is differences in a player’s arm. Different websites have made different metrics surrounding an outfielder’s throwing ability, however, these metrics do not use statcast information. Additionally the websites don’t release their methods for compiling these statistics, so for consistency’s sake, and until the MLB releases statcast information on outfielder’s throwing ability, we can only use FSR to judge the fielding ability of players, not all aspects of their defense.

If we look at league wide Star Rating averages for ‘15, ‘16, and ‘17 we see a trend, in 2015 the league SR was 1.29, 1.17 SR in 2016 and so far in 2017 it is 1.11. I would guess this downward trend is because better defensive shifting of fielders is reducing the number of balls being hit to difficult areas. If we look at the numbers we get an even stronger sense of this.

In 2015 batted balls with less than a 50% chance of being caught (aka 4 and 5 star catch opportunities) made up 41% of all fielding opportunities for outfielders, in 2016 that dropped to 37%, and to date in 2017 it’s dropped again to 35%; whereas fielding opportunities with a better than 50% chance of being caught have risen accordingly from 59% to 63% to 65%, respectively. This means either batters just so happen to be hitting more balls directly at the outfielders or, more likely, the outfields in baseball are using defensive shifts to limit the number of hard-to-get-to balls which improves their chances of making a catch on any given ball.

What this means is that we can’t really compare Star Ratings year over year because each year the number of opportunities at each catch difficulty are fluctuating. The fix for this is easy; if we adjust each player’s Star Rating to the league average Star Rating for that year, we get a more consistent measure. Here’s what the formula for a new statistic called Fielder Star Rating (FSR for short) looks like;

FSR = Star Rating – League Average Star Rating

In addition to allowing us to compare FSRs between two different years, this makes Fielder Stars much more intuitive; if a player’s FSR is positive, the player has been an above average outfielder and if it is negative, the player has been below average in the outfield. A player that has an FSR pretty close to 0 would be about league average.

The Pirates team FSR in 2015 was -0.083, or .083 stars worse than the league average. The 2016 Pirates had a team FSR of -0.051. So far this year the Pirates outfield has a team FSR of -0.046. So interestingly the Pirates have not had an above average fielding outfield in the statcast era, but each of these numbers are pretty close to 0, so they weren’t far off of league average either.

This season, the Pirates outfield has been a regular complaint of fans and commentators alike, despite actually being one of the better fielding outfields, statistically, that the Pirates have had in recent years.

Here’s a list of Pittsburgh Pirates 2017 outfielders, with at least 10 fielding opportunities, ranked by their Fielder Star Rating;

 

Name Fielding Opportunities FSR
Adam Frazier 32 0.488
Starling Marte 39 0.330
Gregory Polanco 69 0.126
Andrew McCutchen 110 -0.124
John Jaso 42 -0.225
Jose Osuna 29 -0.830

 

Not so ideal?

The Pittsburgh Pirates ideal outfield of McCutchen, Marte, and Polanco has a combined FSR of 0.036, putting them slightly above league average. Frazier has been a bright spot when in the outfield, putting together the best outfielding season of the bunch. The other two backup outfielders, Jaso and Osuna, have combined for a terrible -0.472 FSR.

This tells us a few things. For one, the Pirates could not have found a much better fielding replacement for Marte than Frazier. Looking at two of the popular names for possible trade targets for the Pirates, J.D. Martinez (-0.337 FSR) and Jay Bruce (0.245 FSR) were both worse fielders than Frazier this year.

These players would certainly have been offensive upgrades over Frazier, but if that is the measure, they would also have been upgrades over Marte this season; so calling for an offensive upgrade over Frazier is also calling for an upgrade over Marte. Furthermore, Adam Frazier has a .740 OPS, significantly better than Marte’s .680 this season, so Frazier has actually been an upgrade over Marte both offensively and defensively.

The next thing that this  list tells us is that Andrew McCutchen really does need to be moved to right field as he is objectively the worst outfielder between Marte, Polanco, and himself. It isn’t just this year either, in 2015, Cutch had a -0.221 FSR, in 2016 it was -0.196. Compared to Marte and Polanco who have been above average fielders for all 3 years, it becomes clear that McCutchen needs to be moved.

To put a number to it, if McCutchen had the same fielding opportunities as Polanco did in right field, and the same catch rate at the various difficulties as he does in center, he would have a -0.015 FSR, which is pretty close to league average, and a big improvement over his current -0.124 FSR, pretty close to 10 times worse. Additionally, if we make the same assumptions, but put Marte in center, Polanco in left and McCutchen in right, this outfield sees its FSR jump from 0.036 to 0.134, which is a pretty drastic improvement, considering all we’re doing is shuffling around players.

The last thing that this tells us is that, defensively at least, the Pittsburgh Pirates still do need another backup outfielder, beyond Frazier. With the health concerns of Polanco this year, it would be a smart hedge for the 2018 Pirates to have a decent bench bat that can also slot in and play a competent outfield.

[perfectpullquote align=”full” cite=”” link=”” color=”” class=”” size=””]The Pittsburgh Pirates ideal outfield of McCutchen, Marte, and Polanco has a combined FSR of 0.036, putting them slightly above league average.[/perfectpullquote]

John Jaso and Jose Osuna have been a black hole when it comes to fielding; moreover, Jaso has been a pretty bad hitter. Osuna has at least showed that he is a legitimate major league hitter, but has also been the worst outfielder in the league in terms of FSR, meaning that not one other player in the entire league has been worse in the outfield than he has.

Fortunately this hole in the roster seems to have been at least partially filled for next year in the form of Sean Rodriguez. Rodriguez has a .422 FSR as an outfielder since 2015, making him more than a competent outfielder. As Pirates fans learned in 2015 and 2016, Rodriguez can be a productive major league bat coming off the bench.

The 2017 season has been something of a letdown for Pittsburgh Pirates fans at every position, except for First Base, Second Base, and maybe Shortstop.

That being said, the 2018 Pirates seem primed to have one of the better outfields in baseball. Between Marte, Polanco, Frazier, Rodriguez, and McCutchen, both offensively and defensively, this outfield should be producing at a high level.

Using their 2017 numbers, this core of outfielders projects to have a combined 0.084 FSR in 2018, the first year of an above league average FSR the team has had.

If we assume McCutchen will move to right, polanco to left and marte to center that number improves to 0.167 FSR, which would put among the very best outfields in baseball.

Even if McCutchen is traded in the offseason, minor league prospects like Austin Meadows and Jordan Luplow look as though they will be, at the very least, competent major league outfielders.

This is, of course, barring several suspensions, or injury plagued years, some other unforeseeable circumstances, or some combination of the three. Even with one or maybe two of those types of problems arising, next year’s Pirates team does seem equipped to handle it. With the utility player aspect of players like Josh Harrison, Sean Rodriguez, and Adam Frazier, this hedge against major issues like these arising seem to be across the entirety of the Pittsburgh Pirates’ fielding core.

Ultimately this is a reason to be optimistic about the 2018 Pirates team. As unexciting as calling up prospects and waiver trades may be, the Pirates use of these things have set them up with a 2018 outfield that projects to be above average for the first time in the statcast era. Furthermore this depth also extends to the infield, meaning that the Pirates should be able to absorb some of the shocks, that things like injuries can cause to a team, in the 2018 campaign.

If the Pittsburgh Pirates can add a decent 3rd Baseman/2nd Baseman and move Josh Harrison to 3rd, the only weakness I can see is a shaky bullpen. A bullpen which looks like it is finally being solidified with quality relievers from Indianapolis. While the 2017 season has been an enormous disappointment for the Pirates and their fans, the 2018 team is already shaping up to be legit contenders for the NL Central.

Arrow to top