One of the things you learn in a basic statistics class is the difference between the kinds of data you can get. While this is rudimentary, it is easy to get them confused when you are looking at fairly complex data. No one really confuses nominal data. These usually take the form of yes or no and tend to be binary in nature. The confusion usually comes when we look at the difference between ordinal data and interval data. This happens when we do something like this. When data isn’t scaled in an identical (or even similar) then the combination becomes ordinal because we simply cannot trust the combination. This is even if each individual source was interval in nature.
Ordinal data is what we use when we want to simply rank one player over another. In our case, we are ranking the ten catchers in the Hall of Fame in terms of fielding. The problem is when we start looking at multiple sources we start looking at wins in some cases and runs in other cases. We are then looking at each one in isolation and standing them next to another that we look at in isolation. When we start combining these we start to get into some serious trouble. This is especially true if we attempt to make any conclusions about the actual value of a player. The differences in value are what we would call interval data. Interval data is the holy grail of data because it not only says who is more valuable than another, but by how much.
When we look at players from different eras and compare their fielding numbers we need to eliminate two words from our vocabulary: greatest and best. Those are conversations best left for the sports bar. We also get into trouble when we start using terms like difficulty and importance. When criticizing fielders, some passionate fans will tend to get defensive and say something like, “well, how much baseball do you play?” or “I’d love to see you go out there and catch.” I’ll admit right now. I never played catcher. When I was a kid I played in the outfield so I am saying nothing about the importance or difficulty of the position. At least I’m not saying it directly.
That being said, the data does say something about the relative importance of the individual skills a catcher must possess. Rating catchers is always more difficult than any other position because they must call the game, block pitches in the dirt, and control the running game of the other team. In the current game we also include pitch framing in that equation. As strategy has changed, the relative importance of all of these skills fluctuates. For instances, controlling the running game was huge throughout the 1960s, 1970s, and 1980s. So, catchers from that period could accrue more value if they were excellent at that particular facet of the game. Conversely, during the live ball era (1920s – early 1940s) hardly anyone stole bases. So, one could be brilliant or not and you wouldn’t see a range of value there.
So, I would have a hard time saying that a Johnny Bench or Gary Carter was a greater fielder than a Gabby Hartnett based on the numbers. Greatness implies that the skills were superior and while that may be true, it is not true based on the numbers. Players are always better understood when they are compared with their own generation. We know more about Hartnett when we compare him with Mickey Cochrane and Bill Dickey then when we compare him with the catchers from the 1970s. Those that saw both could weigh in, but that would be a more qualitative analysis. This becomes increasingly problematic when we include new analysis like pitch framing. The modern catchers become much more understood when we include that, but we can’t include it for the earlier catchers. While we have play by play data going back to the beginning of the century, that play by play data won’t tell us if an individual pitch should have been called a strike or a ball if the catcher were better at framing it.
You undoubtedly did not open this up to hear my caveats about fielding, but it represents the very best reason why I am not comfortable naming a best catcher of all-time. At least, I’m not using the data here to define it. Give me a beer and a good game on television and I’ll throw down on any player, but I’m leaving the charts and graphs at home. We will split the fielding data into two categories. First, we will look at the major career data for our three sources. Keep in mind, since some use wins while other uses runs we will not combine them. Our WAR and WS from the previous article has already embedded that in their totals. Here, we are distilling out the numbers to get a sense as to who was the most valuable fielder of all-time.
The above represents my best attempt to normalize the data. The win shares we simply divided by five to make it roughly equal to defensive WAR from baseball-reference. Unfortunately, I struggled to normalize the total zone runs from Fangraphs because it would change the rankings. Typically, we would add ten runs per season to the totals because that would convert from a comparison against the average to a comparison with the replacement level player. In the end, the desire to present the data as those sites/sources wanted ended up winning out.
However, this is where our discussion of ordinal versus interval data comes in. The above numbers still don’t make a whole lot of sense when you put them together because all three sources have considerably different opinions of how much value individual events (or skills) have. Those change between eras as they change between platforms. However, if we show the same table and replace the win totals with their simple rankings in those systems we see something quite remarkable
I would surmise that we are actually seeing more variance here than at any other position. Different sources treat each individual skill a catcher must possess differently and with the different weights can come wildly different values. Still, only two players are more than two places apart across the board. When win shares is dropped that moves to zero. Four of the players are exactly the same including the most valuable and least valuable.
That being said, this is one of the areas where we have to be careful. Notice how Carlton Fisk drops when compared to the average as opposed to replacement level. He caught for over 20 years, so he managed to accumulate a lot of value. That’s not the same thing as being the greatest or the best, but it is important to note. This is especially true when we start looking at things like Gold Gloves, Win Share awards, and total zone awards. Lacking in those awards does not mean you lack value. It just means you were never the best in any one particular season.
Moreover, this distinction can be seen more at the bottom of the scale than the top. Mike Piazza was universally regarded as a poor defensive catcher (although he rated highly in pitch framing), but depending on the source he still had value when you compared him to the replacement level catcher. This is where we include the remarks of our critics when they say, “I’d like to see you go out there and do it.” Why yes, we aren’t saying any of these guys were a complete buffoon. It’s all comparative in nature and unfortunately you are stacking each up against the very best to play the position. Some aren’t going to come out looking good in every single category.
All that being said, we will finish our journey by looking at individual season honors. We do have some limitations here. Total Zone awards aren’t an official honor, but we only have data going back to 1953. The Gold Glove awards were awarded in both leagues for the first time in 1958. So, some of our players will not have any chance to win anything and others only a limited opportunity. However, win shares awards were added back through all of our players careers. When all things are considered, these are fairly meaningless, but I know many of you will be interested, so we close with this table.
Win shares is the great equalizer thanks to James’ hard work in breaking down the data for us season by season. When we see how evenly distributed the awards go we can see that greatness is really more evenly distributed than the basic numbers would indicate. It definitely is more evenly distributed than the Gold Gloves themselves. However, we return to our original caveat: greatness and value are two entirely different things. Just because Carlton Fisk won no awards doesn’t mean he has zero value. It’s just that he had the misfortune of catching at the same time as Johnny Bench, Gary Carter, Bob Boone, Jim Sundberg, and Ivan Rodriguez at the end of his career.