As baseball junkies we are always looking at ways to compare players to one another and see who is better. WAR is a commonly used stat, but I've heard many of us suggest that it is not an accurate way to describe bullpen pitchers. I've spent some time with this and I have decided that this is true. I also have an idea for a new better stat to rate bullpen pitchers.
The Problem with WAR for Relievers
WAR is a statistic that measures wins above replacement that is based on playing time, so for pitchers this is calculated on innings. WAR counts every inning as if it the same, but we know it is much more difficult to get out the heart of the order then the bottom. For starting pitchers this is not a problem because they pitch ~200 IP so they end up facing 800-1000 batters. Also, they typically go through the order a couple times so they face good and bad hitters equally. Relief pitchers, on the other hand, pitch 40-70 IP and will face 200-400 batters per year, but what are the quality of those batters? In addition some pitchers are specialists that only face righties or lefties and some relievers are called on to face any handed batter. Are Pedro Feliciano's 60 innings the same as Brain Sock's 70 innings? Using WAR there is no way to tell. Additionally relievers are at the mercy of their managers to bring them into favorable or unfavorable match ups. There are too many variables to compare relief pitchers innings.
A Potential Solution
There has to be a way to normalize relief pitchers statistics to determine how well they really did in a season. One potential answer to this is to actually measure the quality of the hitters faced. You can measure the OPS vs. handedness for hitters the pitcher faces (I don't know if you would use yearly or monthly stats) and divide the success the batter has versus that pitcher by the previous number (actual OPS vs pitcher/OPS vs handedness of pitcher). This statistic could be used in two ways, first it would tell you which relievers face the best hitters or threw the hardest innings during the season and second it gives you a normalized scale to measure all relievers on. In this statistic numbers lower then 1 would be good and numbers over 1 would be bad.
Is OPS the best stat to use to measure hitter quality?
Should yearly stats be used for the hitter or monthly stats?
Is there a stat already like this that exists?
How could you collect this data , Linux script?
Is it appropriate to take hitter split stats?
How should park effects be taken into account?