Selective Application Of Statistics

by James Kannengieser on Dec 24, 2009 6:00 AM EST

A common refrain of those who do not like their baseball analysis inundated with statistics is something along these lines:

There are three kinds of lies: lies, damned lies, and statistics.

This is usually uttered by those unwilling to let statistics disprove perceptions. In general, this is a silly statement. Statistics exist to measure whatever it is they set out to measure and nothing more. Numbers alone cannot deceive, as long as the benefits and limitations of such numbers are understood. Stats like wOBA and UZR are frequently cited here, but only because the writers and most members of the community comprehend the context neutral linear weights concept behind wOBA and the sample size limitations of UZR. Criticizing statistics without fully understanding their strengths and weaknesses happens more than it should, unfortunately, and is a major pet peeve. That said, it is true that statistics can be used by people to unfairly support biased arguments.

A post about Mike Cameron at Metsmerized provided an example of the thought that stats are sometimes used to bolster pre-conceived biased positions:

I think that more and more people would accept advanced metrics if the stats weren’t used so often to strengthen just one side of an argument.

I’m not knocking advanced metrics, but I do find fault with them when they are used to make unfair comparisons.

The point here seems to be that advanced metrics are used as a weapon to trumpet players like Cameron who, atleast in the eyes of the MMO writer, isn't as good as Fangraphs suggests. This couldn't be further from the truth. Advanced stats are cited only because the methodologies behind their calculation have been investigated, questioned and eventually accepted by the best minds in the sabermetric community. If I invented a stat and continued citing it despite being discredited by Tom Tango or Colin Wyers, my credibility would be shot. WAR, UZR, Plus/Minus, etc., have been scrutinized by the likes of The Hardball Times and Statistically Speaking and been accepted as worthwhile metrics. It's not some nonsense created with the intention of making money or fooling people. The other point about the inability to compare the values of players who play different positions is also invalid, as posts like this one by Mark illustrate.

I like to think I'm generally unbiased in any analysis I provide so coming up with an example of my own selective application of statistics is impossible. Endy Chavez is one of my favorite players but I wouldn't project a .350 wOBA as part of an argument in favor of signing him. However, I'd like to offer an example of how even an enlightened writer might appear to be selectively using numbers to agree with an agenda. Howard Megdal of SNY, MLBTR and about three dozen other outlets is a well known Oliver Perez fanatic (just look at the Baseball-Reference sponsor for the RMS ~~Titanic~~ Perez). Unsurprisingly, he has often provided opinion on Derek Lowe, a pitcher the Mets reportedly pursued last offseason before eventually settling on Perez. In a post at MLBTradeRumors from October about Lowe being available in a trade, Howard wrote:

Lowe is coming off of a season with a 4.67 ERA along with a strikeout rate of just 5.1 per nine innings. He certainly didn't finish strong, with a 5.05 second-half ERA, and a 6,23 [sic] mark from September 1 on.

ERA isn't my pitching statistic of choice but sure, Lowe had a somewhat disappointing season. At his debate site, the recommended The Perpetual Post, Howard participated in a Lowe vs. Perez discussion in November:

I think it represents the likelihood that Lowe, fresh off of an 88 ERA+, is likely to be around that or below it for the remainder of his three years and $45 million.

A healthy Perez has been considerably better than the 2009 edition of Derek Lowe. Time and reality are converging to bring Derek Lowe’s career to an unceremonious close. It isn’t time to declare Atlanta the winner in the choice of Lowe over Perez just yet.

Again, ERA+ isn't a great metric for evaluating a pitcher's performance but I can live with it. That is until reading this piece by Howard at SNY about Mike Pelfrey:

Pelfrey had a 3.72 ERA last year, while even Sunday's stellar performance only lowered his 2009 ERA to 4.83. But let's take a closer look.

Fangraphs has his FIP at 3.96 last year, 4.18 this year -- a negligible difference in performance over the two seasons.

Wait a second, why is Pelfrey afforded the FIP treatment but Lowe isn't? Lowe's 2009 FIP was 4.06, significantly better than his ERA. He was a victim of an inflated BABIP and subpar defense backing him up, much like Pelfrey. Why not consistently utilize stats for player evaluation? Having read Howard's work for some time now, I doubt he consciously wanted to unfairly paint a negative picture of Lowe. However, this is an example of how a known affinity for one player over another, and subsequent assessment of those players, might cause some readers to question a writer's intentions. In the case of Cameron and Jason Bay, reliable statistics show that the gap between them isn't as large as many think. That this doesn't jibe with most fans' perceptions of the two players has little bearing on whether it is true. Rational people can have differing opinions. Those opinions will certainly be respected as long as they're not served up with a helping of "Get your noses out of the books, and keep your eyes on the ball."

Certain metrics are largely ignored here, with good reason. For pitchers, W-L record and ERA are generally useless. For hitters, RBI doesn't really tell us anything. For fielders, errors and fielding percentage have become archaic. Better metrics are consistently applied and there isn't much bias in player evaluation. Regardless, should any of the AA writers or community members appear to be twisting stats to fit a pre-conceived agenda, I would expect a commenter to blow the whistle. If it happens, please point it out.

Also, happy holidays. Or, alternatively, bah humbug!

Tweet Comment 8 comments | 5 recs |

Do you like this story?

Comments

I hope Howard responds to this in poetry form.

And also, I hope everyone had a Happy Festivus!

The Mets lobby Omar for a plan, and his plan, he likes his plan. The problem is that he didn't write his plan down 'cause that makes it paperwork, and that’s false hustle... Know what I’m sayin’?

by Evan_S on Dec 24, 2009 6:52 AM EST reply actions

I like that idea, Evan!

Happy to discuss! Taking isolated sentences out of three different pieces seems rather… selective, so I’ll give some context.
For MLBTR, Tim Dierkes, my editor, prefers more standardized metrics for easier access. For Perpetual Post, the piece was illustrating value- and I use ERA+ in those instances because, unlike FIP, they easily measure relationship to league average. With SNY.tv, I have the opportunity to use advanced metrics, and FIP illustrated the larger point of a different piece.
But there are two larger points illustrated in each of the latter two pieces (as James knows, MLBTR posts are designed to be short, and highlight potential trades, not analytical stat pieces).
For Lowe, peripherals got appreciably worse-his K-rate went down a full K-per-nine, his walk rate went up a full BB-per-nine, and his ground ball rate was the lowest of his career. As James neglects to mention, Lowe’s FIP jumped from 3.26 to 4.06. I went with the components that help make up FIP instead with Lowe, to illustrate the full extent of his decline in each of these areas.
For Pelfrey in 2009, the walk rate and K rate was each fairly similar to 2008. The ground ball rate actually got better in 2009. Yet the perception was that Pelfrey had a fantastic season in 2008 and a lost season in 2009. FWIW, I detailed these components in the Pelfrey piece as well.
In other words, all of that is in the latter two pieces James linked to here. Always appreciate peer review, though, so by all means, check out the articles themselves and let me know if it seems like the logic in the two of them (and the overlapping point made in the brief MLBTR post) are at odds!

by Howard Megdal on Dec 24, 2009 10:23 AM EST reply actions

I wasn't aware of Tim's MLBTR policy

Especially since writers regularly cite FIP and UZR/150. Even you have cited FIP at MLBTR!! And used it, might I add, to demonstrate that a pitcher’s ERA was deceving because of a better FIP. Also, here’s a post by Tim himself featuring UZR/150. Strange for him to have such a policy then go against it! Fair response otherwise I suppose.

by James Kannengieser on Dec 24, 2009 10:49 AM EST up reply actions

Not a policy, but a preference

To be clear, Tim doesn’t have a policy opposing it- just a preference that, all else being equal, we use the more accessible metric- which makes sense to me.

by Howard Megdal on Dec 24, 2009 11:32 AM EST reply actions

I generally take the argument employed by Metsmerized to mean

“If I understood a few more advanced statistics slightly better, I could very well well see myself using them to strengthen just one side of an argument without a full understanding of the subject and its possible counter-arguments,” since its hard to accuse someone of something you don’t fully understand.

Its also true that there’s something of a philosophical responsibility in applying complicated methods of analysis, especially when trying to justify an argument after the fact. But all that really means is you need to be willing to question your own findings, especially if you or someone else finds evidence to suggest something contrary, and watch for the trap of fixing yourself to your conclusions based on pride or some other sort of emotional attachment.

"All energy flows according to the whims of the great magnet

What a fool I was to defy him"

-HST

by Mark Himmelstein on Dec 24, 2009 2:33 PM EST reply actions

FIP has one possible flaw

Every year, due to fluctuating baBIPs, as well as the year-to-year differentials in ground ball rates, fly ball rates, line drive rates, home run rates and differing K-to-BB rates (and park factors, perhaps), with all considered, I think FIP should at least be compared on a year-by-year basis along with those other stats to effectively dictate how a pitchers’ performance changes. Differentials should be used more often, and if a discernible trend could be detected statistically and fundamentally, then a line of best fit can be logically extrapolated to predict as safely as possible, what the pitcher’s value will be heading into the future. With that said, this could have a major impact in deciding which trades could be most beneficial to a team. Obviously, a high correlation rate is needed comparing projections to output, before a particular method of forecasting could be developed, with the above mentioned standards in mind.

FIP is not flawed, fundamentally speaking, but it can not measure with marksmanship, the true “skill” of a pitcher without taking other factors, like park factors et al., into consideration. FIP is only accurate in the context of the situations involved, but not in the context of the mean expected throughput, with regards of a comparison to neutral conditions.

As a matter of personal opinion and research, I’m still convinced that Greg Maddux is the greatest pitcher to have pitched the last 25 years, no matter what the stats say. He and that baseball were truly one and the same – a control-freak of a pitcher, he was. Fascinating is the fact that he finished with a K/9 rate of 6.06 and still finished with a FIP of 3.26! Also the best defensive pitcher, perhaps, of all time.

"The picture looked like I was in the dugout, but they got it all wrong. I absolutely was never in the dugout."

- Mr. B.V. Incognito

by sj10689 on Dec 27, 2009 6:07 AM EST reply actions

It's not the data but the descriptive analysis

This is an issue that often arises through my work. Many people cite ‘average’ without stating what measure of central tendency they mean (oops, unintended pun there). I ask about the distribution and effects of skewness and kurtosis I receive puzzled looks. The problem is most systems attempt to fit data into some type of regular function. When we are talking of something as complex as human activity this becomes extremely difficult as we are always trying to fit attach every variable with some consistency. How can we show someone who is a clutch hitter, someone who makes the critical out in the field or when pitching. For example, if a fielder only attempts to field what they can get they may be 100%, but are they better than the fielder who tries to get everything and makes 98%? The latest entry is UZR, which seems to resemble a Myers-Briggs style analysis. This is dependent on the relative validity of the selection of the axes. As a newcomer to the more exotic measures available to the baseball fraternity I look forward to improving my knowledge on this topic.

by Metabroad on Dec 27, 2009 11:33 AM EST reply actions

reliable statistics show that the gap between them isn’t as large as many think.

example?

over the last two years, cameron had been about a .350 wOBA guy. jason bay, ~.390. it’s pretty clear who the better offensive contributor is

and defense you ask? (you didn’t really ask, i just like phrasing it like that) cameron’s career UZR/150 in CF (which i’m sure you know is adjusted by a full win over a LFer) is 5.7. jason bay as a career LFer? -8. and he appears to be trending downward, which does not appear to be the case with cameron (though it might be inevitable). either way, cameron is a little more than 2 wins better than bay with the glove

prefer overall value?

3 out of the last 4 years, cameron has been worth 4+ WAR. bay hasn’t surpassed that mark in what will be 4 years once the 2010 season starts. and it sure isn’t due to his offensive numbers.

now, obviously, you need to look at who you’re upgrading over. the cardinals are my favorite team. the mets, while being only a passive interest are my second favorite. there was much clamoring on the cards boards about signing cameorn to play LF. of course, if he’s playing LF, his defense is not as valuable. you need to drop about a full win off his totals, and then regress to the mean. (and if he’s playing CF, colby is playing LF or RF which makes his value all but disappear if he repeats his ‘09 offensive season) so as a LFer, it’s pretty reasonable to expect cameron to be about a 2.5 WAR player next season. not worth the $10MM that was being bandied about previously. probably worth the contract that he got from the sox, though. and bay, at about the same value is definitely not worth the $15MM+ that he will command at about the same value. he will also cost more years.

so which player is more valuable to an NL club? a resounding mike cameron. tucked shirts be damned

Of course, hope means being cut down on some street corner, as you run like mad, by a random bullet.

by prophetjohn on Dec 27, 2009 9:07 PM EST reply actions