SABR-Toothed Triber: You Down with FIP? (And BABIP!)

“A great deal of what is perceived as being pitching is in fact defense.” – Bill James, 1988

Last week, we talked about wOBA and how it can help us measure a player’s offensive contributions better than many of the more conventional metrics.  This week, we’ll try to tackle pitching, perhaps in a more roundabout manner.

Let me start with a story.  The story begins in 2000 and involves a paralegal in Chicago with too much time on his hands.  I swear this story has something to do with pitching statistics, and we’ll get to the Cleveland Indians pitching staff in due time.  Bear with me…

Our paralegal’s name is Voros McCracken, and he liked to play fantasy baseball.  He needed to know whom to draft for his team, but as he looked at the numbers, he couldn’t come up with a way to separate pitching statistics from fielding statistics and plain old dumb luck.  And if he couldn’t do that, how could he predict which pitchers would perform well, and which would do poorly?  Like I said, he had time on his hands, so he looked to see if a pitcher’s ERA did a good job of predicting future ERA, and he found a ton of variation.  For example, when he looked at Greg Maddux, his ERA in 1998 was 2.22 but in 1999 it was 3.57.  What gives?  Was Maddux a great pitcher in ’98 but just a bit above average in ’99?  Voros crunched the numbers, looked at splits, gnashed his teeth, but he couldn’t understand why Maddux allowed so many more hits (and runs) just a year after he had been so dominant.  Then, Voros looked at other players, and the same thing happened.   Kevin Millwood, Maddux’s teammate at the time, went from being fringe-average in ’98 (4.08 ERA) to a stud in ’99 (2.68 ERA).  Weird.

And then Voros asked a critical question: What if a pitcher controls much less than we think?  What if, in fact, the pitcher has essentially no control over whether a ball hit into the field of play is an out or not?

So he checked.  First he gave Maddux another look.  In ’98, when opposing hitters put the ball in play against him, they batted .272.  In ’99, they batted .334.  Then Voros looked at Kevin Millwood.  In ’98, Millwood’s batting average on balls in play (BABIP) was .323 but in ’99 it was .247.  In front of the exact same Atlanta Braves defense!

This was, suffice it to say, interesting news, and Voros has since done some seminal work with what he calls Defense Independent Pitching Statistics (DIPS).  Basically, he demonstrates that pitchers can control (i.e. repeat with consistency, regardless of luck and defense) just a few things:

1. What percent of batters he strikes out.
2. What percent of batters he walks (or hits with a pitch).
3. At what rate he gives up HRs (this one causes some arguments, maybe we’ll get to that in the comments).

That’s it.  Everything else is dependent on defense or luck: Wins, Losses, Runs, Earned Runs, even Hits.  None of these stats have been demonstrated to be repeatable skills.

Well, Voros blew some people’s minds.  Some people thought he was crazy.  It gives too much credit to strikeout pitchers!  What about a pitcher who consistently induces weak grounders?  It just can’t be true! 

But some others thought he might be onto something.  So they started working to see if there was a stat or formula that might be able to isolate the things a pitcher can control.

The first stat used to evaluate a pitcher’s performance regardless of defense and luck was called Fielding Independent Pitching, or FIP (I still pronounce the whole thing—“F-I-P”—but I know most people pronounce it like it rhymes with “zip”).  And it only takes into account strikeouts, walks, hit batsmen and HRs allowed.  Here it is:

[(HR*13 + (BB + HBP – IBB)*3 – K*2)/IP] + 3.2

It looks complex, but don’t get too bogged down in the formula.  Basically, the first part—HR*13—says HR are bad; they’re going to raise your FIP.  Walks and HBP demonstrate another “negative skill,” so they’re added together.  But before those free passes are multiplied by a coefficient, we have to subtract out IBBs, since they are typically managerial decisions and shouldn’t be attributed to a pitcher’s skill.  Finally, strikeouts are good!  So we subtract off a factor to account for the strikeouts a pitcher accumulates, divide the whole thing by innings pitched, and add a number (usually around 3.2, but it changes from year to year) to make the whole thing comparable to the league’s ERA scale.  Tada!!  Not so scary.

But what’s the point?  Why not just use ERA and call it a day?  Well, there are a few points, I suppose.  The first is that FIP is NOT a replacement for ERA.  ERA is a fine measuring stick of what happened while a pitcher was on the mound.  It just combines the parts for which the pitcher was directly responsible and those parts for which he wasn’t.  FIP is one tool that helps us solve McCracken’s problem of measuring pitcher performance versus the mirages of luck and defense.  Second, FIP does a better job of predicting what a pitcher will do in the future than ERA (this is what stat-heads mean when they say it has a “high r value” or “a strong predictive value”). 

As an example, let’s look at some historical data on CC Sabathia:

There’s a lot of data there, so let me highlight a few points that I find interesting.  First, look at the variation in FIP and ERA: the difference between CC’s highest and lowest ERA is 1.67 whereas for FIP, it’s 1.30.  I mention this only to demonstrate that FIP is typically more consistent than ERA over a player’s career.  (This should be the case, since defense and luck aren’t involved in FIP.)  Second, when Sabathia’s FIP was lower than his ERA (like in 2002), it means that he actually pitched better than his ERA would suggest. 

So let’s look at 2002 and some of his “peripherals”—a fancy word for those stats in the middle—to see what we find.  His BABIP is fairly normal: .290-.300 is around league average, and pitchers don’t have much control over the variation in their respective BAPIPs.  His walk rate that year actually looks high, which should surprise us, since walks inflate your FIP.  Even more, his strikeout rate is low compared to his career norm.  So what gives?  Well, look at the HR rate compared to his career norms.  He had an exceptionally good year with the long-ball in ‘02, and since HR are so important in the FIP formula, his total FIP was below his ERA, and considerably so.

Now look at his Cy Young season.  Believe it or not, CC was unlucky in 2007: his FIP was lower than his ERA.  Why?  Well, his BABIP was the highest of his career!  Yet he somehow pitched his way out of the trouble caused by those extra base hits falling in by cutting his walk rate to the lowest of his career and striking out 7.80 batters per nine innings!  And people wanted to give the Cy Young to Josh Beckett!  Dopes.

Going from the picture of consistency to…something else, let’s look at Cliff Lee’s numbers: 

 Typically, you don’t see so much variation in a player’s FIP from year to year (though, it should be noted that Cliff’s variation in ERA is higher), so let’s look to see if we can figure out what Cliff figured out from 2007 to 2008.  First, look at 2007.  

Yes, he was awful, but not as awful as his ERA suggests—his FIP was nearly a full run lower than his ERA.  Why?  His 2007 BABIP was certainly high, at .313.  Nevertheless, when you’re walking more than three batters per nine innings and letting up an alarming number of HRs, you’re not going to have a lot of easy outings.  Obviously, Lee figured something out in 2008 (Did I need to give you a chart for that?  Probably not.)  His BABIP was right around normal, but look at how much he cut his walks from the previous year!  And now look at HRs!  He cut both by a factor of three.  When you can do that, you’re going to see some big improvements.  

Now look at 2009.  He wasn’t so lucky with BABIP: it was the highest of his career.  But he struck out more than ever before, had a low walk rate, and continued to minimize his HRs allowed.  And that’s what FIP is really all about: your underlying performance can be determined by just a few fundamental things: walks, strikeouts, and HRs.

With that said… How about the current crop of Tribe pitchers?  Let’s look at the starters from ’09 (I’ll leave out a few, since there were over 673 starting pitchers on our roster last year): 

Ummm.  Yikes.  Well, I guess since we got this far, we might as well go a bit further, but that is one ugly chart. 

Let’s start by trying to find some good news.  Look at the BABIP column.  Remember that BABIPs are generally around .290-.300, and that variations are largely due to luck.  Well, last year, our pitching staff was remarkably unlucky.  Sowers was the only one to post a reasonably average BABIP; the others ranged anywhere from sorta unlucky (Laffey) to struck-by-lightning-twice-in-one-day unlucky (Carrasco).  The other mildly good news is that nearly every pitcher on our team had an FIP below his ERA, which suggests that there is more underlying talent on the pitching staff than was demonstrated last season.

But that’s about it as far as the good news goes.  Fausto Carmona walked nearly as many batters as he struck out, and has done so for two consecutive years.  For the sake of comparison, in his last good season, 2007, that ratio was 2.25 strikeouts for every walk.  Justin Masterson is the only one on the list who strikes out more than six batters per nine innings, but his walk rate (5.63 per 9) is unacceptably high.  That weakness is compounded by his obvious difficulty with left handed batters, making him a questionable starter in the long-term picture.  David Huff, who has some potential, needs to cut his walks and HRs if he’s not going to miss more bats, and there’s no reason to think he will. 

More bad news?  Carl Pavano was our best non-Cliff-Lee starter last year.  And now he’s going against us with Twins for an entire season.  Fun!

Alright, I promise I won’t leave you so down in the dumps.  There’s reason for hope.  Let’s look at one more chart before we leave you for the week:

Not completely dissimilar.  Sure Player A strikes out more people than Player B, but he also allows more HR and walks nearly twice as many batters. 

Now if you’ve been reading closely, you’ll recognize Player B.  It’s David Huff’s 2009 campaign.  Player A?  One Clifton Phifer Lee, circa 2004.  This isn’t to say that Lee’s career path is typical—it isn’t.  But there are some underlying similarities, both in makeup, pure stuff, and some early tough luck.  Look, I’m not convinced Huff will become half the pitcher Lee became, but in a season like 2010, when all we have to talk about on the pitching staff is potential upside, it’s a fun thought.

As this post is already getting a bit long-winded, I’ll leave some of the other advanced pitching statistics for another time, but I should mention that not everyone who’s looked seriously at the problem of evaluating pitcher skill believes FIP is the best model.  I’ll be honest: I have some issues with completely ignoring batted-ball data, for one, and there is some evidence that pitchers have less control over their HR rates than FIP assumes.  xFIP and tRA are two other pitching stats that take different approaches to the skill-versus-luck problem; respectively, they attempt to negotiate the issues surrounding HR rates and batted ball types that some have questioned in the FIP model. 

Additionally, Baseball Prospectus just unveiled their new pitching stat, SIERA, which I’ve not yet had time to digest in detail (though I’m simultaneously impressed and nonplussed, if that’s possible).  Maybe some time soon, we’ll look at these other approaches, and what they have to tell us about the Indians pitchers.

As always, feel free to ask questions below, and I’ll do my best to point you in a direction of an answer.  See you next week!

Thanks again to the guys at WFNY for picking me up as an occasional contributor.  The Voros McCracken anecdote is borrowed in part from Moneyball: The Art of Winning an Unfair Game written by Michael Lewis (or by Billy Beane’s evil supercomputers, depending on whom you ask).  Much of the research in this series is built on ideas from The Book: Playing the Percentages in Baseball, the ongoing work at FanGraphs, StatCorner, The Hardball Times, and Tom Tango’s blog, and the countless other blogs and books that refuse to stop thinking and arguing about baseball. 

—

(AP Photo/David Richard)