In other words, let’s say we have Scrappy Cleftchin. He has a 1.23 FIP (really good!) and only one blown save! Obviously this is because he’s an awesome player (and with a name like that, how couldn’t he be?!).

Let’s take Fatso McSucksalot on the other hand. He has a 3.45 FIP (pretty good) and 10 blown saves. Man, he must have problems and be worse than Scrappy Cleftchin, right? I mean, 5 blown saves? Clearly he is no good under pressure.

Only here’s the thing:
Scrappy Cleftchin: 5 IP, 1 save opportunity
Fatso McSucksalot: 120 IP, 80 save opportunities

In other words, the 1.23 FIP is over a really small sample size, and his rate of getting saves is 0% (0/1). Fatso McSucksalot, on the other hand, has pitched twice as many innings as the standard closer at a pretty good rate, and he has a 94% save rate!

It only gets worse when you think about it more.

Sometimes a pitcher comes on to get a save in a 1-run ballgame with the bases loaded and the heart of the batting order coming up. Sometimes a pitcher comes into the game with a 3-run lead, nobody on, and the bottom of the order coming up.

Yet both count equally as saves (and both count equally as blown saves for that matter). So how can we really judge that sort of thing properly?

My basic point is that before we remake the graph, we need to rethink what it’s showing, and whether it’s actually appropriate. In this case, the best graph would be no graph, or just to say, “Papelbon blew it last night” or “Lidge has really choked this year”. Save yourself the space, and all that.

Or you could go into the data and actually show what FIP has to do with save percentage. Mix in leverage index and you may be able to get something halfway decent as a conclusion.

I agree, removing Lidge’s data point was an action I took without justification. The test is to compute the supposed regression including Lidge, then plot the residuals, and determine whether they follow a normal distribution. A chi squared test would be a more discerning test than a simple box plot. When I get a free minute.

How about making a box-plot of the regression residuals? Use the Deming regression you demonstrated earlier since both ratings have error associated with them. If Lidge’s point is an outlier then, you probably can justify removing it.

In the only statistics class I ever took, the first thing the professor told us was: “There are lies, damn lies, and statistics.” The approach I like to take is to visualize the data to get a feel for it, then try to apply a statistical test that supports what my visual intuition is telling me. However, like Rebecca, I know that I can get myself into trouble quickly with statistics. Its best (I think) to stick to tests that you understand and use them consistently rather than trying to find a test that supports your position.

I this case, I think we both agree that there isn’t any correlation between FIP and blown saves, we just discussing the best way to show that.

Lidge’s blown saves are the outlier in this chart.