Contrived stats and other discussion about metrics

[shaggydog2000]

Not to sound flippant, but to return to basic principles, I think the average person is looking for a measure of how well a guy is performing.

In Ye Olden Days, BA was pretty simple in that regard - .200 is the Mendoza Line and .300 plus is All Starish. I remember when I first came across OPS - I thought "cool - but I have no emotional understanding of what these numbers mean, the way I do for a guy hitting .340. I still feel that way occasionally, when running into something like "Dude, the guy's FARKIFIPFSH is 1.456!"

The first stat I look at for hitters is wrc+, because of how it is normalized, a good summary of total hitting ability, and the fact that I usually go to Fangraphs for stats and that is the normalized stat they show. If I went to bbref, I'd probably be pretty happy with OPS+, and for less stat inclined people that would be a lot easier to explain. The two lineup pretty closely most of the time, so it doesn't bother me if people use one or the other. It's easier to use a normalized stat than trying to figure out what the average OPS is now or was 20 years ago. You can compare players across eras more easily. And you can avoid applying standards that might have been true when you were first paying attention to them, but are now completely different.

For pitchers I like K%,BB%, and xFIP. If xFIP- was on the first page of Fangraph stats I'd probably look at that. The rate stats are pretty predictive of success at pitching. And xFIP helps to cut through the short term luck when it comes to small sample size pitching numbers. Some players do have long term variances between ERA and xFIP, but that's why I look at the player's past history to see if they have that in past.

 

[LeastSculptedJew]

My question is what does it tell us? JBJ isn't going to replay his last 25 games again. So what does a CI around his 25 game performance add? It's an honest question.

To my mind what a confidence interval adds is information about how small the sample size is, in the units that are being measured.

If someone tells me that JBJ has an OBP of 0.400, with a 95% CI of 0.150 to 0.600, then I immediately know that my conclusion should be "Hunh. Based on this information, I can tell that JBJ shows up on a baseball field periodically and does something or other."

Whereas if they tell me that JBJ has an OBP of 0.400, with a 95% CI of 0.385 to 0.410, then I can say "Wow, based on this information, I have good reason to believe that not only is JBJ a really good hitter, but, barring any trends or unforeseen circumstances, he will continue to be a very good hitter"

If they tell me that JBJ has an OBP of 0.400 based on 100 plate appearances, I know that I shouldn't trust that number -- but, unlike in the first case above, it's not immediately clear how much I should distrust that number. The CI helps me know.

(To be absolutely clear, and add more geeky complexity, this is all about how certain we are about JBJ's "true, underlying" OBP based on data to date. If we want to guess about how he will do in the next 25 games, the error bars get even wider: in addition to our uncertainty about his true OBP, there's the fact that his next 25 games will vary around that (best, uncertain, estimate) because it's only 25 games.)

 

[Devizier]

Whereas if they tell me that JBJ has an OBP of 0.400, with a 95% CI of 0.385 to 0.410, then I can say "Wow, based on this information, I have good reason to believe that not only is JBJ a really good hitter, but, barring any trends or unforeseen circumstances, he will continue to be a very good hitter"

You can't say that, because the CI refers to your confidence that given the impossible scenario that JBJ could reproduce his previous X plate appearances, he would have the same result. But this isn't an experiment. You can't reproduce baseball games.

 

[The Best Catch in 100 Years]

The first stat I look at for hitters is wrc+, because of how it is normalized, a good summary of total hitting ability, and the fact that I usually go to Fangraphs for stats and that is the normalized stat they show. If I went to bbref, I'd probably be pretty happy with OPS+, and for less stat inclined people that would be a lot easier to explain. The two lineup pretty closely most of the time, so it doesn't bother me if people use one or the other. It's easier to use a normalized stat than trying to figure out what the average OPS is now or was 20 years ago. You can compare players across eras more easily. And you can avoid applying standards that might have been true when you were first paying attention to them, but are now completely different.

For pitchers I like K%,BB%, and xFIP. If xFIP- was on the first page of Fangraph stats I'd probably look at that. The rate stats are pretty predictive of success at pitching. And xFIP helps to cut through the short term luck when it comes to small sample size pitching numbers. Some players do have long term variances between ERA and xFIP, but that's why I look at the player's past history to see if they have that in past.

I was a big xFIP guy until I read this: http://www.baseballprospectus.com/article.php?articleid=14603
I still think it can be useful as a quick and dirty way to see what a guy's FIP would be like with a more normal HR rate (which can be quite fluky) but any given pitcher's HR allowed per pitches-made-contact-with is a better predictor of his HR rate going forward than his FB% (the premise of xFIP is the opposite).

 

[glasspusher]

Department of put up or shut up, here's a graph from 0 to 1 of the correlation coefficient graph data that smastroyin posted: