Bill James' War on WAR

[jon abbey]

Don't think I've seen this posted anywhere here yet, a broadside from Bill James in the wake of the Altuve/Judge MVP discussion about how WAR connects to runs, not wins, which we all knew but James points out that when a MVP candidate is on a team that finishes way below their projected Pythagorean record (Judge/Yankees), their WAR should be docked correspondingly.

"Look, there is a general relationship between runs and wins, a normal relationship, and there is a specific relationship, based on this specific player and this specific team. If you evaluate Altuve and Judge by the general and normal relationship of runs to wins, then it appears that Judge is almost even with Altuve. But if you evaluate them by the specific relationship of Altuve’s runs to the Astros wins and Judge’s runs to the Yankees wins, then Altuve moves up and Judge moves down, and a significant gap opens up between—large enough, in fact, that Judge drops out of the #2 spot, dropping behind Eric Hosmer of Kansas City."

https://www.billjamesonline.com/judge_and_altuve/

 

[jon abbey]

Also for me this connects to another issue I have with WAR, is that they seem to ignore the specific context of defensive plays. If a guy goes over a fence to steal a grand slam with two outs, that is four runs saved by him, but I am pretty sure it is not credited at close to that.

 

[The Gray Eagle]

LOL I was just about to start a thread on this, with this same exact title.

James published a follow-up the other day:
https://www.billjamesonline.com/mvp_followup/

 

[Scoops Bolling]

Also for me this connects to another issue I have with WAR, is that they seem to ignore the specific context of defensive plays. If a guy goes over a fence to steal a grand slam with two outs, that is four runs saved by him, but I am pretty sure it is not credited at close to that.

WAR is context neutral, that's one of the underlying tenets of the stat: you only give players credit for the things they directly control. A fielder has no control over whether there are no men on base, or three, so you strip that out; same thing with batters, if they hit a HR, you don't give them additional credit just because the guy ahead of them got on base. The point is to isolate the things that a given player does, so you that you can compare them as close as apples to apples with another player. If two fielders go over the same fence to catch a ball, but one is dealing with two men on base because the shortstop booted an easy double play while the other guy's shortstop made the play, neither has outperformed the other in making the catch. The catch is the exact same, their performance is identical, the only difference is that one guy's teammate fucked up, and the other guy's didn't. If the point is to compare "which of those two outfielders is better", that's why you strip out context and isolate the things under a given player's control.

If you care about context, then just use WPA.

 

[Rovin Romine]

WAR is context neutral, that's one of the underlying tenets of the stat: you only give players credit for the things they directly control. A fielder has no control over whether there are no men on base, or three, so you strip that out; same thing with batters, if they hit a HR, you don't give them additional credit just because the guy ahead of them got on base. The point is to isolate the things that a given player does, so you that you can compare them as close as apples to apples with another player. If two fielders go over the same fence to catch a ball, but one is dealing with two men on base because the shortstop booted an easy double play while the other guy's shortstop made the play, neither has outperformed the other in making the catch. The catch is the exact same, their performance is identical, the only difference is that one guy's teammate fucked up, and the other guy's didn't. If the point is to compare "which of those two outfielders is better", that's why you strip out context and isolate the things under a given player's control.

If you care about context, then just use WPA.

But WAR is an aggregated counting stat, thus context (injuries, platooning, development curves, aging) matters if you're going to use it as a predictive tool.

 

[OurF'ingCity]

But even WPA can be misleading. If one player hits a 1-run HR in the first inning and another player hits a 1-run HR in the 10th inning of what turns out to be a 2-1 game, I would say each of those players contributed about equally to the win (the second player's HR will be more exciting and memorable, to be sure, but that by itself doesn't mean the second player played "better" that game).

But by WPA, the second player's HR would be ranked as MASSIVELY more important than the first player's and so something like WAR but based on WPA would be just as misleading/wrong as James claims WAR currently is.

WPA is helpful on the margins - it is probably true that certain players perform slightly better in high leverage situations than others - and it's useful for quickly seeing the story of a game (there's usually a pretty good correlation between the player with the highest WPA for a given game and the player an average viewer would consider the "star" of that game), but I don't think it adds much beyond that. (And some of the stats James throws around like RISP or "late innings of close games" are just worse versions of WPA.)

 

[jon abbey]

Yeah, ideally there should be something combining WAR and WPA, but maybe it would have to be too subjective.

 

[Snodgrass'Muff]

Yeah, ideally there should be something combining WAR and WPA, but maybe it would have to be too subjective.

I'm pretty sure that was called win shares.

 

[Savin Hillbilly]

Also for me this connects to another issue I have with WAR, is that they seem to ignore the specific context of defensive plays. If a guy goes over a fence to steal a grand slam with two outs, that is four runs saved by him, but I am pretty sure it is not credited at close to that.

But that sums up in a nutshell what WAR is and isn't. It's not a measure of win value produced. It's a measure of demonstrated competence at things which are pretty well established to correlate to wins. That's a hugely important distinction when it comes to making acquisition decisions, because what matters for deciding how much to invest in player X isn't how much actual win value he produced last year, it's how much win-relevant competence he demonstrated last year.

 

[jon abbey]

So James' issue is that WAR connects to a team's Pythagorean record, not their actual one, which can make it a bit misleading for looking at past production. So wouldn't an easy fix for that be to just add a coefficient of actual wins/Pythagorean wins and add an additional stat multiplying WAR by that?

 

[snowmanny]

In order to answer what question? Who was the best player, absent all context, this year? Who was the most valuable player in terms of contributing to run differential this year? Who was the best player in terms of contributing to wins this year? Who was the best player in terms of contributing to postseason seeding this year? Ignoring aging curves and all context which player would you theoretically draft first if you were replaying the season? Edit: or who was the best player as defined by some combination of adding runs, adding wins, contributing to postseason contention, possibly big performances especially against rivals, with a slight discount for not being a position player, including every bit of context we can identify, this year? The edit being something along the lines of what a lot of voters have traditionally considered, I think.

Ed2 I see I continue to lead the league in commas above replacement.

 

[charlieoscar]

What about looking at which player whose absence would have the biggest impact on batting, pitching, fielding for each team?

For example, with the 2016 Red Sox, David Ortiz led the club in 2B/HR/RBI/BB/OBP/SLG/OPS and nearly led the club in RBI. If you take his numbers out, the team's numbers drop considerably.

Rhetorically speaking, was there another player, if his numbers were removed, who would have had such an impact on his team's overall numbers?

Similarly, subtracting Porcello's stats from the 2016 Red Sox pitching totals would make their numbers look worse (but not to the extent of removing Ortiz's from the offense)? When comparing these players to like players from other teams, fielding could be used as a "tiebreaker." If, say, an RF's offensive stats had a similar impact on his teams numbers as did those of a DH on another team but the outfielder also added to his value with his fielding, then the RF would be more valuable.

A major problem is that MVP is not defined. Some look at gaudy numbers; some look at gaudy numbers without realizing that the player's team could have won anyway because of the strength of the rest of their team; some think that since a team won the pennant they must have the MVP. As things now stand, I don't think that it is fair to say a pitcher can't be the Most Valuable Player; however, it might make more sense to have a Most Valuable Pitcher and a Most Valuable Hitter, with fielding playing a side role.

I haven't suggested putting league-average, or the like, replacement players in when subtracting player's output from their team's output because of two things:
1. Players play half their games in one particular park, and,
2. Players don't all play the same opponents.
So you would need to make intricate park factor adjustments and park factors are not the most reliable numbers.

 

[Dr Manhattan]

Can someone explain in layman terms why the batters contributions have to be scaled to the teams wins? I mean if Judge and Altuve do exactly the same thing in each and every plate appearance, but the Yankees closer blows a bunch of saves and the Astros dont, but everything else pitching and defense wise is the same, then one team has won a lot more games but none of that had anything to with Judge or Altuve. How do you reconcile the notion of having contributions related to a team wins stat, while maintaining the notion of evaluating individual stats in a context neutral to all the other stuff your team does which is nothing to do with you as a batter. Shouldn't it simply be about the runs you create per plate appearance (or if you like, total runs created over a season to reward turnout). Of course theres defense and so on, I mean, just as a hitter. If Judge creates X runs over a season and Altuve creates X runs over a season, why is one any less statistically valuable depending on the other runs created / saved by the other players on their respective teams. You're not "pretending" the Yankees won 100 games instead of 90 you're saying that they created the right amount of runs to do so, whether they did or not can't possibly be attributable to Judge can it?

 

[Gdiguy]

Can someone explain in layman terms why the batters contributions have to be scaled to the teams wins? I mean if Judge and Altuve do exactly the same thing in each and every plate appearance, but the Yankees closer blows a bunch of saves and the Astros dont, but everything else pitching and defense wise is the same, then one team has won a lot more games but none of that had anything to with Judge or Altuve. How do you reconcile the notion of having contributions related to a team wins stat, while maintaining the notion of evaluating individual stats in a context neutral to all the other stuff your team does which is nothing to do with you as a batter. Shouldn't it simply be about the runs you create per plate appearance (or if you like, total runs created over a season to reward turnout). Of course theres defense and so on, I mean, just as a hitter. If Judge creates X runs over a season and Altuve creates X runs over a season, why is one any less statistically valuable depending on the other runs created / saved by the other players on their respective teams. You're not "pretending" the Yankees won 100 games instead of 90 you're saying that they created the right amount of runs to do so, whether they did or not can't possibly be attributable to Judge can it?

Did you read the article?

I don't mean this in a snide way - the answer to your question is the central section of the article:

The potential problem with this approach is that it holds each of the Yankee players to responsible for the shortage of expected wins proportional to their runs created. It could be that it wasn’t Judge who was responsible for this shortfall, but Jacoby Ellsbury or Gary Sanchez or Chase Headley.

But when you look at Aaron Judge’s situational data, it quickly becomes apparent that Judge is not only proportionally responsible for the Yankees poor performance in close games, but that he is more than proportionally responsible. He is disproportionately responsible for the Yankees poor performance in close games. And, while Judge’s situational stats vary from poor to terrible, Altuve’s vary from solid to sensational...

You're right, if the Yankees closer blowing games is the reason they won 90 instead of 100 it's just Judge's fault. But Judge's hitting stats indicate that he was a much better hitter in blowouts than close games, and so his total runs created are worth less wins than the average conversion would suggest.

 

[Max Power]

So James' issue is that WAR connects to a team's Pythagorean record, not their actual one, which can make it a bit misleading for looking at past production. So wouldn't an easy fix for that be to just add a coefficient of actual wins/Pythagorean wins and add an additional stat multiplying WAR by that?

WAR is basically connected to second order Pythagorean record, which is based on expected runs scored and allowed rather than actual runs scored and allowed (standard Pythagorean). If you're concerned about who helped win the games as they were actually played on the field, WAR isn't going to help you much.

 

[PedroKsBambino]

I am glad James is speaking out more---I have felt that since his 'Fog' article his voice is one that is needed in the stat-conscious community.

One of James' great strengths has, and continues to be, an ability to recognize impefection and see gray. Many (of course not all) of the folks who want WAR to be 'perfect, all in one' stat just don't understand it well enough to make the case they want to make; we see that now in basketball around some who adhere to RPM. One can quite easily say "most of the time 'clutch' doesn't exist" and "but in a few rare cases, it likely does" and while that's harder, it's very likely more accurate. So as James walks through his five concerns about WAR, he deserves a serious read and I think, overall, he's right.

As he notes in the article, many people confuse 'true talent' and 'impact on wins'. His example is the lottery ticket, but the argument applies in many contexts. An RBI is a real thing that actually occurred; it can, of course, be overrated and incorrectly attributed fully to 'skill' but it also did occur---and is thus not worth zero, etiher, if we are looking backwards at impact. It may well be worth zero, or at least close to that, if we're trying to assess true talent and looking forward. But those are two different inquiries.

 

[jon abbey]

Nate Silver weighs in on Twitter:

Nate Silver‏Verified account @NateSilver538 2h2 hours ago
Been catching up on the WAR debate and I think Bill James makes the more convincing case. Player win totals should tie back to team win totals.

Or maybe you have two versions: - pWAR (predictive WAR), which tries to remove "luck", and - dWAR (descriptive WAR), which doesn't and is more of an accounting system, a la Win Shares. Some of the current versions of WAR fall into sort of an uncanny valley between pWAR and dWAR.

 

[Gdiguy]

Nate Silver weighs in on Twitter:

Nate Silver‏Verified account @NateSilver538 2h2 hours ago
Been catching up on the WAR debate and I think Bill James makes the more convincing case. Player win totals should tie back to team win totals.

Or maybe you have two versions: - pWAR (predictive WAR), which tries to remove "luck", and - dWAR (descriptive WAR), which doesn't and is more of an accounting system, a la Win Shares. Some of the current versions of WAR fall into sort of an uncanny valley between pWAR and dWAR.

I think the ultimate answer is to go in-between WAR and WPA - WAR effectively says 'given X number of hits / HR / etc, what value of runs did the player produce'. What you want is more 'given X hits with runners on 2b, x hits with runners on 3b, y HRs with runners on, X hits in blowouts, X hits in close and late games', what value of runs did the player produce times what is the relative weight of those hits in terms of wins

It's much more of a pain in the ass, but those weights should be not that hard to calculate (since they're not player-specific, you have a ton of data at this point), and it's not like you need to calculate WAR by hand. Yeah, it means you need to calculate it off of play-by-play data instead of box scores (or season-level stats), but that's not that hard to do now

 

[Dr Manhattan]

You're right, if the Yankees closer blowing games is the reason they won 90 instead of 100 it's just Judge's fault. But Judge's hitting stats indicate that he was a much better hitter in blowouts than close games, and so his total runs created are worth less wins than the average conversion would suggest.

But for example in football, statistics show that the good teams actually DO beat up on the bad teams, and that blowouts are indicative of something positive, I've always wondered whether the same is true in baseball. "Piling on" the poorer opponents you face is possibly something that elite hitters actually do, so it makes no sense to penalise them for that. I might try and find some data to back this up but I would be extremely surprised if all of the elite hitters stats were not clearly loaded up by beating up on the crappier pitchers.

 

[charlieoscar]

But for example in football, statistics show that the good teams actually DO beat up on the bad teams, and that blowouts are indicative of something positive, I've always wondered whether the same is true in baseball. "Piling on" the poorer opponents you face is possibly something that elite hitters actually do, so it makes no sense to penalise them for that. I might try and find some data to back this up but I would be extremely surprised if all of the elite hitters stats were not clearly loaded up by beating up on the crappier pitchers.

Off hand, I'd say no, with the first thing coming to mind is the handedness of the batter/pitcher match-up. Take Ortiz: in 2016 he had an OPS of .808 vs.LH starters and 1.070 vs. RH starters and take Steve Carlton (LHP) who went 27-10 for the 1972 Phillies who finished 59-97. That would be an extreme match-up but I'm guessing Ortiz might not "pile on."

 

[Dr Manhattan]

I'd be surprised if it wasnt true in baseball but obviously need some data to back this up. A similar study was done IIRC in cricket. This does not mean that the best hitters don't have the ability to do better against elite pitchers, it was more that the best hitters have statistics that show a significant extra "loading up" stats against the poorer pitchers. In other words, the difference between Ortiz and a replacement player is more skewed against the poor pitchers. It would be difficult to think any other result would be the case, logically speaking if you think about it. The conclusion being that you shouldnt just penalise good players because they piled on against the chaff. Thats what good players are supposed to do.

 

[Gdiguy]

I'd be surprised if it wasnt true in baseball but obviously need some data to back this up. A similar study was done IIRC in cricket. This does not mean that the best hitters don't have the ability to do better against elite pitchers, it was more that the best hitters have statistics that show a significant extra "loading up" stats against the poorer pitchers. In other words, the difference between Ortiz and a replacement player is more skewed against the poor pitchers. It would be difficult to think any other result would be the case, logically speaking if you think about it. The conclusion being that you shouldnt just penalise good players because they piled on against the chaff. Thats what good players are supposed to do.

Again... did you read the article?

In the late innings of close games (100 plate appearances), Judge hit .216 with a .780 OPS. But when the Yankees were 4 or more runs ahead or 4 or more runs behind (112 plate appearances), he hit .382 with an OPS of 1.500.

In the late innings of close games, Jose Altuve hit .441 with a 1.190 OPS. When the Astros were 4 or more runs ahead or 4 or more runs behind, Altuve hit .313 with a .942 OPS.

In what Baseball Reference identifies as "high leverage" situations, Judge hit .219 with an .861 OPS. In medium leverage situations he improved to .297 with a 1.058 OPS, and in low leverage situations he hit .299 with a 1.115 OPS. Altuve hit .337-.377-.329 in those three situations.

The question isn't if good players in general have skewed numbers, the question is 'given two players who were responsible for the same value in terms of "runs", are some runs worth more than other runs in terms of adding wins'. In many cases it's hard to deconvolute that, but between Judge and Altuve it seems relatively straightforward - a run in a high leverage situation is strictly more valuable than one in a low leverage situation in terms of winning baseball games.

 

[charlieoscar]

The lead-off batter in a game hits a home run and there is a leverage index assigned to that hit, which I suspect is not very high. There is no more scoring in the game so that home run turned out to be the key hit. Did the assigned leverage index change as the game went along or is it static? Wouldn't a bases-empty HR with two outs in the bottom of the 8th in a scoreless game be rated higher?

Obviously, the chance of having a 1-0 final when the run is scored in the 1st is a lot lower than having a 1-0 final when the run is scored in the 8th, but isn't the effect of the two home runs the same?

 

[snowmanny]

The lead-off batter in a game hits a home run and there is a leverage index assigned to that hit, which I suspect is not very high. There is no more scoring in the game so that home run turned out to be the key hit. Did the assigned leverage index change as the game went along or is it static? Wouldn't a bases-empty HR with two outs in the bottom of the 8th in a scoreless game be rated higher?

Obviously, the chance of having a 1-0 final when the run is scored in the 1st is a lot lower than having a 1-0 final when the run is scored in the 8th, but isn't the effect of the two home runs the same?

I don't know. Let's say you hit a two-run home run in the bottom of the first off the other team's #4 starter and your team wins 2-1. Is that the same or not the same as a two-run home run in the bottom of the ninth to win 2-1 given the situation and the crowd and the gatorade bath risk and the WPA and the fact that you are presumably being pitched to by someone who was specifically out there because of his superior skill and coolness under pressure and overall clutchiness and maybe even his specific ability to get you out? And is any of that better than hitting a home run in the bottom of the fifth with your team ahead or behind 9-0? And if it's a home run just inside the pesky pole is that really better than a triple to the triangle? I'll hang up and listen.

 

[charlieoscar]

Isn't that the point? One applies a statistic based on averages to specific incidents and I'm not sure how much weight should be given it. Isn't that like flipping a coin ten times, getting eight heads, and deciding you have a flawed coin?

 

[AB in DC]

Or maybe you have two versions: - pWAR (predictive WAR), which tries to remove "luck", and - dWAR (descriptive WAR), which doesn't and is more of an accounting system, a la Win Shares.

I think you're on the right track separating predictive and descriptive stats.

But I really wish we could get away from the mentality that seems to view WAR as the be-all and end-all of baseball stats. WAR, like any other metric, is an attempt to answer one question -- I'm not enough of a sabermetrician to describe that question in a sufficiently precise manner, but I can guarantee that that question is not "Who should win the MVP?"

Some of the current versions of WAR fall into sort of an uncanny valley between pWAR and dWAR.

I had a very similar argument with Aaron Schatz about DVOA a few years ago. If you mix predictive elements and descriptive elements into a single metric, you're going to get a result that's not going to be very good at being either.

 

[Rovin Romine]

I think you're on the right track separating predictive and descriptive stats.

But I really wish we could get away from the mentality that seems to view WAR as the be-all and end-all of baseball stats. WAR, like any other metric, is an attempt to answer one question -- I'm not enough of a sabermetrician to describe that question in a sufficiently precise manner, but I can guarantee that that question is not "Who should win the MVP?"

I had a very similar argument with Aaron Schatz about DVOA a few years ago. If you mix predictive elements and descriptive elements into a single metric, you're going to get a result that's not going to be very good at being either.

I agree. There are differences between the value of what actually occurred (in context), vs. how much an individual player's striving during a given season might have helped an average team on paper, vs. how we might value that player's basic skill set to contribute to a future team. There can be no single metric that answers those different concerns.

Ironically, that's sort of the foundation of sabermetrics in the first place, yes? The idea that simple stats themselves (BA, RBI, SB) don't describe what's going on precisely enough to make good judgments about players and performances?

 

[charlieoscar]

SABR's Director of Publications just sent out an email looking for peer reviewers for articles that might appear in the future. One of the subjects is, "Can WAR be Simplified for Fans?" Found that amusing given some of the discussion in this thread.

 

[Savin Hillbilly]

WAR, like any other metric, is an attempt to answer one question -- I'm not enough of a sabermetrician to describe that question in a sufficiently precise manner

I'm not either, but I'll take a shot: "How much ability to contribute to wins did the player demonstrate with his performance?" Or to phrase essentially the same idea differently, "How many wins would a team typically add to its record by swapping a replacement-level player for a player of this demonstrated ability?"

 

[OurF'ingCity]

Or to phrase essentially the same idea differently, "How many wins would a team typically add to its record by swapping a replacement-level player for a player of this demonstrated ability?"

I think this is a pretty good description of what WAR is trying to get at. My problem with James' article is that he seems to take as a given that the MVP award is for the player who most contributed to his team's wins for the past season, but I'm not sure that is the only legitimate way to vote for an MVP. Another way to look at it is "the player who most raised the likelihood that his team would win as compared to a replacement player." That could be measured via WPA or some other stat. And an even simpler way to look at it would be "who was the best player this year" - in other words, who put up the best individual performance regardless of team or record. This is to some degree what WAR attempts to measure. I think most if not all MVP voters intuitively if not consciously merge all three of those standards when voting for MVP.

James obviously can't be saying (or, if he is, he can't really mean) that a player's value for MVP purposes is wholly tied to how his team performed even if that performance had nothing to do with the player himself. For example, imagine a scenario where a player hits 100 HRs and leads every other hitting category and is also a great fielder, but his team goes 0-162 anyway because the rest of the team is a bunch of Little Leaguers. Reading James literally, this player would have zero value despite the fact that he put his team in a better position to win every game than if he hadn't been on the team and put up an absolutely amazing individual performance (needless to say his WAR would be off the charts).

Or to think of another example - take the same awesome player above, but say that due to weird luck all his hits came in games his team lost and he never got a hit in a game the team won. Again, I wouldn't consider that player to have zero value because, again, he put his team in a better position to win than otherwise and put up great individual stats.

 

[Sampo Gida]

James will always plug the middle IFer with little power (relatively) over the power hitting corner OFer. Thats who he is. In this case he was mostly right and I am glad he pointed out the misuse of WAR as the central metric in MVP debates.

That said I felt his most interesting point, or shall I call it a confession, was here.

" First, we do not, in fact, "know" that there is no such thing as an ability to hit better or worse in a key situation. We do know that MOST deviations from normal performance in clutch situations are the result of luck, rather than ability, and we cannot prove that those deviations are not 100% due to chance—but we can’t prove that they are 100% due to chance, either. The data would look very much the same as it does whether those deviations were 100% due to chance or whether they were 70% due to chance. We do not, in fact, know which one it is.

I acknowledge that, in the 1970s and 1980s, sabermetrics reached a consensus on this issue, and I acknowledge that I was part of that consensus. But we wrong. We jumped the gun. We should have remained agnostic on the issue until more convincing analysis is done."

This opens up the "RBI's are a useless stat" worm that the science settled long ago since it was said to be all luck and opportunity. Like with WAR one must use RBI's wisely and adjust for opportunity and run environment, but RBI as well as Runs scored play a role in the MVP debate IMO even if you think luck weighs in more than clutch.

Looking at Judge and Altuve ability to drive in ROB and uaing BP's Obi% stat one sees that Altuve and Judge are about equal (14.5% Judge, 14.2% Altuve). Judge of course drove himself in a lot more often even though he beat up mostly lesser teams in lower leverage situations.

Betts probably deserved more MVP love. 19.0% OBI rate which was 2nd in AL among players with more than 300 ROB. Playing for an offensively challenged team at times and playing outstanding defense in a difficult RF (unlike YS3). Too late now.

 

[charlieoscar]

This opens up the "RBI's are a useless stat" worm that the science settled long ago since it was said to be all luck and opportunity. Like with WAR one must use RBI's wisely and adjust for opportunity and run environment,

I get what you are saying but when you use an exaggerated argument like a batter who bats 1.000 but never hits a HR and never comes to the plate with runners on, which means he never gets an RBI, versus a pitcher who bats .000 but always comes up with the bases loaded, who will probably get a few RBI, then the argument slants towards having men on base.