Jump to content


Yo! You're not logged in. Why am I seeing this ad?

Photo

Evaluting Defense - by Dave Cameron


This topic has been archived. This means that you cannot reply to this topic.
9 replies to this topic

#1 philly sox fan


  • SoSH Member


  • 9748 posts

Posted 26 January 2006 - 11:26 PM

A nice review with a bunch of links I've been meaning to bookmark. Instead I'll try to remember this is here.

USS Mariner

#2 Vermonter At Large


  • SoxFan


  • 3060 posts

Posted 07 February 2006 - 09:01 AM

A nice review with a bunch of links I've been meaning to bookmark.  Instead I'll try to remember this is here.

USS Mariner

<{POST_SNAPBACK}>


That's a good piece and, in my view, highlights both the good points and the limitations of defensive metrics. I think its a must-read for anyone who quotes any defensive metrics in a post here.

#3 possumbait


  • Dope smacked himself


  • 4149 posts

Posted 09 February 2006 - 02:06 PM

After slapping EV on the topic, I looked into this, and came across this.

The range in 'probabalistic runs' is consistent with EV's notion of 20-30 run variance between good and bad, if you provide some allowance for luck and sample size. Going in, I do confess to some skepticism on the amount of runs being talked about.

So here is the opening statement on methodology:

To do this, I turned to Chris Dial's methodology, in which a run value is assigned to each out a defender makes. For instance, 98.7% of outs made by a SS prevent a single. A single is worth .47 runs. Each out is also just that: an out, which has a value of roughly .28 runs. So, 98.7% of the outs made by a SS prevent .75 runs. (The other 1.3% account for 1.06 runs each, because sometimes those screamers by the SS get through the gap and go to wall for extra bases.)


Chris Dial goes into some extensive detail, but the key operating assumption is a typical out for SS is worth .75 runs 98.7% of the time, as stated above. That sounds astronomically large to me!

Let me make an assumption that there are 43 PA's per game per side. If an out saves .75 runs on a single, I might expect that a team that hits 16 singles (without any walks or any XBH or double plays) would score 12 runs on average.

See my problem? Am I not reading this right? Did this get addressed a year ago, and I was tuned out?

#4 finnVT

  • 1055 posts

Posted 09 February 2006 - 06:57 PM

Let me make an assumption that there are 43 PA's per game per side.  If an out saves .75 runs on a single, I might expect that a team that hits 16 singles (without any walks or any XBH or double plays) would score 12 runs on average.

See my problem?  Am I not reading this right?  Did this get addressed a year ago, and I was tuned out?

<{POST_SNAPBACK}>


This isn't quite right... each of those 12 singles on it's own is worth .47 runs, so they should score 5-6 runs. The rest of the value is that by those 12 pa's being singles, you also get 12 more pa's. In stating that the result of the game is 12 pa's, you're assuming that those 12 extra pa's (worth .28 each) are outs.

There's another piece of it, though. Each of those 12 singles has some positive value (.75 etc), but there are a number of corresponding plays in the game which have negative value (any out, basically). If you want the runs scored for the entire game, you can't just look at the good events, you need to include everything.



I think....

#5 Vermonter At Large


  • SoxFan


  • 3060 posts

Posted 09 February 2006 - 11:05 PM

This isn't quite right... each of those 12 singles on it's own is worth .47 runs, so they should score 5-6 runs.  The rest of the value is that by those 12 pa's being singles, you also get 12 more pa's.  In stating that the result of the game is 12 pa's, you're assuming that those 12 extra pa's (worth .28 each) are outs. 

There's another piece of it, though.  Each of those 12 singles has some positive value (.75 etc), but there are a number of corresponding plays in the game which have negative value (any out, basically).  If you want the runs scored for the entire game, you can't just look at the good events, you need to include everything. 
I think....

<{POST_SNAPBACK}>

The problem with assigning values like this is that its not all attributable to the fielder. Looking at team runs, I predict the linear weights that can be directly attributable to the pitchers on the team, either through hitting or through pitching residuals (walks, HBP, BK, WP, SB-CS). I don't assign any value to the outs at all at this point. Actual runs vary +/- 100 from these projected runs. I call these defensive runs, but they aren't really that at all. Individual pitchers show great consistency in their individual defensive run rates, so a certain percentage of those runs are attributable to the pitching staff, either plus or minus. Pitcher defensive run values are, like team runs, positive or negative values, but pitcher values are very consistent. Generally, pitchers who strike out a lot of batters and/or pitchers who get weak contact give up negative defensive runs. Pitchers that get hit hard give up positive defensive runs.

So the point is that only a certain percentage of those defensive runs are attributable to the fielders. The worst team in the A.L. last season in defensive outs was KC, who gave up 98 defensive runs. The KC defense was pretty bad, but their pitching staff was horrible too. Tampa Bay was a close second worst, and they should have had better defensive numbers than they did. Again, it was horrible pitching that doomed them. Their (2B+3B) /FO ratio was the worst in the league, but they have a good, speedy defensive OF. Most of those extra base defensive runs had to have been attributable to their pitching.

So, I think that the actual run value that can be assigned to fielders is actually very small - perhaps only +/- 10-20 runs per team per season. Even those runs are probably too many, because defensive positioning is another factor that leads to defensive runs, but can't be quantified. I wouldn't be surprised if the actual difference between a really good shortstop and a mediocre one is something on the line of +/- 2 runs per season.

Edited by Vermonter At Large, 09 February 2006 - 11:06 PM.


#6 possumbait


  • Dope smacked himself


  • 4149 posts

Posted 10 February 2006 - 09:10 AM

This isn't quite right... each of those 12 singles on it's own is worth .47 runs, so they should score 5-6 runs.

There. Right there is my problem. You are right that a single should only rate a single's worth of run probability. Conversely, an out should only rate an out's worth of run probability.

What the Dial methodology does is give an out not only an out's worth of run probability, but the lost base hit's worth as well. That should go both ways, and as the calculation for the amount of runs expected on 16 singles, it doesn't scale when you sum a hit's worth and an out's worth of run probability on a single play.

It makes much more sense to call an out an out, and ascribe a .28 value to it. In doing so, you make the run differentials on defense not as high as 20-30 for a first baseman, but more like 5-10. That is a margin of a single win or two based on 1b defense, and I can begin to believe that.

On the other hand, it seems VaL is a little too pessimistic. What is that 'certain percentage' of defensive runs not attributable to fielders? It seems the creation of outs is a largely pitching neutral metric based on the relative non-variance of BABIP among pitchers.

I can well believe 1-2 wins at 1b, or 3-4 wins as ss, due to defense between the best and the worst at those positions. Scaling the creation of outs with the run value of an out (and NOT the lost base hit) makes sense. I think WARP/win shares do this pretty well, and it is only the propensity of folks wanting to tinker that creates the urge to improve on it, not a flaw in the systems per se.

Like I said in the Petagine thread, this all seems to me that people want to write 'Moneyball II' to make defense the be all and end all.

#7 Vermonter At Large


  • SoxFan


  • 3060 posts

Posted 10 February 2006 - 02:10 PM

There.  Right there is my problem.  You are right that a single should only rate a single's worth of run probability.  Conversely, an out should only rate an out's worth of run probability.

What the Dial methodology does is give an out not only an out's worth of run probability, but the lost base hit's worth as well.  That should go both ways, and as the calculation for the amount of runs expected on 16 singles, it doesn't scale when you sum a hit's worth and an out's worth of run probability on a single play.

It makes much more sense to call an out an out, and ascribe a .28 value to it.  In doing so, you make the run differentials on defense not as high as 20-30 for a first baseman, but more like 5-10.  That is a margin of a single win or two based on 1b defense, and I can begin to believe that.

On the other hand, it seems VaL is a little too pessimistic.  What is that 'certain percentage' of defensive runs not attributable to fielders?  It seems the creation of outs is a largely pitching neutral metric based on the relative non-variance of BABIP among pitchers.

<{POST_SNAPBACK}>


I'm probably too pessimistic, and I haven't isolated pitching runs completely yet, but here is why standard range stats are flawed:

Range stats are generally based on the percentage of balls hit into the zone and the number of outs that are recorded by the corresponding fielder. It assumes that an average number of balls are fieldable, based on league averages. That's fine, except that they don't (to my knowledge at least) account for team pitching. Bad pitching yields harder hit balls, and harder hit balls yield fewer actual fielding chances. The more bad pitchers there are on a team, the more unfieldable balls there are to field. So good pitching/bad pitching are a significant part of those numbers that are assigned by zone states that should not be attributable to fielders. I can't even begin to measure how important positioning is to the range stats, but I think its probably more significant than actual range and pitching. Although I haven't quantified fully yet, I suspect that actual fielding skills are probably only 20-30% of the UZR-type numbers.

Secondly, I think that conventional linear weight calculations for outs are very wrong. In calculating batting runs, I don't factor outs into the equation. Outs have no effect at all on run creation in a linear model. They are, in effect nulls. Teams score runs through non-outs, or in other words what they do in between the 27 outs that each team generally makes in a game. This isn't quite true - the linear model expects a certain distribution of outs and non-outs, and teams score more or fewer actual runs through non-out clustering.

In defensive runs, which are runs scored (or not scored) outside of the linear equation - typically +/- 15% of total runs scored, outs do have quasi-linear weight, but nothing approaching conventional weights. Strikeouts definitely have negative weight, since they are non-contact events, but contact outs that require fielding do not behave in a linear fashion. In some cases, outs probably have trace positive value, while in others the opposite. It depends on the situation.

So defense isn't about making fielding outs at the major league level, its about making plays in a consistent manner on fieldable balls that prevents non-outs from occurring. Again, I think fielding range is a small part of this, smaller than consistency (errors/non-errors), smaller than pitchers limiting contact by hitters, smaller than positioning.

Does any of this make sense?

Edited by Vermonter At Large, 10 February 2006 - 02:20 PM.


#8 Vermonter At Large


  • SoxFan


  • 3060 posts

Posted 10 February 2006 - 02:16 PM

It seems the creation of outs is a largely pitching neutral metric based on the relative non-variance of BABIP among pitchers.


<{POST_SNAPBACK}>


This can't make sense. If BABIP is non-variable, and outs are non-variable (27 per game) where is the variation in runs scored? BABIP does not correlate directly to runs scored. The differential has to be somewhere.

#9 LahoudOrBillyC


  • Indian name is Massages Ellsbury


  • 3853 posts

Posted 17 February 2006 - 11:43 AM

This can't make sense.  If BABIP is non-variable, and outs are non-variable (27 per game) where is the variation in runs scored?  BABIP does not correlate directly to runs scored. The differential has to be somewhere.

<{POST_SNAPBACK}>

VAL, the difference is HRs, BBs, and Ks. There have been several studies that have shown that BABIP is almost all luck and the ability of the fielders, and the other three categories are what determines how well a pitcher will do in the long run.

#10 Vermonter At Large


  • SoxFan


  • 3060 posts

Posted 01 March 2006 - 02:15 PM

VAL, the difference is HRs, BBs, and Ks.  There have been several studies that have shown that BABIP is almost all luck and the ability of the fielders, and the other three categories are what determines how well a pitcher will do in the long run.

<{POST_SNAPBACK}>


Sorry Lahoud, it was a semi-rhetorical question. That's why I use individual rates instead of BABIP in my analyses.