20 replies to this topic

[absintheofmalaise]

On Hardball Times there is a very interesting article by David Gassko. He believes he has come up with a new metric to replace UZR since it is no longer available.

Let me tell you something: I'm a fan of UZR. I love it, I love it, I love it. So when Mitchel Lichtman (also known as MGL—he's the man who publishes UZR) decided to stop making it public last season, I was crushed. UZR was really by far the least flawed, best thought-out fielding system out there. It was better than Zone Rating because it incorporated all balls in play. It was better than David Pinto's PMR because it had better park factor applications and did not include putouts for infielders. It was better than Clay Davenport's DFTs because it used much more detailed data.

Anyways, let me try to explain this system in as few paragraphs as possible. First, I split up a team's Balls In Play (BIP) based on the number of ground balls, fly balls and line drives they allowed. This allows me to estimate how many outs should have been made by infielders and how many outs should have been made by outfielders. For infielders, only grounders count, while for outfielders, I use outfield line drives and fly balls less home runs.

Next, I find a team's BIP against left-handed batters and right-handed batters. Using "Position Rates" published by Charlie Saeger, I estimate how many balls in play were hit to each fielder.

These are the two biggest advantages of my system over any other non-PBP system. Because I use batted ball data and because I know how balls were put into play by left-handed batters and how many were put into play by right-handed batters, I can come up with a razor sharp estimate of each fielder's "chances." Therefore, the greatest disadvantage a non-PBP system has in comparison to a PBP system is minimized.

He uses it to rate all players except for 1B, pitchers and catchers. The ratings are for 2004. He'll publish 2005 after the season.
He uses RAA as a measuring tool. The Sox rated by position are, minimum 450 ABs in 2004:

Graffanino 13
Bellhorn -11
Rent -23
Youks 4.8
Mueller 0
Manny -23
Damon 1.8
Payton 39
Kapler 2

Measuring Range

Edited by absintheofmalaise, 29 September 2005 - 12:25 PM.

[Doza]

I noticed that Jeter is listed as a -16 RAA, which is interesting considering that RF gurus and mediots all agree that he had one of his best years defensively. I find it amusing that this is probably true in the context that Jeter's rating (either UZR or RAA) is usually -20 or lower. So yes, -16 would be a career year.

The only issue I have found is that this model does not use fly ball outs for infielders apparently (unless I read this incorrectly). Not that this is a huge issue, since the vast majority of outs an infielder would make are on the ground.

Also I didn't know Rent was so bad in 2004. If thats the case, then his performance in 2005 isn't really that far off from what should have been expected.

Edited by Doza, 29 September 2005 - 01:24 PM.

[Bowlerman9]

One problem I have is that if it is meant to replace UZR's consistency, than it shouldnt have major disagreements with UZR.

MGL made it known that in July, despite his high error total (~20 at the time), Renteria's UZR was +2. That makes sense if you think about it. His errors cost the team probably around 15 to 20 runs, but his outstanding range probably saved about the same amount.

I'm shocked to see this metric has Renteria rated so low. I'm curious why there is such a disagreement between the two.

[mr guido]

This season ER's range certainly hasn't impressed me. I haven't seen a move to his right to speak of. Numbers wise, aside from trailing the majors in SS fielding %, he is near the bottom in range factor and zone rating. FRAA lists him as -21 runs. Of course the study above is for 2004, but I wouldn't expect any 2005 numbers to disagree.

It sounds like the algorithm linked is basically a proxy for an actual 'good' analysis of the PBP data, so the numbers are hardly definitive (like every other fielding stat). For the most part the article makes plenty of claims but doesn't do anything to back them up. For instance he says that Pinto's park factors are bad and his are better... but there's no mention of how he applies his own park factors. So, who knows what the system is actually like.

[PedroKsBambino]

There's a thread on Primer discussing this metric. The consensus was that it had promise but needed tweaking in a bunch of ways.

Primer Discussion

One interesting thing in the Gassko article is that there's a fair amount of disagreement amongst his approach, UZR, and Pinto's approach. One still has to conclude that fielding measurement is far, far more art than science at this point, IMO.

[SoxJox]

One still has to conclude that fielding measurement is far, far more art than science at this point, IMO.

<{POST_SNAPBACK}>

Totally agree. Another criticism is that UZR and other systems had no way of distinguishing between hard hit grounders and less hard hit. I used to wonder how in the world you'd ever be able to do that, unless sometime in the far term you have laser sensors mounted throughout the park. But the more I've thought of it, the less I've come to believe it really means that much because, on average, just about each position over the coourse of an entire year will see about the same number of those kinds of darts that handcuff a fielder and often prevent a clean fielding play. Thus, the effect is to cancel itslef out on average across all infield positions. Maybe it's not so clean a case for outfield and line drives vs. fly outs, but my sense is that it is.

Edited by SoxJox, 29 September 2005 - 03:04 PM.

[PedroKsBambino]

That's a good point, and I don't have much reason to disagree with you about the distribution of sharply hit vs squib hits. But I also think until there's a study of it we don't really know with a great deal of confidence, either.

I am really curiuos about the internal metrics teams are using. There was a lot of speculation here that the Sox internal metric was similar to UZR; hard to know, obviously. The A's have spoken about having an internal metric, and you'd have to assume others do as well.

[dnramo]

IIRC, there was something wierd with 1B's relative to 2B's and 3B's relative to SS's in this metric. This was from the offseason, though.

[absintheofmalaise]

David Gassko asked me to post this about his metric.

I'm making no claims that this metric is better than any PBP metric, though I do think it should do better than Zone Rating, and possibly PMR. However, what I am saying is that ZR and PMR can be complemented by this metric. Both ZR and PMR have multitudes of problems (and my metric  has its problems as well--mainly that it doesn't use actual PBP data), but those can be corrected somewhat by combining them with my metric. If you use just my metric, you'll end up with a pretty good measure of fielding, but you are obviously better off combining it with PMR or ZR.

(And as a side note, I don't use park factors; they are only important in a couple of outfields: Coors, left field in Fenway, and maybe San Diego).

[Bdanahy14]

I noticed that Jeter is listed as a -16 RAA

Did he release this years ratings? I thought that he only took the data from 2004.

[PedroKsBambino]

Is Gassko responding to this thread directly...and if so, is he a Sox fan? We could use more original analysis pieces such as his on here, right?

[absintheofmalaise]

Is Gassko responding to this thread directly...and if so, is he a Sox fan?  We could use more original analysis pieces such as his on here, right?

<{POST_SNAPBACK}>

I don't know if he's responding to it directly. I emailed him to let him know I was posting it here and he asked if I would post for him. He's in MA. I'll find out.

[bowiac]

Totally agree.  Another criticism is that UZR and other systems had no way of distinguishing between hard hit grounders and less hard hit.  I used to wonder how in the world you'd ever be able to do that, unless sometime in the far term you have laser sensors mounted throughout the park.  But the more I've thought of it, the less I've come to believe it really means that much because, on average, just about each position over the coourse of an entire year will see about the same number of those kinds of darts that handcuff a fielder and often prevent a clean fielding play.  Thus, the effect is to cancel itslef out on average across all infield positions.  Maybe it's not so clean a case for outfield and line drives vs. fly outs, but my sense is that it is.

<{POST_SNAPBACK}>

One of the reasons that UZR was so radically better than other defensive systems(and it was radically better), was because it actually did take this into account. UZR included data on every ball hit into play, and how hard hit it was. Lichtman purchased this data from either STATS, or Elias(I can't recall which), which has three spotters at every game to measure this very thing. It's also the reason that historical UZR data is impossible to reconstruct, because this only started being tracked in like 1999.

[Spacemans Bong]

One problem I have is that if it is meant to replace UZR's consistency, than it shouldnt have major disagreements with UZR.

MGL made it known that in July, despite his high error total (~20 at the time), Renteria's UZR was +2. That makes sense if you think about it. His errors cost the team probably around 15 to 20 runs, but his outstanding range probably saved about the same amount.

I'm shocked to see this metric has Renteria rated so low. I'm curious why there is such a disagreement between the two.

<{POST_SNAPBACK}>

On the other hand, it agrees with UZR on almost everybody else, which means it's entirely possible that UZR is wrong on Edgar as well. Also, MGL works for the Cardinals, who are Edgar's previous team. The Cardinals are not a dumb franchise, and it's not impossible to think that what he tells STL about Edgar's UZR is different than what he "reveals" to the public. It's not really very likely, but not impossible either.

Totally agree.  Another criticism is that UZR and other systems had no way of distinguishing between hard hit grounders and less hard hit.  I used to wonder how in the world you'd ever be able to do that, unless sometime in the far term you have laser sensors mounted throughout the park.

<{POST_SNAPBACK}>

Wasn't MLB planning for something like this? I swear I heard something about how they would be basically Questecing a ballpark with cameras everywhere to track flyballs and groundballs, and that this could revolutionize defensive statistics. I don't know what happened to it.

BTW, I believe MGL purchased his data from STATS.

Edited by Spacemans Bong, 29 September 2005 - 04:57 PM.

[Rough Carrigan]

So, Damon's barely better than average in the field (or was last year before his arm getting even worse this year) and has had a year on offense that's indistinguishable from his first year with the Red Sox (.804 OPS in 2005 versus .797 OPS in 2002) remember thinking what a great bargain Damon was at $8 million per year in 2002? No? I didn't think he was either.

Why, again, should we give him 3 years later and 3 years older, a huge 4 years 40 million dollar contract?

[dnramo]

So, Damon's barely better than average in the field (or was last year before his arm getting even worse this year) and has had a year on offense that's indistinguishable from his first year with the Red Sox (.804 OPS in 2005 versus .797 OPS in 2002) remember thinking what a great bargain Damon was at $8 million per year in 2002?  No?  I didn't think he was either.

Why, again, should we give him 3 years later and 3 years older, a huge 4 years 40 million dollar contract?

<{POST_SNAPBACK}>

The league got a little bit more pitcher friendly this year so that .804 is better than his .797 in 2002, but in general I agree with you. Damon will get an offer of more than he's worth to the Sox because he has a pretty batting average and plays CF.

[Spacemans Bong]

The league got a little bit more pitcher friendly this year so that .804 is better than his .797 in 2002, but in general I agree with you. Damon will get an offer of more than he's worth to the Sox because he has a pretty batting average and plays CF.

<{POST_SNAPBACK}>

Yep. Not only that, he's immensely marketable because of his hair.

2004 was a great year, but he's basically had several years like 2004/5, but turned into a superstar because of his hair and the fact he plays for a high profile team.

He's a product of hype.

[Todd Benzinger]

The Cardinals are not a dumb franchise, and it's not impossible to think that what he tells STL about Edgar's UZR is different than what he "reveals" to the public. It's not really very likely, but not impossible either.

BTW, I believe MGL purchased his data from STATS.

<{POST_SNAPBACK}>

Why would he make Edgar, a departed FA, seem better than he is in order to please the cards? Seems like it would work the other way.

STATS has three spotters at every game... And all they do with that data is sell it to MGL? Wow.

[absintheofmalaise]

Is Gassko responding to this thread directly...and if so, is he a Sox fan?  We could use more original analysis pieces such as his on here, right?

<{POST_SNAPBACK}>

Yes he was responding directly to the thread. Lifelong Sox fan. He grew up in Mass. Looks like he lurks here and might have already applied to be a member.
And PKB, I agree that his type of analysis would be good here.

And he wanted to note that UZR does look at how hard a ball is hit.

[philly sox fan]

Yes he was responding directly to the thread. Lifelong Sox fan. He grew up in Mass. Looks like he lurks here and might have already applied to be a member.
And PKB, I agree that his type of analysis would be good here.

And he wanted to note that UZR does look at how hard a ball is hit.

<{POST_SNAPBACK}>

If I know what I'm doing - and I may not - I activated his account. Having him give it a try.

[Phil Nevin 23]

Here's another paper damming the defensive capabilities of CI:

Derek Jeter by Gabriel Desjardins of ProTrade