[gator92]

Tangotiger, on Jan 14 2008, 01:37 PM, said:

gator: I wasn't trying to distinguish between hitters and pitchers.

I'm trying to distinguish between players. You have a certain % of hitters that remain cool, another % that get a bit nervous. Another that gets alot more nervous. And others just crumble. This applies to hitters and pitchers.

So, when you have a cool hitter facing a cool pitcher, then the expected performance and actual performance will match (over a long period of time). If you have a cool hitter facing a crumbling pitcher, he will do great. And a cool pitcher (say Mariano Rivera) facing a crumbling hitter will do great.

OK, I understand. That's another reason detecting the signal is going to be so tough here, as you have to try to apportion responsibility for the event among the pitcher and the hitter (and perhaps the catcher's pitch-calling and the manager's play calling, etc. etc.) Makes me wonder if this will ever be settled, or as you say, it may not ever be apparent except in retrospect for a player, which makes it interesting but not actionable between the lines...

[Tangotiger]

Actionable. That's the perfect word.

Hustle is actionable. Laziness is actionable. Smart baserunning is actionable. Great hitting is actionable.

Clutch performance? Unless you come across Rich Ankiel, when is it actionable? Putting ARod as your #8 hitter for one game? If Torre really believed it, he'd have him there for half the season at least. When can you really base your decision on what you believe to be clutch?

[Tangotiger]

Quote

I can't measure your dedication to or love for the game. I believe both exist absolutely in you and would defend you vehemently to anyone if I ever had to. I think your love of the game does you much good and in fact has led to what is arguably your life's passion. Do you disagree?

And if it wasn't clear with my reply post, I have stipulated that clutch does exist. And my dedication and love can be measured. It's measured in what I do and how I say it.

However, suppose I was mute, I never wrote anything, never said anything, never went to the ballpark, and just watched it on TV, with no Neilsen box. What does it give you? Nothing at all. With no measurement of me at all, I may as well not exist to you or to the Redsox. My baseball love exists to me, but to no one else.

That's what clutch is: it exists, but can't be measured. To the extent that it can't be measured, what value does it give you, in terms of making it an actionable item? None at all. It just sits there, like a blob, providing no value to anyone other than to itself.

The value of clutch is to fans is really after the fact, it's remembering what Ortiz did in the past. It does almost nothing for what he will do in the future. Unless you are Mariano Rivera. And at this point in the game, isn't it too late to do anything about it?

[yep]

Again, I'm just thinking out loud here, and it might be totally impossible to measure or detect "clutch" until a player's career is nigh over.

But for the sake of argument, let's say we successfully come up with a measure for clutch (even if it's just of past performance), then take a bunch of historical players whose careers were definitively clutch. This gives us a data set from which we start to look for predictive characteristics that might show which players are likely to be or become clutch.

I agree completely that by the time a guy has a large enough history of clutch situations, detecting his "clutchness" is likely to be no longer useful, at least in terms of evaluating that particular player. But if we can identify clutch as a genuine skill or characteristic of that player and others, then perhaps we find that clutch players consistently have scrunchy eyebrows, or never swing at early-count breaking stuff, or hit to the opposite field in late innings or whatever. Those *are* measurable characteristics, and we can start to look at them and see if any of them might have predictive value relating to future "clutchness."

Cheers.

[LoweTek]

Tangotiger, on Jan 14 2008, 05:05 PM, said:

And if it wasn't clear with my reply post, I have stipulated that clutch does exist. And my dedication and love can be measured. It's measured in what I do and how I say it.

However, suppose I was mute, I never wrote anything, never said anything, never went to the ballpark, and just watched it on TV, with no Neilsen box. What does it give you? Nothing at all. With no measurement of me at all, I may as well not exist to you or to the Redsox. My baseball love exists to me, but to no one else.

That's what clutch is: it exists, but can't be measured. To the extent that it can't be measured, what value does it give you, in terms of making it an actionable item? None at all. It just sits there, like a blob, providing no value to anyone other than to itself.

The value of clutch is to fans is really after the fact, it's remembering what Ortiz did in the past. It does almost nothing for what he will do in the future. Unless you are Mariano Rivera. And at this point in the game, isn't it too late to do anything about it?

Well Tango, in a way I think we agree and in another I think we disagree. I appreciate your responses and thoughts. In what started as a debate about clutch maybe I took it off topic somewhat by reacting to your contention '..if it can't be measured...' Everything in my life's experience, be it as a player on a baseball field, a coach or manager observing same or in business, where my professional endeavors depend greatly on objectively measureable improvement, success and predicting it or more importantly influencing it is a complex mix of measureable and immeasureable factors. This mix is interpretable only by the timely, intelligent and thoughtful application of both by thinking human beings. For example, if one Terry Francona does not intuitively understand the power of the redemption motive, we don't celebrate 2004. We diverged on objective v. subjective. I should admit I have read some but not a lot of your work. I simply sensed you were being overly exclusive of the value comparison between objectively measureable data v. real things that exist and impact outcomes. For all the measureable stats out there; in the end I think it it is those factors less measureable that give the baseball quantitative analysts their raison d'etre and not the other way around.

I hope you are able to 'measure' my dedication to and love for the game as well.

If I'm behind a run or two and there are two outs and two runners in scoring position, I want David Ortiz in the batter's box if I can't be in the box myself. If I lose the game, I know I have put forth the best I can with performers who are unfazed if not unusually motivated by such a moment.

[Eric Van]

Tangotiger, on Jan 14 2008, 04:34 PM, said:

His knee injury caused him to hit worse in pressure situations than in other situations? I'm comparing his performance in crucial situations to non-crucial situations, in the same year. Why would his knee injury only affect him sometimes?

No, his knee injury negated his previous demonstrable ability to excel in the highest leverage situations.

His "clutchiness" numbers in his great clutch run were derived almost entirely from performing mind-bogglingly well in the highest leverage situations.

In 2006, he had a 4.34 WPA with LI < 3.65, and 3.69 in 13 PA with LI > 3.65.

Last year, he had a 4.36 WPA with LI < 3.65, essentially identical, and again had 13 PA with LI > 3.65, but had just a 0.45 WPA. The difference between the two years in these 13 highest lev situations was 3.24 wins, which is almost all of the 3.96 difference in clutchiness that FanFraphs shows.

Quote

Just once I'd like someone to give me a list of 10 names, before the season, and proclaim them the best clutch hitters.

Well, what you really mean, of course, is for someone to give you a list of 10 clutch guys and 10 anti-clutch guys and for the first group to outperform the second in the clutch, and for the person to be able to do this year after year.

My gut feeling is that if you take all the noise out of the data, this is a solvable problem. It clearly does not yet work if you merely take the clutch numbers from FanGraphs.

[Tangotiger]

Eric Van, on Jan 15 2008, 12:23 AM, said:

No, his knee injury negated his previous demonstrable ability to excel in the highest leverage situations.

His "clutchiness" numbers in his great clutch run were derived almost entirely from performing mind-bogglingly well in the highest leverage situations.

In 2006, he had a 4.34 WPA with LI < 3.65, and 3.69 in 13 PA with LI > 3.65.

Last year, he had a 4.36 WPA with LI < 3.65, essentially identical, and again had 13 PA with LI > 3.65, but had just a 0.45 WPA. The difference between the two years in these 13 highest lev situations was 3.24 wins, which is almost all of the 3.96 difference in clutchiness that FanFraphs shows.
Well, what you really mean, of course, is for someone to give you a list of 10 clutch guys and 10 anti-clutch guys and for the first group to outperform the second in the clutch, and for the person to be able to do this year after year.

You still haven't told me why his knee injury impacted his performance with LI > 3.65, but not his performance with LI < 3.65.

[Tangotiger]

LoweTek, on Jan 14 2008, 11:46 PM, said:

Well Tango, in a way I think we agree and in another I think we disagree. I appreciate your responses and thoughts. In what started as a debate about clutch maybe I took it off topic somewhat by reacting to your contention '..if it can't be measured...' Everything in my life's experience, be it as a player on a baseball field, a coach or manager observing same or in business, where my professional endeavors depend greatly on objectively measureable improvement, success and predicting it or more importantly influencing it is a complex mix of measureable and immeasureable factors. This mix is interpretable only by the timely, intelligent and thoughtful application of both by thinking human beings. For example, if one Terry Francona does not intuitively understand the power of the redemption motive, we don't celebrate 2004. We diverged on objective v. subjective. I should admit I have read some but not a lot of your work. I simply sensed you were being overly exclusive of the value comparison between objectively measureable data v. real things that exist and impact outcomes. For all the measureable stats out there; in the end I think it it is those factors less measureable that give the baseball quantitative analysts their raison d'etre and not the other way around.

I hope you are able to 'measure' my dedication to and love for the game as well.

If I'm behind a run or two and there are two outs and two runners in scoring position, I want David Ortiz in the batter's box if I can't be in the box myself. If I lose the game, I know I have put forth the best I can with performers who are unfazed if not unusually motivated by such a moment.

Alright then. Given the choice between Ortiz or Manny or anyone else on the team, you want Ortiz. Now, what happens if Manny outperform Ortiz in this situation for the next 100 PA? Are you still going to choose Ortiz? At what point do you not choose Ortiz?

As for the subjective, my Fans' Scouting Report, which I believe in, is *entirely* subjective. I don't know of any other sabermetrician willing to rely on as much subjective data as I do.

[Tangotiger]

Eric, how about this for a challenge:

I ask people for their team's top clutch hitter, the one guy they want with the game on the line (and preferably also not their BEST overall hitter, as that defeats the point!).

After the season is over, I select the 50 PA that were the most crucial for each guy (using Leverage Index, or whatever criteria you or someone else wants to choose).

Do this for all 30 teams. And let's see who performs better in these 1500 PA: the clutch hitter, or the top overall hitter (who was also not chosen as top clutch).

Sounds fine?

[yep]

Bellhorn, on Jan 14 2008, 12:42 PM, said:

yep - good thoughts here; I would just offer two points in response:

1) It's not so much that the greater impact of clutch runs has been discounted (see this article by Tangotiger, for example) - it's that the run totals are probably so small that they still don't add up to very much. For example, in The Book (which I highly recommend, if you're interested in topics like this) the greatest observed clutch performance from the period of 2000-2004 was by Bret Boone. Even if we imagine that this was somehow a statistically real phenomenon (and somehow predictable in advance), this was still only worth 12.2 "clutch runs" higher than what would have been predicted by his non-clutch level of performance, if I'm doing my calcuations right. Using the estimate that clutch runs have a win probability impact three times greater than non-clutch, that's still only 3.6 wins added over a five-year period...

Understood 100%

My thought was not that a hypothetical "Mr. Clutch" would produce a vastly better W-L record, it was he could tip the balance in the situations that really matter, specifically in the post-season. In every way, it is exactly those last few wins that are the most valuable to a team that is already constructed to make it to the playoffs.

We have (or at least I have) grown accustomed in the past few years to thinking that roster construction is mostly a question of building a team that can get to the dance in October, and from there it's mostly luck. And from that POV, teams like the Red Sox are pretty well shored-up. They are perennial playoff contenders and appear to have the resources to continue to be, so we should expect a certain frequency of championships just based on the odds if nothing else.

But if "clutch" *is* a repeatable skill, and especially if "choking" is a consistent anti-skill, then it may be that the playoffs are not quite so subject to the vicissitudes of fortune, and that some teams are built to succeed in the post-season and others are not in ways that do not necessarily correlate directly with their regular-season WL record or with other "generic" predictors of winning percentage.

If there is some Bruce Banner of a player out there who suddenly becomes the Incredible Hulk in the bright lights of Fox-broadcast elmination games, then his value to a team that expects to be in regular playoff contention could be much greater than another Bruce Banner who never got hit with the clutch gamma rays, but who otherwise looks identical. Moreover, and perhaps more importantly, if there is some other wax-winged player who for whatever reason has an inherent tendency to fail when he gets close to the sun, that player's actual value could be significantly less than it appears based on conventional measures, especially to a team that already expects to play in October.

Again, I'm not saying any of this is so, and I have no idea how to look for it, or whether it could ever be forecast, presuming it could even be detected. But it does seem to me that most of the investigations of whether clutch "exists" seem to be set up wrong. I'm not sure how you would measure it-- clutch situations are one of those things that we know when we see but are hard to define. But the more I think about it, measures such as win expectancy and derivatives thereof seem a little wide of the mark to me, and seem to be tainted with the presupposition that the appearance of clutch is an illusion to be accounted for, rather than a phenomena to be measured.

I think Tangotiger is absolutely right that in order for this to be anything more than a parlor discussion of hypotheticals, we would need to have a meaningful measure of what a "clutch situation" is, and I'm not sure how to get there yet.

Cheers.

[Tangotiger]

Clutch situations:
http://www.insidethebook.com/li.shtml

Now, I made my proposal. If someone wants to be part of the solution, I'd like to hear other proposals. Giving me reasons why my proposal won't work, without offering another, is being a politician.

So, let's hear 'em...

[gator92]

I volunteer to throw darts at a wall covered with all 30 teams' 40-man rosters, and beat both lists

[LoweTek]

Tangotiger, on Jan 14 2008, 07:33 PM, said:

Alright then. Given the choice between Ortiz or Manny or anyone else on the team, you want Ortiz. Now, what happens if Manny outperform Ortiz in this situation for the next 100 PA? Are you still going to choose Ortiz? At what point do you not choose Ortiz?

When he retires.

Tangotiger, on Jan 14 2008, 07:33 PM, said:

As for the subjective, my Fans' Scouting Report, which I believe in, is *entirely* subjective. I don't know of any other sabermetrician willing to rely on as much subjective data as I do.

We're ok Tango. I promise to put your book in my queue with the other three so I'll be better informed going forward. Thanks for the indulgence and I remain most respectful of your views, FWIW.

Sincerely, Retired 2B and goaltender in Section 17.

This post has been edited by LoweTek: 15 January 2008 - 07:45 AM

[Tangotiger]

LoweTek, on Jan 15 2008, 12:44 PM, said:

When he retires.We're ok Tango. I promise to put your book in my queue with the other three so I'll be better informed going forward. Thanks for the indulgence and I remain most respectful of your views, FWIW.

Sincerely, Retired 2B and goaltender in Section 17.

I actually also contributed to the "talking past each other", since the term "measurement" may be implied as data-based. I meant "measurement and/or observations". This would therefore cover everything that data or our eyes would capture as something in a persistent nature (hustle, quick bat, etc). It is "clutch" (i.e., remaining cool throughout, while someone else crumbles a little to alot) that I contend can only be captured with a great deal of data (at which point it now becomes useless to us) and not very much with our eyes (other than Rick Ankiel, maybe Mariano, and a tiny handful of others).

Anyway, I'll get my Clutch project going...

[yep]

Tangotiger, on Jan 14 2008, 08:15 PM, said:

Clutch situations:
http://www.insidethebook.com/li.shtml

Now, I made my proposal. If someone wants to be part of the solution, I'd like to hear other proposals. Giving me reasons why my proposal won't work, without offering another, is being a politician.

So, let's hear 'em...

Heavens forfend that I should be likened to a politician for failure to produce other ideas!

I actually have a couple ideas, but it's going to take me some time to work them out, and it'll probably be at least next week before I can get to it. Basically, I have in mind to come up with some kind of situational filter based on standings and games remaining, that would focus on the state of the team's season as of game time. Ideally, the closer a team is to an elimination game, the higher a number it would spit out, with the value rapidly declining as the team is either further ahead or behind, or as the number of games remaining increases, and maybe turning negative if the team has clinched or been eliminated. This filter could then be applied to any flavor of win probability or even just conventional stats to see whether the player performs any differently when the season (and not just the game) is on the line. If it could be made to produce a percentage with a max value of 1 for an elimination game, I think it would be pretty straightforward to "weight" a player's performance in critical games against his norm.

I am emphatically a lightweight at this kind of stuff, so if anyone cares or wants to take a crack at developing the math, it won't hurt my feelings any. I'm kind of thinking that maybe "games behind if win" and "games behind if lose" should both be incorporated in order to give more weight to direct face-offs with division, wild card, or playoff opponents. If anybody has any suggestions for a better way to look at or evaluate these things, please advise.

And again, for the record, I make no contention that this will be a terribly revealing excersize. I am for the most part a "clutch skeptic." But I would like to see if there's a way to examine clutch that is closer to how fans, players, and baseball people have tended to perceive it for the last 100 years, and see if there's anything there.

Cheers.

edit: typo, and changed "multiplier" to "filter" (thanks to Tangotiger)

This post has been edited by yep: 15 January 2008 - 11:03 AM

[Tangotiger]

I'm not against the idea of incorporating the standings into this.

I want to make two important points:
1. We are going to select two players from each team. So, you might have say David Wright as the best overall hitter for the Mets, but Carlos Beltran as the "clutch" hitter. So, the standings angle won't have as much impact, since we're looking at two players from the same game.

2. The "multiplier" that I have in Leverage Index will only be used to denote thresholds. I will not be weighting a 10.0 LI game more than a 3.0 LI game. Basically, any PA with an LI of 2.5 or 3.0 and above is part of the "crucial PA" pool.

Now, if you want to argue that a 2.0 LI PA in the last game of the season has more "tension" than a 5.0 LI PA in the first game of the season, or a 10.0 LI PA in a game where you are ahead by 10 games in the standing, that's fine, we can do that. But, keep the above two points in the back of your mind. We are not going to try to distinguish between the contexts that David Wright and David Ortiz face. We are distinguishing between the Wright/Beltran contexts.

[Bellhorn]

yep, on Jan 15 2008, 10:06 AM, said:

I actually have a couple ideas, but it's going to take me some time to work them out, and it'll probably be at least next week before I can get to it. Basically, I have in mind to come up with some kind of situational multiplier based on standings and games remaining, that would focus on the state of the team's season as of game time. Ideally, the closer a team is to an elimination game, the higher a number it would spit out, with the value rapidly declining as the team is either further ahead or behind, or as the number of games remaining increases, and maybe turning negative if the team has clinched or been eliminated. This multiplier could then be applied to any flavor of win probability or even just conventional stats to see whether the player performs any differently when the season (and not just the game) is on the line. If it could be made to produce a percentage with a max value of 1 for an elimination game, I think it would be pretty straightforward to "weight" a player's performance in critical games against his norm.

Off the top of my head, I would suggest multiplying LI for the given plate appearance by the change in playoff odds (e.g. from Baseball Prospectus) implied by the result of the game, scaled by some factor to make the numbers more manageable. In theory, this has the advantage of giving greater weight to games played against post-season rivals; it does have a practical disadvantage in that BP doesn't publish hypothetical playoff odds, but it should be easy enough to come up with a good estimate.

Perhaps one could classify a PA as clutch if either the in-game LI meets a certain threshhold, or if LI weighted by "change in playoff odds" multiplier meets the threshhold - this still allows for clutch situations based on the intrinsic value of the game once the season has been decided one way or the other.

In The Hardball Times annual, Bill James has an article on clutch that accounts for the opponent, the standings, and the point in the season, though he doesn't mention the precise formula that he used. Interestingly, Ortiz still did well in 2007 by this definition of clutch: 315/435/607 over 106 PA.

This post has been edited by Bellhorn: 15 January 2008 - 10:51 AM

[yep]

Tangotiger, on Jan 15 2008, 10:19 AM, said:

I'm not against the idea of incorporating the standings into this.

I want to make two important points:
1. We are going to select two players from each team. So, you might have say David Wright as the best overall hitter for the Mets, but Carlos Beltran as the "clutch" hitter. So, the standings angle won't have as much impact, since we're looking at two players from the same game.

2. The "multiplier" that I have in Leverage Index will only be used to denote thresholds. I will not be weighting a 10.0 LI game more than a 3.0 LI game. Basically, any PA with an LI of 2.5 or 3.0 and above is part of the "crucial PA" pool.

Now, if you want to argue that a 2.0 LI PA in the last game of the season has more "tension" than a 5.0 LI PA in the first game of the season, or a 10.0 LI PA in a game where you are ahead by 10 games in the standing, that's fine, we can do that. But, keep the above two points in the back of your mind. We are not going to try to distinguish between the contexts that David Wright and David Ortiz face. We are distinguishing between the Wright/Beltran contexts.

Sorry, I think I may have communicated my intent poorly.

RE: the distinction in your last paragraph, that's exactly what I'm looking for-- a way to look for performance differences during games where the season is on the line. With that in mind...

I will stipulate in advance that what I have in mind is completely unfair to potential clutch players on teams that do not breed "clutch" situations. The stuff I would be looking at basically excludes anyone on the Royals from consideration, because they never have an opportunity play in a "clutch" game (although they may have clutch moments in individual games).

BUT (and this is an important distinction, I think), I am not doing anything that would penalize them for being on a bad team, just saying that none of their plate appearances meet the criteria that we are filtering for. Kind of like if a hitter had never faced LHP.

So... what I am looking for is *not* Wright vs. Beltran, I am looking for Wright's production as weighted by the "clutchness" of the game. If Wright has not had many opportunities to perform in "clutch games," then we simply will not get a very strong read on him one way or the other.

RE: your point in paragraph 2-- What I have in mind is really just a sort of filter, or a way to produce weighted "splits" of performance in clutch/non-clutch games. It could be applied to any statistical measure in any way anyone saw fit. If, for instance, a game scores "1" on the clutch rating, then the player's numbers from that game go into the clutch column. If it scores "0" they go into the "regular" column. If the score is somewhere in-between, then the numbers are weighted proportionately (please bear in mind I still haven't figured out how to do any of this yet... ). "Multiplier" is a really poor word choice, now that I think of it, because it makes it sound like I am weighting the player's performance rather than the situation. I'm gonna change it in the original post.

In any case, I am not telling anyone what measure to use to evaluate a player's performance in "clutch games," just trying to identify which games are likely to bring out any hypothetical "Mr. October" superpowers. From there we can do whatever analysis seems like fun. Players who have been in more clutch situations will show more meaningful (but not necessarily better) results. Players who were never in a critical game will simply not make it to the list. My intent right now is just to collect a data set, not to define the best way to analyze it.

[Tangotiger]

Ah, meeting a double-threshhold, rather than multiplying, is more appealing.

After all, game 163 will have virtually all the PA more tension-filled, even if the LI is below 1.0, and some random game in the season has an LI of 10.0.

But, if we set certain threshholds, like any LI over 4.0, regardless of standings, all the way down to an LI of 0.5 for a game 163 situation, that will likely mimic how a person will perceive the tension, since not every player is solely focused on just making the playoffs (otherwise, why would Royals players show up?). There is a certain amount of tension a player perceives simply based on the game being played, and then an added tension based on the playoffs.

[yep]

Tangotiger, on Jan 15 2008, 11:04 AM, said:

Ah, meeting a double-threshhold, rather than multiplying, is more appealing.

After all, game 163 will have virtually all the PA more tension-filled, even if the LI is below 1.0, and some random game in the season has an LI of 10.0.

But, if we set certain threshholds, like any LI over 4.0, regardless of standings, all the way down to an LI of 0.5 for a game 163 situation, that will likely mimic how a person will perceive the tension, since not every player is solely focused on just making the playoffs (otherwise, why would Royals players show up?). There is a certain amount of tension a player perceives simply based on the game being played, and then an added tension based on the playoffs.

Yeah, absolutley, and my intent is not to discount your own or anyone else's excellent work in identifying "in-game" clutch situations, only to take a look at performance specifically in tense games and see whether players perform differently. Anyhow, it's still all pretty vague and fungible in my head, and there's obviously some work with a spreadhseet to do, so I am actively open to suggestions if anybody's got 'em.

Bellhorn's idea with the BP playoff odds might be promising if there were a way to access the archives, and I'd have to think through whether having the stuff weighted by adjusted standings is good or bad. For these purposes, I almost think that the adjusted standings flies in the face of the hypothesis we would be testing.

Cheers.