The Greatest of All Time
- Thread starter Deathofthebambino
- Start date
Super Nomario said:
3rd in pts/drive overstates how good the Pats' offense was because t[SIZE=14.4444446563721px]hey had great starting field position (ranked 2nd). [/SIZE] 15th in NY/A understates it, since it doesn't account for TDs or INTs, and one of NE's strengths was turnover avoidance (T1st for fewest turnovers, 3rd in TOs/drive) But they were 6th in ANY/A, which does include INTs and TDs. NE also finished 6th in offensive DVOA, 5th in passing offense DVOA, and 10th in yards/drive. They were 5th in Drive Success Rate (number of 1st-down series that result in a first down or TD). I don't really see the big disconnect between the per-play and drive stats.
I understand your larger point that we shouldn't confuse what we can easily measure with what we want to analyze (the Streetlight Effect), but I'm not convinced there's really that much difference between them here.
I really wish, as the thread starter, that I had more time to put together a full post of my thoughts on this topic, but that's just not in the cards. However, to take off briefly on the attempts to compare quarterbacks like Manning and Brady, I think folks are missing an interesting point, and one I brought up in another thread recently.
Instead of simply looking at Brady vs. Manning, or Montana, particularly, when it comes to playoff games, how about you look at their body of work vs. the quarterbacks they actually played against in those games. As was briefly mentioned by someone else, it's impossible to accurately compare Brady's playoff statistics to a guy like Montana or Manning solely because every playoff game is not played under the same exact conditions. Home/away, cold/warm, snow/rain, indoors/outdoors, etc. I generally hate football statistics in every way when trying to settle debates like this, but how can anyone possibly compare Brady's passing stats in -10 degree wind chill, or 30 degree driving rain storms against a guy like Manning playing indoors or Montana playing in Southern California? The passing game, more than any other, is affected by these things, and IMO, it's pretty clear that nobody in the history of the NFL has shown the ability to adapt and win in every type of conceivable situation, at home or on the road, like Tom Brady. I've seen Peyton Manning puke all over himself repeatedly in person when the weather turned shitty in Foxboro. I've never seen Brady do that because he went into a dome on the road. I've never seen anyone lead their team back in the middle of a blizzard in the playoffs like I've seen Brady do. Freezing rain? No problem.
I think someone put together the statistics of the opposing QB's in the playoff games against the Patriots and compared them to Brady in another thread, and the gap is eye-opening (even though, in a lot of those games, they were playing from behind and forced to throw, adding to their numbers).
Football is a game played in almost every type of conditions, and IMO, it's a huge part of the game, and has as much of an effect on who wins and who loses as almost anything else. Brady and the Patriots seem to routinely struggle against Miami on the road when they are scheduled in the early part of the season, but aside from that, I can't think of another situation that would lead me to say "I'd rather have someone else at quarterback due to the conditions."
I've seen the question asked a few times here before "If you had one quarterback to play one game for you, who would it be?" And some folks say Montana, some say Brady, some might say Rodgers, some might say Marino. But how about this question "If you had one quarterback to play one game for you, without knowing ahead of time if it was going to rain, snow, be warm or below zero or be indoors or outdoors, home or away, who would you pick?" That's the reality of the NFL, and IMO, there is only one right answer when you look at it like that, and it's Brady. That's what truly makes him the greatest of all time. Period.
Instead of simply looking at Brady vs. Manning, or Montana, particularly, when it comes to playoff games, how about you look at their body of work vs. the quarterbacks they actually played against in those games. As was briefly mentioned by someone else, it's impossible to accurately compare Brady's playoff statistics to a guy like Montana or Manning solely because every playoff game is not played under the same exact conditions. Home/away, cold/warm, snow/rain, indoors/outdoors, etc. I generally hate football statistics in every way when trying to settle debates like this, but how can anyone possibly compare Brady's passing stats in -10 degree wind chill, or 30 degree driving rain storms against a guy like Manning playing indoors or Montana playing in Southern California? The passing game, more than any other, is affected by these things, and IMO, it's pretty clear that nobody in the history of the NFL has shown the ability to adapt and win in every type of conceivable situation, at home or on the road, like Tom Brady. I've seen Peyton Manning puke all over himself repeatedly in person when the weather turned shitty in Foxboro. I've never seen Brady do that because he went into a dome on the road. I've never seen anyone lead their team back in the middle of a blizzard in the playoffs like I've seen Brady do. Freezing rain? No problem.
I think someone put together the statistics of the opposing QB's in the playoff games against the Patriots and compared them to Brady in another thread, and the gap is eye-opening (even though, in a lot of those games, they were playing from behind and forced to throw, adding to their numbers).
Football is a game played in almost every type of conditions, and IMO, it's a huge part of the game, and has as much of an effect on who wins and who loses as almost anything else. Brady and the Patriots seem to routinely struggle against Miami on the road when they are scheduled in the early part of the season, but aside from that, I can't think of another situation that would lead me to say "I'd rather have someone else at quarterback due to the conditions."
I've seen the question asked a few times here before "If you had one quarterback to play one game for you, who would it be?" And some folks say Montana, some say Brady, some might say Rodgers, some might say Marino. But how about this question "If you had one quarterback to play one game for you, without knowing ahead of time if it was going to rain, snow, be warm or below zero or be indoors or outdoors, home or away, who would you pick?" That's the reality of the NFL, and IMO, there is only one right answer when you look at it like that, and it's Brady. That's what truly makes him the greatest of all time. Period.
Here's something I realized today:
The guy who will eventually take over Tom Brady's spot as the GOAT probably isn't even in college yet.
edit: or it's Jimmy G.
The guy who will eventually take over Tom Brady's spot as the GOAT probably isn't even in college yet.
edit: or it's Jimmy G.
And also to be clear, I'm not saying weather is the only variable, nor home/away. There is also the ability of the players around him, which has been touched on. And the biggest one is, of course, the level of the defense they are playing against.
It would be just fantastic if there was one set of numbers that was able to take all of this stuff into account, and spit out a true comparison between QB's from different era's, but that's just not the case. Football statistics suck at settling this stuff because of the size of the samples (although in Brady's case, we're reaching some pretty damn good sample size when it comes to the playoffs) and the number of variables that can't be taken into account. At some point, you just have to watch the games, and look at the results and think about who you think would get it done and thus, who you'd rather have in that situation. I just can't think of any scenario where I'd pick a QB to lead my team over Brady. I'm not so blind as to dismiss other's arguments, but at this point (and for me, this was the case long before yesterday), I can't see anyone rationally being able to leave Brady out over someone else. He's won and won and won and won, in every situation imaginable, over the course of a decade and a half, with no signs of regression at this point.
It would be just fantastic if there was one set of numbers that was able to take all of this stuff into account, and spit out a true comparison between QB's from different era's, but that's just not the case. Football statistics suck at settling this stuff because of the size of the samples (although in Brady's case, we're reaching some pretty damn good sample size when it comes to the playoffs) and the number of variables that can't be taken into account. At some point, you just have to watch the games, and look at the results and think about who you think would get it done and thus, who you'd rather have in that situation. I just can't think of any scenario where I'd pick a QB to lead my team over Brady. I'm not so blind as to dismiss other's arguments, but at this point (and for me, this was the case long before yesterday), I can't see anyone rationally being able to leave Brady out over someone else. He's won and won and won and won, in every situation imaginable, over the course of a decade and a half, with no signs of regression at this point.
I knew I should have avoided this thread, but I can't help myself, because the more I think about it, the more aggravated I get. I consider myself a student of the game, and it's history, and it drives me crazy when people want to distill everything down to numbers.
Let's put the numbers aside for a second. Think about Brady in these terms. Who has ever been better at managing the game, knowing the situation and/or leading his team than Brady? Almost everyone here watches other teams play every week, and the difference between Brady and just about everyone else is on display every week, and it's that shit that we take for granted. I'll bet people can count on one hand the number of times Brady has made a big mistake that cost the Patriots a game, or a little mistake for that matter. He runs the two minute drill like it's second nature. You never worry about things like clock management, or him throwing a ball to a receiver short of the sticks, or dropping a snap, or causing an illegal procedure or false start when in the hurry up. He doesn't miss open receivers when the game is on the line, he doesn't fumble it away when he gets hit from behind, he knows where the pressure is coming from and is ready for it. How many times have you seen the Patriots have a "broken play" on offense in the past 15 years? He moves around in the pocket like most of us move around in our bedrooms.
All it takes is watching 25 other QB's every Sunday to see just how much better he is at ALL of this stuff than everyone else, and these aren't things that are just instinctual like Russell Wilson's scrambling ability or Rodgers ability to throw the deep ball or Elway's ability to throw a football 95mph. These are things that are drilled over and over and over and practiced until no thought is required any more. We are so, so damn lucky to see a guy do this stuff and make it look so easy. These are things that cost teams football games week in and week out. When you are 160-47 or whatever the crazy W/L numbers are, it doesn't happen because you are throwing for 300 yards and 3td's every week. It's because you can throw for 300 and 3 every week, but you also do all of the little stuff when it matters to put your team in a position to win. And really, this is all just the tip of the iceberg on why he's the greatest of all time.
Let's put the numbers aside for a second. Think about Brady in these terms. Who has ever been better at managing the game, knowing the situation and/or leading his team than Brady? Almost everyone here watches other teams play every week, and the difference between Brady and just about everyone else is on display every week, and it's that shit that we take for granted. I'll bet people can count on one hand the number of times Brady has made a big mistake that cost the Patriots a game, or a little mistake for that matter. He runs the two minute drill like it's second nature. You never worry about things like clock management, or him throwing a ball to a receiver short of the sticks, or dropping a snap, or causing an illegal procedure or false start when in the hurry up. He doesn't miss open receivers when the game is on the line, he doesn't fumble it away when he gets hit from behind, he knows where the pressure is coming from and is ready for it. How many times have you seen the Patriots have a "broken play" on offense in the past 15 years? He moves around in the pocket like most of us move around in our bedrooms.
All it takes is watching 25 other QB's every Sunday to see just how much better he is at ALL of this stuff than everyone else, and these aren't things that are just instinctual like Russell Wilson's scrambling ability or Rodgers ability to throw the deep ball or Elway's ability to throw a football 95mph. These are things that are drilled over and over and over and practiced until no thought is required any more. We are so, so damn lucky to see a guy do this stuff and make it look so easy. These are things that cost teams football games week in and week out. When you are 160-47 or whatever the crazy W/L numbers are, it doesn't happen because you are throwing for 300 yards and 3td's every week. It's because you can throw for 300 and 3 every week, but you also do all of the little stuff when it matters to put your team in a position to win. And really, this is all just the tip of the iceberg on why he's the greatest of all time.
Deathofthebambino said:
And it drives me crazy when people make silly homerish "eye-test" arguments that have no empirical support. Sure, Brady may do all the things you mentioned well, but this is a GOAT discussion -- all the candidates do all those things pretty well. We're talking about fine margins here. You have no idea whether Brady is better than Montana or Manning at avoiding illegal procedure penalties.
Your post is full of entirely unsupported empirical claims. It turns out that many of them are wrong. For example, in your paean you mention that Brady doesn't fumble when he gets hit from behind. This is something we can investigate with data.
Here are fumble rates for what I would probably consider the top 5 QBs (computed as (passing attempts + rushing attempts + sacks)/fumbles)
Brady: 83.4
Montana: 116.3
Manning: 128.4
Young: 76.9
Marino: 81.1
Of course this is not definitive -- we would also want to know how often these guys got hit. Manning's fumble rate looks great, but perhaps this is partly because Manning is other-worldly at avoiding sacks, and so he gets hit much less and therefore doesn't fumble as much -- and in fact it turns out that this is exactly the case.
If we adjust the fumble rates by sack rate, we can get fumbles per sack:
Brady: .25
Montana: .16
Manning: .26
Young: .18
Marino: .40
Now Montana and Young appear much better at avoiding fumbles. Young's fumble-itis seems to have been due to getting hit much more often (he has a very high career sack rate of 7.9%, because he was terrible at avoiding sacks early in his career -- he had a sack rate of 11.2% through his first six seasons in limited playing time). Marino looked identical to Brady before, but now we can see that's just because he was great at avoiding sacks (he was famous for this in his day), and that when he got hit he fumbled much more often than the others. This analysis has flaws (for example, [SIZE=14.4444446563721px]it would be better to separate out fumbles from being sacked and fumbles from runs and botched snaps, or, failing that, at least to not count kneel-downs in the rushing attempts, but the p-ref data isn't that granular). But it's at least a starting point, which "it seems like Brady never fumbles when he gets hit from behind, so he's the bestest ever!" is not.[/SIZE]
We could go through this exercise with all of the nonsense in your posts. Is Brady especially good compared to other all-timers at the 2-minute drill? P-ref tracks game-winning drives and fourth-quarter comebacks, and somewhere Scott Kascmar has published fourth-quarter comeback success rates (although I can't find it right now). Is Brady unusually good at playing in cold weather, after adjusting for opponent and home field? There is data out there on that. Are the Pats under Brady especially good at avoiding illegal procedure or false start or delay of game penalties? nflpenalties.com tracks penalty data going back to 2009. You don't just get to make facts up, and then make exaggerated claims based on the "facts" you just made up. At least if you want others to take you seriously and to convince people not already convinced.
Deathofthebambino said:
Brady road postseason: 118/198, 59.6%, 1447 yards, 7/6 TD/INT, 81.4 rating
Oppo in those games: 126/214, 58.9%, 1657 yards, 6/9 TD/INT, 75.2 rating
Brady in Super Bowls: 164/246. 66.4%, 1605 yards, 13/4 TD/INT, 95.3 rating
Oppo in those games: 135/223, 60.5%, 1843 yards, 13/7 TD/INT, 93.3 rating
Brady also has Super Bowl records for most completions in a game (he's #1 and #2) and his 9 straight completions on Sunday tie him for 6th longest streak. He's #1 on that list as well with 16 in SB XLVI. The 4 TD passes also ties him for 3rd most in single game.
coremiller said:
I should have expected this in this place because it's pretty well established that folks are married to numbers, and can't read very well. All of this would be great if I had actually argued that Brady fumbled less than anyone else. Except for the part that I, umm, didn't. You extrapolated that from a longer post from this one sentence:
"He doesn't miss open receivers when the game is on the line, he doesn't fumble it away when he gets hit from behind, he knows where the pressure is coming from and is ready for it."
I bolded the qualifier that was meant to apply to the whole sentence. And you know how I know that's true? Because I have the numbers to back it up, which are below. You know how else I know? Because I've seen every single one of Brady's games, including more than half in person, and I'm capable of using my "eyes" to make observations too.
Brady: 160-47
Montana: 117-47
Manning: 179-77
Young: 94-49
Even assuming I was making the argument you think I was making. You think the only flaws in that shit are because p-ref isn't granular enough to separate out botched snaps or kneel downs? Seriously? How about the fact that one guy played a shit ton of football games in freezing temperatures with a ball that feels like a boulder, or pouring rain or snow, while the others played in domes and nice weather for most of their careers? How does your precious empirical data account for that? Whether or not it can account for the difference between a botched snap or a kneel down or a handoff is about the 900th reason the conclusion is faulty. In the end, you managed to distill the difference between these guys down to a tenth of a point, and you think it proves anything, when the methodology itself can't possibly take into account dozens of variables that affect the numbers? You don't want to use your eyes to learn about the game or make observations and form conclusions, then so be it. Just go with W/L record and call it a day. At least that one is about as honest as it gets.
If you think you could go through this exercise with all of my points, feel free. I hope to God you're better at it, and have better information than the shit you just posted about fumbles because that showed me absolutely nothing.
You don't think I've read Scott Kascmar's study? Here it is: http://www.coldhardfootballfacts.com/content/captain-comeback-career-records-the-clutch-for-active-qbs/16571/
Wasn't all that hard to find either. You know who is #1 in active leaders for comeback success rate as of the date he published that in 2012? Yeah, Brady, even though he spent a good deal of time trying to use other factors besides your precious numbers to diminish the gap between Brady and Rodgers. He even went so far as to post a "case study" using Brady's failure to bring the Pats back in the 2008 Super Bowl. That guy is nothing, if not transparent. Numbers be damned.
Here's the thing. I've done the homework. I know what the numbers say, and I know what they don't say. I've been around here long enough to know that if you post bullshit, someone will call you on it. When you feel like you have that "gotcha" moment to prove something I said is false, go ahead and go for it. I'm always open to a good debate. Until then, I'm happy to state what I know to be facts. I'll let others try to disprove them, if they can.
By the way, this is just not all that true. Steve Young was a scrambling/running QB before we had guys that specialize in it like Russell Wilson. If I'm not mistaken, a designed running play that went for a loss by a QB back then was deemed a sack. It's why Randall Cunningham led the league in sacks taken year in and year out, while also leading the league in rushing for QB's. Those stats wouldn't look anything like they do under today's scoring.
Your statistics are worse than the eye test.coremiller said:
Possible confounds off the top of my head:
-some QBs face rushers better coached to strip the ball.
-holding longer means more strips
-pocket motion/running gives more strips
-changes in rules about hitting the QB means players try for strip more
-weather
Come on. Half of the battle when using numbers is knowing when they are appropriate. Baseball yes on offense and pitching, weaker on defense. Football, rarely so.
Edit: not to mention stats issues: give us a std error on those rates. And do they vary over time?
I think the numbers add something to the conversation and enjoy coremillers posting.
I think its impossible to make a definitive statement about who is the best player ever or coach ever or anything in the NFL. So much is team and context dependent. He numbers aren't as definitive as in baseball, but they're at least as helpful as empirical observations.
I think its impossible to make a definitive statement about who is the best player ever or coach ever or anything in the NFL. So much is team and context dependent. He numbers aren't as definitive as in baseball, but they're at least as helpful as empirical observations.
The problem is more of statistical hubris than anything else... like claiming that a rebuttal to a point about interceptions that involved INT rate is cherry picking and then throwing down this response:Stitch01 said:
As kolbitr mentioned, citing a statistic that doesn't include the biggest sample of lousy play is just a little problematic (especially more so when claiming it is a "comprehensive analysis" that comes out "only one way").coremiller said:
1) Your initial sentence was poorly structured. Grammatically it reads that the qualifier applies only to the clause before the comma. Don't get mad at me because you didn't write clearly.
2) You are clearly are not bothering to read what I wrote. Nowhere did I say that fumble rate data "distills the differences between these players down to a decimal point." The data I posted was just to answer a specific question: how often do these QBs fumble when hit. I didn't say anything about how valuable that skill is compared to other skills, or how that skill correlates with other skills. Nowhere did I say that the data was the final answer to the question, or accounted for every possible confouding variable. Nowhere did I suggest that the list of problems with the data I myself identified (which, btw, indicates that I obviously don't view the data as the end of the issue) were exhaustive. Of course it doesn't, and isn't. All I said was that the data is a useful starting point. I'm well aware of the limitations of the data. That doesn't mean the data entirely lacks value.
3) Your response to a statistical argument is to argue a bunch of confounding variable issues and then ... cite won-loss record? As if won-loss record doesn't have 1000x more confounding variable problems than the fumble rate data? Seriously? I would hope we're all sophisticated enough to know that QB Winz! is not a good argument. If you disagree, I'll just give up as I'm clearly wasting my time.
4) Some of the confounding variable issues could potentially be solved by more sophisticated data analysis. For example, with more/better data and statistical techniques, you could adjust for era and opposition and weather and home field (although there might be sample-size issues the narrower you go and the more adjustments you do). I'm not sure it would help, because my hunch is that a lot of it washes out over the sample of a 10+ year career, but if you disagree, the proper response is to do the analysis and prove it, not to just wave your hands and dismiss the data altogether.
5) Kacsmar's data is useful (his analysis is not, he's a terrible analyst but his data collection has value), thank you for linking to it (although it's of limited utility for historical evaluations because it only covers active players) You should have just started with that and saved us the trouble.
6) I don't think designed QB runs that gain negative yards are counted as sacks, or ever have been. For example, here is the play by play for Super Bowl VIII: http://www.pro-football-reference.com/boxscores/197501120min.htm At 1:05 in the second quarter, it shows "Terry Bradshaw for -1 yards", which is a different description than it uses for sacks. Failed scrambles get counted as sacks, and running QBs will have more failed scrambles because they try to scramble more, but that's different, those are sacks. There are probably also some scorer mistakes where the scorer confused a QB run for a failed scramble (like on a QB draw), but that's not the same as a general practice. And Young's early-career sack rates are actually quite similar to today's running QB sack rates -- Young had a sack rate of 9.6 over his first 1104 attempts, while Wilson's sack rate so far in 1252 attempts is 8.7, and Kaepernick's in 1117 attempts is also 8.7.
2) You are clearly are not bothering to read what I wrote. Nowhere did I say that fumble rate data "distills the differences between these players down to a decimal point." The data I posted was just to answer a specific question: how often do these QBs fumble when hit. I didn't say anything about how valuable that skill is compared to other skills, or how that skill correlates with other skills. Nowhere did I say that the data was the final answer to the question, or accounted for every possible confouding variable. Nowhere did I suggest that the list of problems with the data I myself identified (which, btw, indicates that I obviously don't view the data as the end of the issue) were exhaustive. Of course it doesn't, and isn't. All I said was that the data is a useful starting point. I'm well aware of the limitations of the data. That doesn't mean the data entirely lacks value.
3) Your response to a statistical argument is to argue a bunch of confounding variable issues and then ... cite won-loss record? As if won-loss record doesn't have 1000x more confounding variable problems than the fumble rate data? Seriously? I would hope we're all sophisticated enough to know that QB Winz! is not a good argument. If you disagree, I'll just give up as I'm clearly wasting my time.
4) Some of the confounding variable issues could potentially be solved by more sophisticated data analysis. For example, with more/better data and statistical techniques, you could adjust for era and opposition and weather and home field (although there might be sample-size issues the narrower you go and the more adjustments you do). I'm not sure it would help, because my hunch is that a lot of it washes out over the sample of a 10+ year career, but if you disagree, the proper response is to do the analysis and prove it, not to just wave your hands and dismiss the data altogether.
5) Kacsmar's data is useful (his analysis is not, he's a terrible analyst but his data collection has value), thank you for linking to it (although it's of limited utility for historical evaluations because it only covers active players) You should have just started with that and saved us the trouble.
6) I don't think designed QB runs that gain negative yards are counted as sacks, or ever have been. For example, here is the play by play for Super Bowl VIII: http://www.pro-football-reference.com/boxscores/197501120min.htm At 1:05 in the second quarter, it shows "Terry Bradshaw for -1 yards", which is a different description than it uses for sacks. Failed scrambles get counted as sacks, and running QBs will have more failed scrambles because they try to scramble more, but that's different, those are sacks. There are probably also some scorer mistakes where the scorer confused a QB run for a failed scramble (like on a QB draw), but that's not the same as a general practice. And Young's early-career sack rates are actually quite similar to today's running QB sack rates -- Young had a sack rate of 9.6 over his first 1104 attempts, while Wilson's sack rate so far in 1252 attempts is 8.7, and Kaepernick's in 1117 attempts is also 8.7.
Agreed on your second point.Stitch01 said:
As for your first: calling people idiots because they distrust the numbers and use their eyes is what I criticized. The numbers are interesting nuggets. They are of similar quality to a statement that "Bogaerts is 10 for 12 in away games against lefty starters in April". Conversation starters, not enders.
To be constructive, if one wanted to find interesting football numbers, I'd think looking for Levitt/economics-style clean identification would be the way to go. Everything else is too context-dependent.
bradmahn said:
I didn't say that DVOA was a comprehensive analysis, and I repeatedly acknowledged its limited sample. Earlier in the thread I linked to career playoff AY/A data, which has Montana way ahead of the other GOAT candidates (but 7th all time behind Bart Starr, a bunch of modern guys, and Joe Theismann) even before era adjustments, as well as Chase Stuart's study of playoff QB performance, which does adjust for era, opponent, and leverage (so that Super Bowls are worth more than wild card games) and finds that Montana is miles ahead of every other QB. I'll post those links again:
http://www.pro-football-reference.com/play-index/pgl_finder.cgi?request=1&match=combined&year_min=1960&year_max=2014&season_start=1&season_end=-1&age_min=0&age_max=99&game_type=P&league_id=&team_id=&opp_id=&game_num_min=0&game_num_max=99&week_num_min=0&week_num_max=99&stadium_id=&game_day_of_week=&game_month=&game_location=&game_result=&handedness=&is_active=&is_hof=&c1stat=pass_att&c1comp=gt&c1val=200&c2stat=&c2comp=gt&c2val=&c3stat=&c3comp=gt&c3val=&c4stat=&c4comp=gt&c4val=&c5comp=&c5gtlt=lt&c6mult=1.0&c6comp=&order_by=pass_adj_yds_per_att
http://www.footballperspective.com/the-best-playoff-quarterbacks-in-the-super-bowl-era/
I would still like to see a comprehensive, non-cherry-picked objective analysis that shows that Tom Brady was a better playoff QB than Joe Montana. Nobody has done one yet.
You make a false assumption here.coremiller said:
There may not exist ANY objective analysis that shows Brady was better or that shows Montana was better.
Some questions cannot be answered with available data. That doesn't mean someone who's smarter can come up with a way to answer them if only they manipulate the data the right way. It means the limited data is insufficient to determine truth from the highly contextual game situations.
crystalline said:
Data doesn't necessarily answer the question. But when the gap in performance is that big after making the obvious era/opponent contextual adjustments (and the gap in playoff performance is pretty big), I think the burden is on those arguing against the data to show what contexts aren't being accounted for and how that would change things, and to demonstrate why we shouldn't follow the data. Non-specific objections that that "the data is limited" and "football is contextual" and "some questions can't be answered by data" aren't good enough.
Data either disprove or fail to disprove a null hypothesis. If data fail, one relies on one's eyes and judgement. My view is your numbers fail to disprove DOTB's hypothesis.coremiller said:
crystalline said:
In addition to Stitch's point, which is right on:
1) I think you have the burden of proof backward. DOTB's hypothesis isn't the null, it's the relationship that needs to be proven. The null hypothesis for every player should be that they are not the GOAT. The burden is on anyone asserting a GOAT candidate to show that is the case. You don't get to rig the game by assuming what you're trying to prove, daring anyone who disagrees with to prove otherwise, and then rejecting contrary data for being insufficiently conclusive. If I posted that Joe Flacco was the greatest QB of all time because I watched him play brilliantly, and rejected all the contrary evidence as problematic data with confounding variables issues and therefore "insufficient to disprove the null hypothesis" that Joe Flacco is the GOAT, everyone would rightly laugh me out of the room.
2) I agree that IF data fails, you rely on observation and judgment as a substitute. That's not what you're doing. You're using observation and judgment to show the data fails. "The data is inconclusive, so I'll rely on what i saw" is very different from "The data says X, but I saw Not-X, therefore Not-X".
I think we are now rehashing what was covered above, but one more comment from me:
I am not using observation and judgement to show the data fails. I and others enumerated several potential confounds above (data was put forth to show fumble rate; confounds raised were weather, variation in offensive style, etc.). The confounds must be addressed, otherwise the data fails.
Since the data failed to show what it was put forth to show, what we have left is observational.
Analogy: UZR. With only play by play data you cannot say anything useful about defense in a one year sample, because the data are insufficient. No improvement on UZR would help with those data. In the absence of other data, the best way to judge defense over a one year sample is good old observational scouting. Which is what teams did before they got new data (camera based tracking of motion and position and precise batted ball trajectory that the public does not have access to). The eyes and brains of human experts are better than data for many questions.
That'll be my last here.
I am not using observation and judgement to show the data fails. I and others enumerated several potential confounds above (data was put forth to show fumble rate; confounds raised were weather, variation in offensive style, etc.). The confounds must be addressed, otherwise the data fails.
Since the data failed to show what it was put forth to show, what we have left is observational.
Analogy: UZR. With only play by play data you cannot say anything useful about defense in a one year sample, because the data are insufficient. No improvement on UZR would help with those data. In the absence of other data, the best way to judge defense over a one year sample is good old observational scouting. Which is what teams did before they got new data (camera based tracking of motion and position and precise batted ball trajectory that the public does not have access to). The eyes and brains of human experts are better than data for many questions.
That'll be my last here.
Hey Death... I agree with the substance of your post. As mentioned by a previous poster, you did make a minor mistake.. Montana played in San Fransisco which is in Northern CA. San Fransisco has been known to get a rainstorm or two, so it isn't like Montana played in perfect weather. However, during the majority of his career the NFC West consisted of the LA Rams, the Falcons and the Saints. Before the Georgia Dome was built the Falcons played outdoors at Fulton County Stadium. Even considering the occasional rainstorm in Atlanta or San Fransisco, it is clear that Montana had the benefit of consistently better weather conditions than Brady. Anyway, I did not mean to distract or take away from your overall point which I agree with.Deathofthebambino said:
richgedman'sghost said:
So you reject the fumble data for sample size and confounding variable reasons, but you see no reason to question DotB's conclusions, which are a) based on a much smaller sample (only the games watched by DotB vs. every play in the p-ref database), subject to all kinds of obvious perceptual and cognitive biases, and are without any systematic basis (which could be done without data analysis, if e.g. you went back and watched lots of film of every great QB)?
It's almost like there's a double standard.
I love reading the back and forth and the numbers to provide context to the debate. My problem is, as death alluded to, when someone acts like a complete dick trying to make someone look stupid or to prove their own intelligence (and that's what coremiller did, then he doubled down after being called out on it) I have been reading this board now for about 14 years. Something that I've noticed that happens all the time is people with not that many posts jump on someone (as death stated, sort of a gotcha method) for being stupid then use a bunch of data to completely drive their point into the ground, basically hoping to gain respect from the older posters here. Sometimes it works...the person who was jumped on was a complete moron and digs himself into an even deeper hole. Sometimes...well sometimes what just happened to coremiller is the outcome.Stitch01 said:
I am actually learning a lot from coremiller's post (as I did from death's post), I just think that there is a little too much of jumping down people's throats when it's not deserved around here
coremiller said:
First thing I want to say is that while I appreciate some folks coming to my defense, it's quite ok guys. I'm a big boy and I don't take anything here personally at all. I've been around here for a long, long time and had many arguments (and argue for a living to an extent), so I understand the back and forth of these arguments completely. I know how they start, proceed and end because I've been involved with or read thousands and thousands of them on this site for years.
Coremiller, I'll start with #2 first. I have no idea why you get the idea that I'm saying anything other than "The data is inconclusive so I'll rely on what I saw." That's exactly what I'm saying and have always said. I promise you there is almost no data on any of these subjects that I haven't read or studied. I've come to the conclusion that it's all so terrible that I've chosen now to not rely upon it, and instead, go to observational data. I can assure you that I'm not the type to dismiss out of hand any data, and I've thousands of posts around here using stats and not using stats, with football and in every other sport. I'm just saying when it comes to judging Tom Brady as the GOAT, stats will probably put him in the top 5 in almost every discussion, it's the observational stuff that clears him to the top.
I'm not daring anyone who disagrees with me to prove otherwise. Well, I guess I am, at some level. However, I have no problem rejecting contrary evidence when it's just so useless. It's why I wrote what I did in the first post you quoted of mine. Football statistics absolutely suck at telling the story. We aren't talking about minor variables that are unaccounted for, like the difference between whether the stats are granular enough to differentiate between a kneel down or a bad snap or whatever. We're talking about massive, huge qualitative issues with the data. We're talking about fumbling without being able to differentiate whether one guy was playing in a 30mph, driving rainstorm while another was playing in a climate controlled dome. Those aren't "confounding variables," they are variables that when ignored render the data not only useless, but misleading. And IMO, misleading statistics are worse than no statistics in some respects.
Football at it's very basic level is a game involving 11 guys. The success or failure of nearly every single play is the result of all 11 guys doing or not doing their job. The statistics just can't adjust for that, just as they can't adjust for all of the other variables we've discussed and about a million more we haven't. Believe me when I say that I appreciate statistical analysis. It's why I found and joined this site in the first place over a decade ago, and when it comes to baseball, I'm a true believer. Football simply doesn't work the same way. Sure, the statistics have come a long way, but even the advanced statistics that people like to point to are, IMO, just putrid at telling the story. It's why I threw out W/L record (with tongue in cheek) in my prior post. I honestly don't believe there are that many statistics in football that are any better than wins and losses, which has massive holes, as we all agree on. I won't even get into the era adjusted or park adjusted stuff. I could talk about this for days, but at the end of it, we won't get anywhere. You can use whatever data you want to prove Montana is better, I can find holes in any data you provide because the statistics just aren't able to tell the story or I can simply find different data (W/L record, counting stats, playoff appearances, come from behind wins, etc.) that says Brady is better and you'll find the holes in that. So, where does it leave us? With observational data.
And with that, we come full circle. Football has to be judged based on what you see. The numbers are fun to look at, and in some cases, can provide some context to what happened, but at the end of the day, they aren't conclusive, not even close. And when they go from inconclusive to misleading, that's when I feel compelled to say something at times. I think there probably is an argument for Montana over Brady (in the playoffs), but I don't think the numbers are going to tell the story because I can always find a number to prove Brady is better and you can always find a number to prove Montana is better. Who gets to decide which one of us has the "better" numbers?
Ha, was actually just about to post that one poster I really wouldn't worry about tone pissing off or derailing a thread with is Deathofthebambino, I don't always agree with him but he gets this message boarding thing.
Death, the biggest problem I have with your argument is 1) the insistence that there is a right answer to the question when everything in football is so team, system, and context dependent (which is an argument not to weight the data like we would for baseball offensive numbers, you cant just plug in a third baseman on another team, make some adjustments, and make a great guess at how good they are) and 2) the observational argument including "Ive seen all of Brady's games and attended half of them" as part of the argument because you certainly didn't attend or watch all of Montana, Marino, or Manning's games which institutes all sorts of cognitive biases.
Death, the biggest problem I have with your argument is 1) the insistence that there is a right answer to the question when everything in football is so team, system, and context dependent (which is an argument not to weight the data like we would for baseball offensive numbers, you cant just plug in a third baseman on another team, make some adjustments, and make a great guess at how good they are) and 2) the observational argument including "Ive seen all of Brady's games and attended half of them" as part of the argument because you certainly didn't attend or watch all of Montana, Marino, or Manning's games which institutes all sorts of cognitive biases.
Stitch, If the question we're talking about is whether or not Brady is the greatest of all time, then I'll bite that there is no right answer, but it's most certainly not the wrong answer. At the end of the day, it's an opinion. Even if we did have a statistic that could show, beyond a shadow of a doubt, that Tom Brady is better than everyone else, whether or not he's the greatest of all time would still be just an opinion, and can't, by definition, be "right" or "wrong." That's the same in every sport. However, I think some opinions are better than others, and whether you want to use observational data or statistics to bolster the opinion doesn't change that at all. My opinion is that Brady is the GOAT, and I think that opinion is based on both empirical, statistical data, and my knowledge of the game and observations of him as a player, as compared to other QB's. Like I said previously, there are very few statistics of relevance that folks can find that wouldn't put Brady near the top, and when you couple that with the observational data, I don't know how you come to another conclusion, but I'm all ears. I'm just not willing to toss aside all observational data because of some statistical analysis with holes the size of a truck in them, and that's what I find disheartening in a lot of the debates on the subject. At some point, folks really need to just put down the pen and paper and watch, and tell me what they see.
I've asked it before and I'll ask it again. If you had one quarterback to play a game for you tomorrow, without knowing ahead of time what kind of players he'd have around him, where it would be, what the weather would be like and who the opponent is, who would you pick? I don't know, based on statistics and observations, how anyone could come to a pick other than Brady, and if that's not as close to a definition of GOAT, something with basically no definition, I don't know what is.
Just to be clear, I would actually argue that seeing the games in person provides less value than seeing them on TV in most instances. While there are some things you can see, hear and feel in person that you can't on television, I would argue that television provides a much better perspective to judge the play on the field for many reasons. But, i will tell you that I've seen probably all but a handful of Manning's games in his career (mostly in fast forward), including about a dozen in person, and I've seen way more of Montana and Marino's games than most would think possible, because I seek them out. I try to catch every single NFL game every week in one way or another, and if I don't see them, I read every play by play and box score of every game (I do that for FBS and FCS football too, and I've tracked the scoring plays of every high school game in Eastern Mass in notebooks going back to 1984, it's a sickness). That said, I'm not and never was trying to say "I watch a lot of football, so I'm right." Far from it. I'm just trying to say that I take the game very seriously, and view myself as a student of it, constantly trying to learn more and more about it. I know the statistics inside and out. I'm not tossing them aside because I think "Stats are stoopid!!!!" or anything of the sort. I just think, in most cases, they suck when it comes to football, even though I've spent more time reading and tracking football stats than anyone should ever even consider doing.
I've asked it before and I'll ask it again. If you had one quarterback to play a game for you tomorrow, without knowing ahead of time what kind of players he'd have around him, where it would be, what the weather would be like and who the opponent is, who would you pick? I don't know, based on statistics and observations, how anyone could come to a pick other than Brady, and if that's not as close to a definition of GOAT, something with basically no definition, I don't know what is.
Just to be clear, I would actually argue that seeing the games in person provides less value than seeing them on TV in most instances. While there are some things you can see, hear and feel in person that you can't on television, I would argue that television provides a much better perspective to judge the play on the field for many reasons. But, i will tell you that I've seen probably all but a handful of Manning's games in his career (mostly in fast forward), including about a dozen in person, and I've seen way more of Montana and Marino's games than most would think possible, because I seek them out. I try to catch every single NFL game every week in one way or another, and if I don't see them, I read every play by play and box score of every game (I do that for FBS and FCS football too, and I've tracked the scoring plays of every high school game in Eastern Mass in notebooks going back to 1984, it's a sickness). That said, I'm not and never was trying to say "I watch a lot of football, so I'm right." Far from it. I'm just trying to say that I take the game very seriously, and view myself as a student of it, constantly trying to learn more and more about it. I know the statistics inside and out. I'm not tossing them aside because I think "Stats are stoopid!!!!" or anything of the sort. I just think, in most cases, they suck when it comes to football, even though I've spent more time reading and tracking football stats than anyone should ever even consider doing.
DotB, I don't actually disagree with much here, except that I think a lot of the statistics while imperfect, add value to the discussion, so long as you recognize their limitations. [SIZE=14.4444446563721px]I learn things from them. [/SIZE]Even misleading statistics can be educational, because figuring out how they are misleading often teaches you more than the statistics themselves do. Mostly what bothers me are your claims that "there is only one right answer when you look at it like that, and it's Brady" and "I'm not so blind as to dismiss other's arguments, but at this point (and for me, this was the case long before yesterday), I can't see anyone rationally being able to leave Brady out over someone else." I just don't think it's that clear cut, and I definitely don't think your observational evidence proves those conclusions. I guess what I don't understand is how you can be so thoroughly aware of all the methodological problems with statistical analysis but not similarly discount observational analysis, which has its own similar problems.Deathofthebambino said:
IMO, football is the ultimate team sport. Dan Marino, by almost any measure, was one of the all-time great quarterbacks. But he never won a ring. Trent Dilfer was pretty much a stiff, but he got a Lombardi. Roethlisberger is a solid QB, but he's not an all-time great, and yet he has more rings than Peyton Manning or Brett Favre or Aaron Rodgers - three guys who are legends.
Team style of play comes into the equation as well. If Lynch had run the ball in from the one, Seattle wins the Super Bowl and Wilson gets another ring, but Lynch gets the stat. Wilson completes it and he gets another ring and his stats look even better. Wilson throws the interception and he loses a ring and gets a big negative mark on his stat sheet. Teams that like to pound it in on the ground mean that the QB has fewer TD passes than others. But are the QBs with fewer TD passes necessarily "worse" than those with more?
What about MVP awards? They have to count for *something*, but goodness, when (yes, different sport; same principle) Ivan Rodriguez wins the MVP over Pedro Martinez, it's hard to take MVP awards seriously. And yet…..they absolutely count.
Data in football is SO hard to really understand clearly. A 3-yard pass turns into a 78-yard touchdown due to the brilliance of the receiver, yet the QB gets full credit for all 78 yards. Conversely, a perfectly thrown ball that goes through the hands of the receiver and deflects to a DB goes not against the receiver, but against the QB for the interception.
This is necessarily going to be somewhat subjective, is what I'm trying to say. Everyone has his own criteria by which to judge, and unless the criteria is something patently absurd ("he has calm eyes"), it's tough to be too critical.
Team style of play comes into the equation as well. If Lynch had run the ball in from the one, Seattle wins the Super Bowl and Wilson gets another ring, but Lynch gets the stat. Wilson completes it and he gets another ring and his stats look even better. Wilson throws the interception and he loses a ring and gets a big negative mark on his stat sheet. Teams that like to pound it in on the ground mean that the QB has fewer TD passes than others. But are the QBs with fewer TD passes necessarily "worse" than those with more?
What about MVP awards? They have to count for *something*, but goodness, when (yes, different sport; same principle) Ivan Rodriguez wins the MVP over Pedro Martinez, it's hard to take MVP awards seriously. And yet…..they absolutely count.
Data in football is SO hard to really understand clearly. A 3-yard pass turns into a 78-yard touchdown due to the brilliance of the receiver, yet the QB gets full credit for all 78 yards. Conversely, a perfectly thrown ball that goes through the hands of the receiver and deflects to a DB goes not against the receiver, but against the QB for the interception.
This is necessarily going to be somewhat subjective, is what I'm trying to say. Everyone has his own criteria by which to judge, and unless the criteria is something patently absurd ("he has calm eyes"), it's tough to be too critical.
A few years ago when Phil Jackson was on his farewell legacy tour, he started a Jordan vs. Kobe debate. My argument for Montana is the same for Kobe. There is at least one ring for Joe (more for Kobe) where he wasn't the best player on his team forget the best player in the world.
coremiller said:
I don't have an issue discounting observational analysis, but I find that when it comes to football, it's much harder to do. For example, let's look briefly at the studies you talked about above that conclude that Montana is world's better than everyone else. To accept the conclusions from those studies, you have to accept as gospel that Adjusted yards per pass attempt is basically the greatest statistic when judging a quarterback's postseason abilities. If I remember correctly, this is a statistic that provides for an arbitrary 45 yard penalty for interceptions and 20 yard bonus for touchdowns. Don't really remember why, don't really care, just know that it's because someone chose that. Why is it the one they used in those studies? Because it's his "favorite" and apparently, is the stat that has the highest correlation to wins. Not sure why we don't just go back to wins and losses if that's the goal, but I'll play along a bit more. But the bottom line is that the entire statistical analysis is based on yards per pass attempt or adjusted net yards per pass attempt. The problem is those stats both have the exact same problems that almost every other statistic does that we talk about, weather, opponents, teammates, etc. The entire study starts with a flawed statistic as it's basis.
Then from there, he "adjusts" for opponent and era in one swoop. How does he do that? By looking at the opponents net adjusted yards per pass attempt. That's it. Nothing else. Then they factor in "leverage" which is nothing more than assigning a value to each game based on the number of teams left at that moment and thus, their "odds" of winning the Super Bowl. For example, at the super bowl level, there are only two teams left, so the expected delta is 50%. At the Championship game level, it's a 25% because there is 4 teams left. I don't think I have to spend too much time explaining why there are inherent flaws in assuming just because a team is in the conference championship, the odds of them winning the Super Bowl is not exactly 25% for every team, every year.
I'm not trying to get into a battle of semantics over the methodology of this study, and if I made some minor mistake in my paraphrasing it from memory, I apologize, but that's not the point. The point is that these are fun little exercises that in the end spit out a list, and we all know everyone loves lists, but these lists don't really tell us anything except that if you accept what the author is choosing to use as his formula, that's the answer you get. It's not conclusive as to anything else, and when I see folks cite something like that, it gets the hair on the back of my neck to stand up because there are a whole lot of folks who see that list and just assume, "Hey, this guy did a ton of work and spent a lot of time on this, so I guess Montana really is the best," when a study like that couldn't possibly prove that. I mean shit, any equation/study or whatever you want to call it that concludes that Jim Plunkett's 1980 playoff season was the 2nd best playoff season EVER for a QB in the NFL just doesn't even pass the smell test, never mind when you dig deeper into the methodology. For those that don't know, Plunkett was 49-92 (53% completion percentage) for 839 yards, 7td's and 3int's and a 96.1 QB rating, through 4 games. He was absolutely dismal in the first two games of that playoffs, played ok in the conference championship, and then went crazy in the Super Bowl, and because of his high pass per attempt and great Super Bowl (which is weighted so much heavier than every other game, even though the playoffs are one and done anyway), he ends up at #2, higher than guys like Aaron Rodgers in 2010, Aikman in 92, and even Montana in 89.
Anyway, I was trying to keep that brief, but obviously failed a bit. For me, observational data is so much more important and easier to confirm or deny than these kinds of studies, which is why I've come to the point where I prefer them. When I write that Tom Brady doesn't get sacked and fumble with the game on the line, or throw a pick that results in his team losing, there may not be a stat for it, but it's not hard to counter. The guy only has 48 losses in his career, including the post-season. How many times did he blow the game down the stretch in those losses? I don't need a statistic to tell me it's not that many by comparison to a bunch of guys with twice as many losses. I can go by memory (for example, the Pats lost 4 games this year, one of which he didn't play, 2 of which they were blown out, which leaves Green Bay, and he did neither in that game), go read a play by play or go watch the film. I think folks think that because observational data is not easy to locate in a numerical form, that makes it subjective or unreliable or impossible to prove or deny, but that's not really true. It's just not as easy to disprove, which bothers a lot of math folks. Sure, it's easy to say, but I don't write things that I don't know to be true. I've done the research and know that when I write "you don't see many broken plays in New England like you do around the league on any given Sunday", it's the truth. Just because the "research" in that case is watching thousands of hours of football or reading hundreds of play-by-play charts and box scores as opposed to taking some stats, smashing them together and calling it a study, doesn't make it any less true. It just makes it harder for someone to disprove.
There is place for observational data, and there is a place for statistical analysis. And IMO, there is room for both, and both should be used, but with football statistics, I don't believe you can necessarily form many conclusions without also observational data, which is not the case in a sport like baseball. I also believe that football is so much harder for people to understand on a micro level that they want to be able to use statistics to tell them what they can't figure out on their own. I just don't think it's possible.
I think real observational data is somewhat hard at football too, at least in terms of splitting hairs among generationally great players, unless you are breaking down all-22 film each week. It might be easier for a quarterback than some other positions just watching on TV, but it's still missing a lot. The example I'm thinking of is the complaint that Brady was locking in on guys or ignoring Amendola early this year. One of the film nuts went through play by play to show that he wasn't really doing that at all, but the stats or watching the game on TV couldn't tell you that. We don't have all-22 for Marino, Montana, or most of Bradys or Mannings career.
I agree with the conclusion that I don't think there's a compelling argument that you can't argue for Brady as the GOAT. I disagree that Brady would be the only correct choice to start a game under the conditions you describe.
Fwiw, in terms of games Brady lost a game down the stretch or fumbled or threw a pick at an inopportune time, would depend on what you mean by down the stretch or inopportune or lost the game. It hasn't happened often, but it has happened, and I don't remember other great QBs doing it very often either.
I agree with the conclusion that I don't think there's a compelling argument that you can't argue for Brady as the GOAT. I disagree that Brady would be the only correct choice to start a game under the conditions you describe.
Fwiw, in terms of games Brady lost a game down the stretch or fumbled or threw a pick at an inopportune time, would depend on what you mean by down the stretch or inopportune or lost the game. It hasn't happened often, but it has happened, and I don't remember other great QBs doing it very often either.
Well, I disagree about the value of ANY/A, which I think is very useful. The TD bonus and the INT penalty aren't arbitrary; they're empirically derived. The penalty for an INT is 45 yards because the average loss of field position on an interception is 45 yards. If you think ANY/A is a fairly good measure of a team's passing game, and I do, then the combined opponent/era adjustment is actually a rather elegant solution. It would be better if the opponent adjustments were then iterated, and if they were adjusted for dome/weather, but it's a good start.
As for Plunkett in 1980, he played one of the better Super Bowls on record, in a difficult passing era, against an excellent passing defense. And he was pretty darn good in the conference championship, too. The leverage index perhaps doesn't put enough weight on his poorer early round performances, but finding nuggets like that, which I would have overlooked otherwise by just looking at conventional stats, confirms for me the value of the study, rather than the other way around.
Here's the thing, though. Observational data isn't any less "data" than statistical data. It has all the same problems as statistical analysis, but in addition It's less systematic and more prone to bias. The box scores and play-by-plays themselves are objective, but you're asking us to trust not those directly, but rather your recollection and analysis of those, which everything we know about perception and cognition and memory tells us is unreliable and prone to all sorts of error and bias (and I don't mean you personally, this would apply to everyone). You complain about stats not being opponent or weather adjusted -- how do we know your recollection properly takes into account opponent and weather, especially for games you didn't actually watch and just skimmed through a box score 15 or 20 years ago? How could you even accurately do such an adjustment cognitively? I would much rather rely on data, where I know where it comes from and how it was derived and I can see the strengths and weaknesses, and figure out how much weight I think it deserves.
FWIW, p-ref has a searchable play-by-play database going back to 1998. You can search for things like "all turnovers by a particular player in the fourth quarter/OT when the score is within 10 points."
Here is that search for Brady: http://www.pro-football-reference.com/play-index/play_finder.cgi?request=1&match=summary_all&search=&player_id=BradTo00&year_min=2000&year_max=2014&team_id=&opp_id=&game_type=&playoff_round=&game_num_min=0&game_num_max=99&week_num_min=0&week_num_max=99&quarter=4&quarter=5&tr_gtlt=lt&minutes=15&seconds=00&down=0&down=1&down=2&down=3&down=4&yds_to_go_min=&yds_to_go_max=&yg_gtlt=gt&yards=&is_first_down=-1&field_pos_min_field=team&field_pos_min=&field_pos_max_field=team&field_pos_max=&end_field_pos_min_field=team&end_field_pos_min=&end_field_pos_max_field=team&end_field_pos_max=&type=PASS&type=PUNT&type=KOFF&type=ONSD&type=FG&type=XP&type=2PCR&type=2PCP&is_complete=-1&is_turnover=1&turnover_type=interception&turnover_type=fumble&is_scoring=-1&score_type=touchdown&score_type=field_goal&score_type=safety&is_sack=-1&include_kneels=-1&no_play=0&game_day_of_week=&game_location=&game_result=&margin_min=-10&margin_max=10&order_by=yards&rush_direction=LE&rush_direction=LT&rush_direction=LG&rush_direction=M&rush_direction=RG&rush_direction=RT&rush_direction=RE&pass_location=SL&pass_location=SM&pass_location=SR&pass_location=DL&pass_location=DM&pass_location=DR
Here is that search for Manning: http://www.pro-football-reference.com/play-index/play_finder.cgi?request=1&match=summary_all&search=&player_id=MannPe00&year_min=1998&year_max=2014&team_id=&opp_id=&game_type=&playoff_round=&game_num_min=0&game_num_max=99&week_num_min=0&week_num_max=99&quarter=4&quarter=5&tr_gtlt=lt&minutes=15&seconds=00&down=0&down=1&down=2&down=3&down=4&yds_to_go_min=&yds_to_go_max=&yg_gtlt=gt&yards=&is_first_down=-1&field_pos_min_field=team&field_pos_min=&field_pos_max_field=team&field_pos_max=&end_field_pos_min_field=team&end_field_pos_min=&end_field_pos_max_field=team&end_field_pos_max=&type=PASS&type=PUNT&type=KOFF&type=ONSD&type=FG&type=XP&type=2PCR&type=2PCP&is_complete=-1&is_turnover=1&turnover_type=interception&turnover_type=fumble&is_scoring=-1&score_type=touchdown&score_type=field_goal&score_type=safety&is_sack=-1&include_kneels=-1&no_play=0&game_day_of_week=&game_location=&game_result=&margin_min=-10&margin_max=10&order_by=yards&rush_direction=LE&rush_direction=LT&rush_direction=LG&rush_direction=M&rush_direction=RG&rush_direction=RT&rush_direction=RE&pass_location=SL&pass_location=SM&pass_location=SR&pass_location=DL&pass_location=DM&pass_location=DR
Brady has 43 (after removing the false positives in the search when somebody besides Brady fumbled). Manning has 50, having played an additional three seasons.
As for Plunkett in 1980, he played one of the better Super Bowls on record, in a difficult passing era, against an excellent passing defense. And he was pretty darn good in the conference championship, too. The leverage index perhaps doesn't put enough weight on his poorer early round performances, but finding nuggets like that, which I would have overlooked otherwise by just looking at conventional stats, confirms for me the value of the study, rather than the other way around.
Here's the thing, though. Observational data isn't any less "data" than statistical data. It has all the same problems as statistical analysis, but in addition It's less systematic and more prone to bias. The box scores and play-by-plays themselves are objective, but you're asking us to trust not those directly, but rather your recollection and analysis of those, which everything we know about perception and cognition and memory tells us is unreliable and prone to all sorts of error and bias (and I don't mean you personally, this would apply to everyone). You complain about stats not being opponent or weather adjusted -- how do we know your recollection properly takes into account opponent and weather, especially for games you didn't actually watch and just skimmed through a box score 15 or 20 years ago? How could you even accurately do such an adjustment cognitively? I would much rather rely on data, where I know where it comes from and how it was derived and I can see the strengths and weaknesses, and figure out how much weight I think it deserves.
FWIW, p-ref has a searchable play-by-play database going back to 1998. You can search for things like "all turnovers by a particular player in the fourth quarter/OT when the score is within 10 points."
Here is that search for Brady: http://www.pro-football-reference.com/play-index/play_finder.cgi?request=1&match=summary_all&search=&player_id=BradTo00&year_min=2000&year_max=2014&team_id=&opp_id=&game_type=&playoff_round=&game_num_min=0&game_num_max=99&week_num_min=0&week_num_max=99&quarter=4&quarter=5&tr_gtlt=lt&minutes=15&seconds=00&down=0&down=1&down=2&down=3&down=4&yds_to_go_min=&yds_to_go_max=&yg_gtlt=gt&yards=&is_first_down=-1&field_pos_min_field=team&field_pos_min=&field_pos_max_field=team&field_pos_max=&end_field_pos_min_field=team&end_field_pos_min=&end_field_pos_max_field=team&end_field_pos_max=&type=PASS&type=PUNT&type=KOFF&type=ONSD&type=FG&type=XP&type=2PCR&type=2PCP&is_complete=-1&is_turnover=1&turnover_type=interception&turnover_type=fumble&is_scoring=-1&score_type=touchdown&score_type=field_goal&score_type=safety&is_sack=-1&include_kneels=-1&no_play=0&game_day_of_week=&game_location=&game_result=&margin_min=-10&margin_max=10&order_by=yards&rush_direction=LE&rush_direction=LT&rush_direction=LG&rush_direction=M&rush_direction=RG&rush_direction=RT&rush_direction=RE&pass_location=SL&pass_location=SM&pass_location=SR&pass_location=DL&pass_location=DM&pass_location=DR
Here is that search for Manning: http://www.pro-football-reference.com/play-index/play_finder.cgi?request=1&match=summary_all&search=&player_id=MannPe00&year_min=1998&year_max=2014&team_id=&opp_id=&game_type=&playoff_round=&game_num_min=0&game_num_max=99&week_num_min=0&week_num_max=99&quarter=4&quarter=5&tr_gtlt=lt&minutes=15&seconds=00&down=0&down=1&down=2&down=3&down=4&yds_to_go_min=&yds_to_go_max=&yg_gtlt=gt&yards=&is_first_down=-1&field_pos_min_field=team&field_pos_min=&field_pos_max_field=team&field_pos_max=&end_field_pos_min_field=team&end_field_pos_min=&end_field_pos_max_field=team&end_field_pos_max=&type=PASS&type=PUNT&type=KOFF&type=ONSD&type=FG&type=XP&type=2PCR&type=2PCP&is_complete=-1&is_turnover=1&turnover_type=interception&turnover_type=fumble&is_scoring=-1&score_type=touchdown&score_type=field_goal&score_type=safety&is_sack=-1&include_kneels=-1&no_play=0&game_day_of_week=&game_location=&game_result=&margin_min=-10&margin_max=10&order_by=yards&rush_direction=LE&rush_direction=LT&rush_direction=LG&rush_direction=M&rush_direction=RG&rush_direction=RT&rush_direction=RE&pass_location=SL&pass_location=SM&pass_location=SR&pass_location=DL&pass_location=DM&pass_location=DR
Brady has 43 (after removing the false positives in the search when somebody besides Brady fumbled). Manning has 50, having played an additional three seasons.
On ANY/A having a high correlation with wins:
In baseball, RBI has a greater correlation with wins than OPS or even WAR.
(In baseball there is good enough data to extend RBI to stats independent of team context like OPS+ and WAR.)
e.g.
http://www.baseballprospectus.com/article.php?articleid=20569
In baseball, RBI has a greater correlation with wins than OPS or even WAR.
(In baseball there is good enough data to extend RBI to stats independent of team context like OPS+ and WAR.)
e.g.
http://www.baseballprospectus.com/article.php?articleid=20569
NFL rules - or points of emphasis - change so much, and so frequently, how can this even be a debate?
Running backs (on talent, not system) used to be first round draft picks...
The sport has no continuity. History can't be measured against present.
Running backs (on talent, not system) used to be first round draft picks...
The sport has no continuity. History can't be measured against present.
How did you remove the false positives? By changing the parameters of your search to not include completed passes (thereby eliminating any fumbles that were not by the QB), the totals change to 38 for Brady and 48 for Manning (which, of course, is a higher per season rate, though narrowly so).coremiller said:
One of the arguements for why the Patriots run has been perhaps more impressive than the 49ers of the 80s/90s, Steelers of the 70s, and Packers of the 60s is that it is a lot harder to keep a team together in the salary cap era, the way the schedule is organized tries to produce parity, etc. Structual differences, in other words.
I wonder if there's some way to quantitate this structural difference, not to prove it exists (it certainly does, the rules are on the books) but rather to show that it does have an effect on outcomes, and thus the ability to rise above this leveling force is an impressive accomplishment. I'd think that some sort of volatility index would speak to this, namely, how well do the final results (W-L record, scoring differential, whatever) for a team in year 1 predict the results of that team the following year.
Is that an approach that makes sense to people?
I wonder if there's some way to quantitate this structural difference, not to prove it exists (it certainly does, the rules are on the books) but rather to show that it does have an effect on outcomes, and thus the ability to rise above this leveling force is an impressive accomplishment. I'd think that some sort of volatility index would speak to this, namely, how well do the final results (W-L record, scoring differential, whatever) for a team in year 1 predict the results of that team the following year.
Is that an approach that makes sense to people?
bradmahn said:
I just did it manually by going through the results and counting the false positives. If you search only incomplete passes, the search doesn't count sack-fumbles, of which Brady has 5 and Manning 2.
Yeah, I think that's the right approach (I think you want to look at how well outlier final results project outlier final results the following year), Ive been trying to think about the best way to do that. My stats training is sharp enough to know the story on Pats fumble rate is bullshit, but too rusty to remember how to do this offhand.alydar said:
My two cents:
Football statistics like ANY/A+ are very useful tools for getting one perspective on QB performance and value but they need to be viewed with a grain of salt, especially in the interpretation of relatively small differences. ANY/A+ is probably one of the best statistics around but once you consider how much it misses (not just in terms of contextual factors like teammates and weather, but as a measure of QB value created)its obviously silly to use it as a tool for rank-ordering among relatively similar players. Off the top of my head, it doesn't capture passing volume per game, games played, performance in high leverage situations, or success rate. So when you look at the ANY/A+ career leaders and see Rodgers 124, Young 123, Manning 122, Montana 121, Marino 119, and Brady 117, the reasonable inference is that the statistic is doing a pretty decent job in roughly defining the universe of top quarterbacks, but not that the rank-ordering is particularly meaningful.
In lieu of better statistics, which might come around someday but are limited both by the nature of the game and the non-transparent way that a lot of advanced football statistics have been developed (ie, DVOA, QBR, WPA), I think its pretty reasonable to use other kinds of observations to make arguments for rank-ordering among relatively similar players. For example, Peyton Manning played poorly (IMO) in all three Super Bowls in which he was involved and Dan Marino played poorly in just about all his most high-leverage playoff games. Given that the ultimate goal of every team is to win the Super Bowl, those are pertinent pieces of data for me. Steve Young started 100 less games than some of the other QBs in this discussion: He has similar efficiency stats over a sample that is like 60% the size. That's a huge factor IMO if you're thinking about career value. Other people could reasonably draw on different kinds of observations to rank-order players at the top.
Football statistics like ANY/A+ are very useful tools for getting one perspective on QB performance and value but they need to be viewed with a grain of salt, especially in the interpretation of relatively small differences. ANY/A+ is probably one of the best statistics around but once you consider how much it misses (not just in terms of contextual factors like teammates and weather, but as a measure of QB value created)its obviously silly to use it as a tool for rank-ordering among relatively similar players. Off the top of my head, it doesn't capture passing volume per game, games played, performance in high leverage situations, or success rate. So when you look at the ANY/A+ career leaders and see Rodgers 124, Young 123, Manning 122, Montana 121, Marino 119, and Brady 117, the reasonable inference is that the statistic is doing a pretty decent job in roughly defining the universe of top quarterbacks, but not that the rank-ordering is particularly meaningful.
In lieu of better statistics, which might come around someday but are limited both by the nature of the game and the non-transparent way that a lot of advanced football statistics have been developed (ie, DVOA, QBR, WPA), I think its pretty reasonable to use other kinds of observations to make arguments for rank-ordering among relatively similar players. For example, Peyton Manning played poorly (IMO) in all three Super Bowls in which he was involved and Dan Marino played poorly in just about all his most high-leverage playoff games. Given that the ultimate goal of every team is to win the Super Bowl, those are pertinent pieces of data for me. Steve Young started 100 less games than some of the other QBs in this discussion: He has similar efficiency stats over a sample that is like 60% the size. That's a huge factor IMO if you're thinking about career value. Other people could reasonably draw on different kinds of observations to rank-order players at the top.
Tangled Up In Red said:
I'm with you here. Trying to define one QB as THE best is nonsensical. The game as played 20 years ago is much different from today's game .....or the one played in the 70's .... or 50's. Brady is certainly in the conversation of the best QBs to play the game - as are a half-dozen others.
And for this time, there can be no doubt:
"The UConn womens basketball coach matched John Wooden with 10 national championships when his Huskies claimed the title Tuesday with a 63-53 win over Notre Dame. In the postgame press conference, Auriemma was asked for his thoughts on how his accomplishments match up against his peers in mens sports.
Suffice to say, Auriemma puts himself in elite company.
We go by whats the most important sport in the country, the NFL, Auriemma said, according to a transcript. Bill Belichick is the only coach worth a (expletive) right now. Everybody else is just trying to catch up to him.
But every other coach that you mentioned, whether its Phil Jackson, (Mike Krzyzewski), Anson Dorrance down in North Carolina, the womens soccer coach, when you accomplish something thats really hard to accomplish, you should be proud of yourself.'
"The UConn womens basketball coach matched John Wooden with 10 national championships when his Huskies claimed the title Tuesday with a 63-53 win over Notre Dame. In the postgame press conference, Auriemma was asked for his thoughts on how his accomplishments match up against his peers in mens sports.
Suffice to say, Auriemma puts himself in elite company.
We go by whats the most important sport in the country, the NFL, Auriemma said, according to a transcript. Bill Belichick is the only coach worth a (expletive) right now. Everybody else is just trying to catch up to him.
But every other coach that you mentioned, whether its Phil Jackson, (Mike Krzyzewski), Anson Dorrance down in North Carolina, the womens soccer coach, when you accomplish something thats really hard to accomplish, you should be proud of yourself.'