24 replies to this topic

[BernieRicoBoomer]

Okay, some of the talk this week about the Pats’ impressive record late in seasons got me thinking of some kind of system to properly value each team’s performance and relative ranking as we get deep into the stretch run. This would not have so much of a predictive value as in a “power ranking”, as it would be an evaluation of their performance versus competition to date. Just a way of putting numbers to it without using the subjective “Paper Tiger” argument.

I figure one of the most important factors should be an opponent’s record coming into each game. This is the best way to assign a degree of difficulty to each win. You should not get bonus points for anything a team did after they played you, nor should you necessarily be penalized if they stunk afterwards. Therefore, I am assigning points to each win equal to the number of wins the team you beat had coming into the game.

The second important factor I believe to be considered is when the victory was earned. Wins should be worth more as you run through the season. The NFL season has become a war of attrition as we all know. As players get beat up and injuries mount we generally start to separate the contenders from the pretenders. So, I am assigning one point for a win in a team’s first game of the season, two for their second game, and on up to 16 points for a win in the final game of the season. Not really sure what if anything should be done to account for teams resting starters once they have things locked up.

Finally, I am giving one bonus point for a win on the road. I am not sure again here if this adds enough weight or if perhaps it should be based on the specific venue. For example, extra points for winning in Seattle…not so much for winning in England vs. St. Louis.
I am not deducting any points for a loss, no matter how ugly or who it comes against. I am merely looking for a relative value of wins and, of course a loss nets you zero points.

In any case, I total up all the points and divide by the number of games played to determine a “value of wins”. I think it’s a pretty interesting way to look at it and if nothing else I enjoyed running through the exercise. Another way to look at it is the “value of wins” less the actual wins or “strength of wins”. Kind of give you an idea of whose record is most inflated (Atlanta being the biggest "Paper Tiger". For what it’s worth here is how the top teams look thus far:

	Value of Wins	Actual Wins	Strength of Wins
New England	9.54	10	-0.46
Denver	8.54	10	-1.46
Houston	8.15	11	-2.85
Atlanta	7.85	11	-3.15
Green Bay	7.85	9	-1.15
Indianapolis	7.62	9	-1.38
San Francisco	7.54	9	-1.46
Washington	6.85	7	-0.15
Seattle	6.77	8	-1.23
Giants	6.31	8	-1.69
Baltimore	6.15	9	-2.85
St. Louis	6.15	6	0.15
Pittsburgh	5.92	7	-1.08
Dallas	5.77	7	-1.23
Cincinnati	5.46	7	-1.54
Chicago	4.92	8	-3.08
Minnesota	4.92	7	-2.08
Jets	4.62	6	-1.38
New Orleans	4.54	5	-0.46
Tampa Bay	4.46	6	-1.54

Please let me know your thoughts, opinions, possible improvements...or if not worthy of a thread, just bag it!

• Back to top

[Vertebreaker]

Although I appreciate the effort, the NFL is so quirky that each win is completely different, and I think it's shrewd to try to put a number on each win. Injuries, for one thing are impossible to calculate. A win over the Steelers with Big Ben is better then a win over the Steelers with Charlie Batch, right? Historically, divisional games are tougher then if you played a team from outside of your division. The Patriots destroyed St. Louis, but San Fran couldn't beat them. The Patriots have always struggled against Miami, even though they were clearly the better team every time. Each NFL game has so many intangibles that the only way to determine the quality of each win is too go back and individually dissect each game.

[Morgan's Magic Snowplow]

I think its a good project but I'm skeptical about some aspects of your methodology. I can understand counting wins later in the season more but assigning 16 points to a Week 16 win and 1 point to a Week 1 win over the same team seems way too extreme. Also, when adjusting for strength of opponent, you should probably factor in their winning percentage not their wins, otherwise you will also end up just counting much more for wins later in the season as all opponents will have more total wins.

[Super Nomario]

I figure one of the most important factors should be an opponent’s record coming into each game. This is the best way to assign a degree of difficulty to each win. You should not get bonus points for anything a team did after they played you, nor should you necessarily be penalized if they stunk afterwards. Therefore, I am assigning points to each win equal to the number of wins the team you beat had coming into the game.

The second important factor I believe to be considered is when the victory was earned. Wins should be worth more as you run through the season. The NFL season has become a war of attrition as we all know. As players get beat up and injuries mount we generally start to separate the contenders from the pretenders. So, I am assigning one point for a win in a team’s first game of the season, two for their second game, and on up to 16 points for a win in the final game of the season. Not really sure what if anything should be done to account for teams resting starters once they have things locked up.

Aren't you double-counting when teams play in this methodology? The Pats beat Denver week 5, which is worth 5 (week) + 2 (wins Denver had coming in). They get more than twice as many points for beating the Jets on Thanksgiving - 12 (week) + 4 (wins coming in) + 1 road. I think just having "wins coming in" counts how late in the season it is without adding bonus points for it being late in the season. Or am I misunderstanding your methodology.

[MalzoneExpress]

Reading the initial post and the commentary made me think that it might be better to:

1. Rather than bonus point for the week, what about making it quarterly? 1 point for wins in games 1-4, 2 points for wins in games 5-8, 3 point for wins in games 9-12, and 4 points for wins in games 13-16. This addresses the "extreme" concern of MMS, but still values wins later in the season more than early in the season.

2. Rather than add total wins, add the win total from the previous 4 weeks. That would be more representative of how the team is playing at the time. For games 1-5, just use the record after game 4 to "normalize" the very beginning of the season. This would address some of the issues raised by SN.

Does this address all the concerns? No, but there is no perfect quick and dirty metric that can. I think your approach can be useful with some tweaks. Thanks for sharing it with us.

[BernieRicoBoomer]

Aren't you double-counting when teams play in this methodology? The Pats beat Denver week 5, which is worth 5 (week) + 2 (wins Denver had coming in). They get more than twice as many points for beating the Jets on Thanksgiving - 12 (week) + 4 (wins coming in) + 1 road. I think just having "wins coming in" counts how late in the season it is without adding bonus points for it being late in the season. Or am I misunderstanding your methodology.

True, it is giving extra weight to later in the season wins. The idea being that we have more complete information on teams the later we go. The more wins a team earns the more likely that they are in fact a tough test for their opponent. Using that and the points for week also keeps the point totals somewhat more relevant to actual wins when you divide by the number of weeks played. A more apt comparison of the quality of the Pats win over the Jets on Thanksgiving is what all the other teams received for points that week, rather than what they received in a much earlier week.

[abty]

What is more impressive? Beating the Ravens when everybody is injured, at home, or on the road when they were solid?
Beating the Broncos in week 3-5 when they were figuring it out or now?
Beating San Fran on the road or home?
Beating the top 5 teams in the NFL with a 10-6 record or only top 3, if you have = # of games vs the top 5 teams, but have 12 wins?

Too many variables in a season to quantify such a thing as 'strength of wins'. This is not directed at you but to those who make a living creating/calculating this shit. I admire the concept but worry people will take it too seriously. Everything is a matter of perspective, basically.

Edited by abty, 11 December 2012 - 01:50 PM.

[BernieRicoBoomer]

Reading the initial post and the commentary made me think that it might be better to:

1. Rather than bonus point for the week, what about making it quarterly? 1 point for wins in games 1-4, 2 points for wins in games 5-8, 3 point for wins in games 9-12, and 4 points for wins in games 13-16. This addresses the "extreme" concern of MMS, but still values wins later in the season more than early in the season.

2. Rather than add total wins, add the win total from the previous 4 weeks. That would be more representative of how the team is playing at the time. For games 1-5, just use the record after game 4 to "normalize" the very beginning of the season. This would address some of the issues raised by SN.

Does this address all the concerns? No, but there is no perfect quick and dirty metric that can. I think your approach can be useful with some tweaks. Thanks for sharing it with us.

Thanks, I will play around with this and try it out with something like this for comparison. I do think it really needs to be weighted heavily though for later in the year, due to the "war of attrition". The team that can win with multiple starters out or banged up is a much better team going into the playoffs then say a team that starts 7-1 when everyone is healthy, but falls apart as the season wears on. Looking at you Chicago. I guess I am doubling up because getting those later wins is impressive, but getting them against teams which also get those wins is doubly impressive.

Hope that makes sense.

[BernieRicoBoomer]

What is more impressive? Beating the Ravens when everybody is injured, at home, or on the road when they were solid?
Beating the Broncos in week 3-5 when they were figuring it out or now?
Beating San Fran on the road or home?
Beating the top 5 teams in the NFL with a 10-6 record or only top 3, if you have = # of games vs the top 5 teams, but have 12 wins?

Too many variables in a season to quantify such a thing as 'strength of wins'. This is not directed at you but to those who make a living creating/calculating this shit. I admire the concept but worry people will take it too seriously. Everything is a matter of perspective, basically.

You are correct, it certainly is far from perfect. For example, it gives your Giants a 7.00 and -2.00 for the regular season last year. We all know how that turned out. What the hell, I'm a CPA and love to throw numbers and data into spreadsheets!

[MalzoneExpress]

True, it is giving extra weight to later in the season wins. The idea being that we have more complete information on teams the later we go. The more wins a team earns the more likely that they are in fact a tough test for their opponent. Using that and the points for week also keeps the point totals somewhat more relevant to actual wins when you divide by the number of weeks played. A more apt comparison of the quality of the Pats win over the Jets on Thanksgiving is what all the other teams received for points that week, rather than what they received in a much earlier week.

If the expectation is that "we have more complete information on teams the later we go", why do you think it makes sense to give everyone that beats Arizona 4 points for their 4 wins even though they haven't won a game since week 4? I don't think it does. Using Arizona as the example, the rolling 4-week window option rewards teams for beating Arizona early when they were playing well. but not later in the season when Arizona is not playing well. The same applies to Chicago as you reference in a later post.

Also, I agree that wins later in the season should be weighed more heavily than early wins, but I feel that a quarterly stepped option (1,2,3,4 or 1,3,6,9 or even 1,4,8,12) seems preferable to a straight linear approach (1,2,3...16). All things being equal, is a win in week 3 that much different than a win in week 2, or is a win in week 14 that much different than a win in week 13?

[BernieRicoBoomer]

If the expectation is that "we have more complete information on teams the later we go", why do you think it makes sense to give everyone that beats Arizona 4 points for their 4 wins even though they haven't won a game since week 4? I don't think it does. Using Arizona as the example, the rolling 4-week window option rewards teams for beating Arizona early when they were playing well. but not later in the season when Arizona is not playing well. The same applies to Chicago as you reference in a later post.

Also, I agree that wins later in the season should be weighed more heavily than early wins, but I feel that a quarterly stepped option (1,2,3,4 or 1,3,6,9 or even 1,4,8,12) seems preferable to a straight linear approach (1,2,3...16). All things being equal, is a win in week 3 that much different than a win in week 2, or is a win in week 14 that much different than a win in week 13?

Don't get me wrong, I like your idea and will definitely try it out and see how it differs. However, look at it this way regarding Arizona's 4 wins...The Rams beat them in week 5 and those 4 points were 44% of their point total that week and 5% of their point total for the year. When Seattle beat them this week it was only 23.5% of their point total this week and 4% of the total for the year. So, it may seem like each team that beats them going forward is getting the same it actually becomes less and less as a percentage.

[MalzoneExpress]

Don't get me wrong, I like your idea and will definitely try it out and see how it differs. However, look at it this way regarding Arizona's 4 wins...The Rams beat them in week 5 and those 4 points were 44% of their point total that week and 5% of their point total for the year. When Seattle beat them this week it was only 23.5% of their point total this week and 4% of the total for the year. So, it may seem like each team that beats them going forward is getting the same it actually becomes less and less as a percentage.

I appreciate your giving my ideas a try. I would like to see how the results differ.

I understand your point about percentage of weekly and yearly points. However for Arizona that difference is ENTIRELY a function of your linear weekly point total adjustment. It looks like you are valuing the "when" (later in the season), but not "who". The "who" that was Arizona in week 5 (good and gives 4 points for having 4 wins) is much different than the "who" that was Arizona last week (bad and still gives 4 points for having 4 wins).

[BernieRicoBoomer]

I appreciate your giving my ideas a try. I would like to see how the results differ.

I understand your point about percentage of weekly and yearly points. However for Arizona that difference is ENTIRELY a function of your linear weekly point total adjustment. It looks like you are valuing the "when" (later in the season), but not "who". The "who" that was Arizona in week 5 (good and gives 4 points for having 4 wins) is much different than the "who" that was Arizona last week (bad and still gives 4 points for having 4 wins).

I disagree on that point though. St. Louis in game 5 got 4 out of a total possible 4 points for beating Arizona. Seattle this week only got 4 out of a total possible 12 points. The Pats made a big move this week because they beat an 11 win team. Seattle really only maintains position by winning because they played a weak Arizona team. The percentages I noted above would be drastically different had Arizona continued to be the team they were back "when".

[BernieRicoBoomer]

If the expectation is that "we have more complete information on teams the later we go", why do you think it makes sense to give everyone that beats Arizona 4 points for their 4 wins even though they haven't won a game since week 4? I don't think it does. Using Arizona as the example, the rolling 4-week window option rewards teams for beating Arizona early when they were playing well. but not later in the season when Arizona is not playing well. The same applies to Chicago as you reference in a later post.

Also, I agree that wins later in the season should be weighed more heavily than early wins, but I feel that a quarterly stepped option (1,2,3,4 or 1,3,6,9 or even 1,4,8,12) seems preferable to a straight linear approach (1,2,3...16). All things being equal, is a win in week 3 that much different than a win in week 2, or is a win in week 14 that much different than a win in week 13?

Okay, here it is using your (1,4,8,12) quarterly stepped option:

	Value of Wins	Actual Wins	Strength of Wins
New England	7.85	10	-2.15
Denver	6.77	10	-3.23
Houston	6.31	11	-4.69
Green Bay	6.23	9	-2.77
San Francisco	6.15	9	-2.85
Atlanta	5.92	11	-5.08
Indianapolis	5.92	9	-3.08
Washington	5.69	7	-1.31
Seattle	5.69	8	-2.31
St. Louis	5.15	6	-0.85
Giants	5.00	8	-3.00
Dallas	4.85	7	-2.15
Baltimore	4.77	9	-4.23
Pittsburgh	4.54	7	-2.46
Cincinnati	4.23	7	-2.77
Minnesota	4.00	7	-3.00
Jets	3.77	6	-2.23
New Orleans	3.77	5	-1.23
Tampa Bay	3.62	6	-2.38
Chicago	3.54	8	-4.46

• Back to top

[MalzoneExpress]

Okay, here it is using your (1,4,8,12) quarterly stepped option:

	Value of Wins	Actual Wins	Strength of Wins
New England	7.85	10	-2.15
Denver	6.77	10	-3.23
Houston	6.31	11	-4.69
Green Bay	6.23	9	-2.77
San Francisco	6.15	9	-2.85
Atlanta	5.92	11	-5.08
Indianapolis	5.92	9	-3.08
Washington	5.69	7	-1.31
Seattle	5.69	8	-2.31
St. Louis	5.15	6	-0.85
Giants	5.00	8	-3.00
Dallas	4.85	7	-2.15
Baltimore	4.77	9	-4.23
Pittsburgh	4.54	7	-2.46
Cincinnati	4.23	7	-2.77
Minnesota	4.00	7	-3.00
Jets	3.77	6	-2.23
New Orleans	3.77	5	-1.23
Tampa Bay	3.62	6	-2.38
Chicago	3.54	8	-4.46

Thanks BRB. Do you treat ties like losses?

[BernieRicoBoomer]

Thanks BRB. Do you treat ties like losses?

Open to suggestions on the ties, but what I did was only give points for opponent's wins and bonus for being on the road. No points for the week.

By the way, I used these numbers to make my selections in a pool this week...see how that goes.

[Mooch]

Open to suggestions on the ties, but what I did was only give points for opponent's wins and bonus for being on the road. No points for the week.

By the way, I used these numbers to make my selections in a pool this week...see how that goes.

I'd be curious to apply this method to prior regular seasons to see if there's any predictive value in the playoffs.

[Saints Rest]

It just seems counter-intuitive to me (and thus something is amiss) that everyone scores a negative. My gut tells me that the total for all teams should zero out. Somehow, an upset should carry more weight. Conversely a near-win against a team you are heavily favored against (such as the Pats vs Bills on 11/11) would carry more negative.

In any case, this exercise starts to feel like an NFL version of the BCS, and for that reason alone . . . "I'm out." </Shark Tank>

[BernieRicoBoomer]

I'd be curious to apply this method to prior regular seasons to see if there's any predictive value in the playoffs.

Here are the numbers for last year's playoff teams using my original method (I ran it with both methods):

	Value of Wins	Actual Wins	Strength of Wins
Denver	6.44	8	-1.56
New Orleans	11.06	13	-1.94
Giants	7.00	9	-2.00
Atlanta	7.63	10	-2.37
New England	10.44	13	-2.56
Pittsburgh	9.38	12	-2.62
Cincinnati	6.19	9	-2.81
San Francisco	10.13	13	-2.87
Baltimore	9.06	12	-2.94
Houston	7.06	10	-2.94
Detroit	6.50	10	-3.50
Green Bay	11.31	15	-3.69

Using the "Strength of Wins" would have successfully predicted 8 of the 11 playoff games. It would not have predicted the Giants to win, since it would've had them losing to New Orleans rather than beating San Francisco. It failed on SF/NO, Houston/ Cinn, and New England/Denver. Not too bad...just one year though.

• Back to top

[Super Nomario]

Here are the numbers for last year's playoff teams using my original method (I ran it with both methods):

Using the "Strength of Wins" would have successfully predicted 8 of the 11 playoff games. It would not have predicted the Giants to win, since it would've had them losing to New Orleans rather than beating San Francisco. It failed on SF/NO, Houston/ Cinn, and New England/Denver. Not too bad...just one year though.

Isn't the Value of Wins column a better representation of team quality than Strength of Wins?

[Reverend]

I actually really like the attempt to control for the endogeneity problem in records, i.e. when someone says "Look at how bad the records of the teams they've beat is!" when the fact that the records are poor may be a function of the fact that the team being looked at beat them--the opponents of successful teams will, by definition, ceteris paribus have worse records than the opponents of poorly performing teams.

Interestingly, and while I agree it can be improved, some of the criticisms of this attempt to evaluate quality of a team apply similarly to a team's actual record.

[nothumb]

it seems to me that the "strength of wins" total punishes teams when they beat up on bad AND good opponents. i don't know if this is something that actually happens enough to be significant, but, for instance, a team that beats a couple of tough opponents but drops a couple of stinkers to mediocre teams would seem to have a higher ratio of quality wins to overall wins than a team that beats the tough opponents AND clobbers a few weak teams. so maybe something where you give some weight to total wins + some weight to strength of wins would help.

using it retrospectively on the 2011 playoffs doesn't say much because it ignores things like tj yates playing for houston, ryan clark having sickle cell, etc. plus if we take it at face value your model would have said DEN is the SB favorite, which doesn't pass the smell test.

i think this is a good idea overall though.

i do think a "last 4" factor, or more generally some kind of "heat meter" for each win would be worth adding.

i also think point-in-time strength of schedule could be factored in when crediting a team for the number of wins its defeated opponents had at the time the game was played. so a really crude way to do this would just be to multiply the opponents' win total by the average winning% of every team it has BEATEN at the time they beat them. you could probably exclude this stat until week 4 or so. i know we are getting to the point where this is hard to punch into a spreadsheet however.

[Zombie Wakefield]

i do think a "last 4" factor, or more generally some kind of "heat meter" for each win would be worth adding.

I also like the "last 4" approach. Let's keep it simple and just look at the "last 4" record of a team week-by week, including games from the pervious season for games in the first few weeks. Over a very long period of time, we would assume that every team's historical "last 4" record will approach .500, so we'll say that anything over .500 means a positive strength of victory, anything under .500 means a negative strength of victory.

As an example, your 2011 New England Patriots:

Week 1 1-3 Dolphins)
Week 2 3-1 Chargers)
Week 3 LOSS (2-2 Bills)
Week 4 3-1 Raiders)
Week 5 3-1 Jets)
Week 6 2-2 Cowboys)
Week 7 BYE
Week 8 LOSS (3-1 Steelers)
Week 9 LOSS (3-1 Giants)
Week 10 3-1 Jets
Week 11 2-2 Chiefs
Week 12 2-2 Eagles
Week 13 0-4 Colts
Week 14 1-3 Redskins
Week 15 4-0 Broncos
Week 16 2-2 Dolphins
Week 17 1-3 Bills

Wins: Opponent 27-25 (+1)
Losses: Opponent 8-4 (+2)

Overall, an above-average slate of opponents. The Patriots feasted on a weak slate of opponents from weeks 11-16 (except for the red-hot Broncos), after holding their own against a strong slate of opponents from Weeks 2-10.

I pulled this data 1 week at a time from football-reference.com; I'd be interested to see what the "last 4" records would be for other teams.

[BernieRicoBoomer]

it seems to me that the "strength of wins" total punishes teams when they beat up on bad AND good opponents. i don't know if this is something that actually happens enough to be significant, but, for instance, a team that beats a couple of tough opponents but drops a couple of stinkers to mediocre teams would seem to have a higher ratio of quality wins to overall wins than a team that beats the tough opponents AND clobbers a few weak teams. so maybe something where you give some weight to total wins + some weight to strength of wins would help.

using it retrospectively on the 2011 playoffs doesn't say much because it ignores things like tj yates playing for houston, ryan clark having sickle cell, etc. plus if we take it at face value your model would have said DEN is the SB favorite, which doesn't pass the smell test.

i think this is a good idea overall though.

i do think a "last 4" factor, or more generally some kind of "heat meter" for each win would be worth adding.

i also think point-in-time strength of schedule could be factored in when crediting a team for the number of wins its defeated opponents had at the time the game was played. so a really crude way to do this would just be to multiply the opponents' win total by the average winning% of every team it has BEATEN at the time they beat them. you could probably exclude this stat until week 4 or so. i know we are getting to the point where this is hard to punch into a spreadsheet however.

You are absolutely correct regarding the "strength of wins" category. I did not originally feel that this was the best measure for who was strongest going into the playoffs. However, I am intrigued by how well it performed in predicting the winners of each game. Obviously this could merely be coincidence and result of small sample size. I was thinking perhaps a combination of all three categories or some type of mathematical equation using actual wins vs. value of wins would yield better results. At least one more complex than subtracting one from the other.

Clearly, having Denver last year and St. Louis this year as the strongest teams does not pass the smell test. Although 7 million Tebow fans can't be wrong, right?

As far as, having a point-in-time kind of running total system as you suggest...something like this is what I orginally wanted to do. I was thinking though of updating each teams "value of wins" every week and using that rather than the actual wins to total up points each subsequent week. I was of course concerned about how much time such a system would take. I think I can do it without too much difficulty once I get an appropriate spreadsheet set up.

My biggest question right now would be whether I continue on and look at more seasons with the existing method or to try out these new ideas to see how they compare?

[Jer]

BernieRicoBoomer,

Have you looked into how teamrankings.com compiles their various rankings?

Our power ratings measure the relative performance of all teams using 100% objective performance stats and sophisticated computational algorithms.
Our proprietary power ratings model iteratively analyzes information on every game, including team performance, home/away status, margin of victory, and other factors. As each season progresses, the power ratings model builds a sophisticated, multi-layered web of knowledge: how each team has performed in various game scenarios, the strength of each team's opponents, the strength of each team's opponents' opponents, and so on.
The power ratings model computes unbiased, quantitative ratings that assess both general (e.g. Overall power ratings) and situational (e.g. Home/Away power ratings) performance for every team. The system is complex, unbiased, and dynamic; whenever a new game result comes in, the power ratings of many teams will typically change as a result.
Like fine wine, power ratings tend to improve with age. As more games are played, the potential impact of good luck on a team's results decreases, and the power ratings model can make more "connections" between teams in conferences of varying strength. For some more insight, read more about how we rank teams.

Football Outsiders also does a good job at incorporating strength of opponent and game outcomes to caculate DVOA.

It seems like much of what you're trying to accomplish is covered in their methodology (along with much much more). Am I missing something? Are you trying to create a more transparent "formula"?