Meaningful or meaningless? Playoff odds discussion

[AB in DC]

Fangraphs has the Nats at 57.5% to make the playoffs, which seems a bit high to me. They have Nats and Phils finishing at about the same records, while BP has the Phils likely to be five games ahead.

Either way, there's nothing like a series against the Marlins to shake a team out of the doldrums.

 

[SirPsychoSquints]

Fangraphs has the Nats at 57.5% to make the playoffs, which seems a bit high to me. They have Nats and Phils finishing at about the same records, while BP has the Phils likely to be five games ahead.

Either way, there's nothing like a series against the Marlins to shake a team out of the doldrums.

BP says 20.2%

 

[Adrian's Dome]

BP says 20.2%

Meaningless numbers are meaningless.

 

[E5 Yaz]

Meaningless numbers are meaningless.

71% of the time

 

[SirPsychoSquints]

Meaningless numbers are meaningless.

Do you have an estimate of the National’s likelihood of making the playoffs? I’m just putting information in here, citing a source, and allowing people to make decisions. You don’t need to shit on it every time.

 

[Papelbon's Poutine]

Do you have an estimate of the National’s likelihood of making the playoffs? I’m just putting information in here, citing a source, and allowing people to make decisions. You don’t need to shit on it every time.

Well, they’re 4 games back of Arizona and it’s not even the deadline, so rosters will change for them and opponents. How much weight are you putting into either of those stats. They could easily be in a playoff spot in a week. They have a pretty soft schedule the next stretch as well. I tend to agree with AD here, projections are pretty meaningless, either for player performance or playoff odds.

 

[SirPsychoSquints]

Well, they’re 4 games back of Arizona and it’s not even the deadline, so rosters will change for them and opponents. How much weight are you putting into either of those stats. They could easily be in a playoff spot in a week. They have a pretty soft schedule the next stretch as well. I tend to agree with AD here, projections are pretty meaningless, either for player performance or playoff odds.

It’s fine to discount or ignore them, or to present your own estimates. To argue no one else should post them (as he has done in another thread and implied here) is absurd.

 

[Papelbon's Poutine]

It’s fine to discount or ignore them, or to present your own estimates. To argue no one else should post them (as he has done in another thread and implied here) is absurd.

I’m not sure absurd is the right word, but if you have a personal issue and think he’s targeting you, PMing him is a better solution imo, or even blocking him.

The original stat cited is meaningless, the counter from another source was as well. A little bit of thought goes further than a computer simulation. They’re 4 games out with two months left, because any stat guy formulates a % one thinks they should adhere to that is what is absurd, imo. There are far too many variables that these models can’t take into account or that the people compiling them don’t know. I’d wager if you took a look, most of these projections are, quite literally, meaningless at this juncture. Shit, what was the % on the Sox in 2011 at this point or even last week of season?

 

[sean1562]

Is this an actual discussion thread or more of a quarantine thread?

At this point, I think it is time for the Nats to sell everyone they don't plan on re-signing next year. I do think they should try and keep Gio, so keep him around, but Harper, Murphy, and every reliever not tied down past this season should be out the door. They arent gonna win a WS with that team, and they should get rid of what the have so they can re-tool with Soto, Robles, Turner and Rendon as their core. Joe Ross comes back next season and I think he is a legit MLB pitcher. I think Rendon is the guy they should lock up long term, they have a lot of money coming off the books this year, and next as well.

The Harper core never really amounted to what they had hoped and I really hope they don't sign on for ten more years. Soto looks like a superstar, build the next team/core around him

 

[dcmissle]

The Nats discussion was bogging down a news and rumors thread as we head into the last 72 hours.

The key to what they do and don’t, I think, is something we have no way of knowing right now — how do they feel about Stras coming back to pitch effectively this season?

If they fear he’s done, they may sell every asset with a 2018 expiration date. If not, they will probably hang in.

That is a reasonable judgment.

But even if it isn’t, the playoff odds have no way of factoring in the availability or non-availability of the Nats #2 starter.

 

[slamminsammya]

I’m not sure absurd is the right word, but if you have a personal issue and think he’s targeting you, PMing him is a better solution imo, or even blocking him.

The original stat cited is meaningless, the counter from another source was as well. A little bit of thought goes further than a computer simulation. They’re 4 games out with two months left, because any stat guy formulates a % one thinks they should adhere to that is what is absurd, imo. There are far too many variables that these models can’t take into account or that the people compiling them don’t know. I’d wager if you took a look, most of these projections are, quite literally, meaningless at this juncture. Shit, what was the % on the Sox in 2011 at this point or even last week of season?

What exactly do you mean by "meaningless"? Having answered that, would you actually wager these predictions are meaningless? I would be happy to take you up on that.

I am curious which variables you think are being left out. Injuries seems like the big obvious one. Probably mental state as well, which is totally opaque but obviously has an impact in sports. What else?

Do you think all these systems are meaningless for all sports, or is it just baseball? It is really a game of luck and noise.

 

[AB in DC]

Either way, there's nothing like a series against the Marlins to shake a team out of the doldrums.

Evidently I spoke too soon. The Nats scored exactly one run in the last 19 innings, including today's two-hit wonder against a Marlins starter coming into the game at 2-10, 4.69.

 

[Papelbon's Poutine]

What exactly do you mean by "meaningless"? Having answered that, would you actually wager these predictions are meaningless? I would be happy to take you up on that.

I am curious which variables you think are being left out. Injuries seems like the big obvious one. Probably mental state as well, which is totally opaque but obviously has an impact in sports. What else?

Do you think all these systems are meaningless for all sports, or is it just baseball? It is really a game of luck and noise.

I'm honestly not sure if you're agreeing with me or arguing, but you answered your own question in the bolded. How did the projections have the Nats finishing before the beginning of the year? How did they have the Sox for playoff odds at beginning of September of 2011?

These models are great and all but, no, they're aren't particularly useful and citing variances between two different models - which was what I was speaking to - is even less so. If the Nats roll off 10 straight wins, that 54% or 20% isn't really predictive, is it? Same if the Phillies stop over performing and drop ten straight. Or a dozen other things happen.

Variables? Yes, obviously injury is biggest. Regression to mean. Hot streaks. Nagging injuries we don't know about. Development issues. Adjustments to adjustments. Etc.

This all started with talking about if the Nats should sell or stay put; my point is with regards to Rizzo, I highly doubt he's looking at what BP says is his playoff probability % when deciding to unload.

 

[JohntheBaptist]

How did the projections have the Nats finishing before the beginning of the year? How did they have the Sox for playoff odds at beginning of September of 2011?

Those numbers are a depiction of the odds of outcomes. These examples were instances where the, whatever it was, 5% likelihood that "x" happened came true. They're not predicting things, they're approximating a likelihood. In hindsight it shows us something that was ultimately true--it took a mindfuck of a collapse to, you know, collapse enough to be sitting at home somehow in October.

That a team with an 8-0 lead in the 8th has a 98% chance of winning doesn't mean it was wrong when the other team comes back and wins. It is interesting context. Plenty of flaws, but I don't think anyone's posting it in declaration of something.

edit--pre-season projections are a different animal obviously and that's not of much value to me either but they're models worth publishing and testing on with results, that's I think the better way to look at them

 

[Adrian's Dome]

Do you have an estimate of the National’s likelihood of making the playoffs? I’m just putting information in here, citing a source, and allowing people to make decisions. You don’t need to shit on it every time.

Would you like me to boil it down to the tenth or the thousandth?

When you cite "information" as a source, but the source is bullshit, don't state it as fact. I shit on projection systems because projection systems are shit, and I'm tired of people posting them as if they're anything substantial or noteworthy. There are a million variables in play that have real, actual effects on what will happen, and the "systems" incorporate exactly none of them into their "estimates."

It is impossible to predict, or even estimate, what will happen. I don't know, you don't know, and no computer knows, either. 20.2% odds of making the playoffs...okay. I'll counter that with a 36.1%.

 

[SirPsychoSquints]

Would you like me to boil it down to the tenth or the thousandth?

When you cite "information" as a source, but the source is bullshit, don't state it as fact. I shit on projection systems because projection systems are shit, and I'm tired of people posting them as if they're anything substantial or noteworthy. There are a million variables in play that have real, actual effects on what will happen, and the "systems" incorporate exactly none of them into their "estimates."

It is impossible to predict, or even estimate, what will happen. I don't know, you don't know, and no computer knows, either. 20.2% odds of making the playoffs...okay. I'll counter that with a 36.1%.

It's impossible to estimate what will happen, so why would anyone buy or sell at the deadline? The Nationals' estimate of their likelihood of making the playoffs as is, vs. if they buy, vs. if they sell is at the core of whether they should buy or sell. We don't have insight into their process for determining this (and yes, they're making these estimates one way or another), so we turn to projection systems that are the best understanding we have given the information available to us. They're also quite explicit about what information they're using and what reliability they give. This allows a framework for a discussion.

Any conversation about buyers vs. sellers is implicitly using a forecast for how likely given teams are to make the playoffs.

 

[Adrian's Dome]

It's impossible to estimate what will happen, so why would anyone buy or sell at the deadline? The Nationals' estimate of their likelihood of making the playoffs as is, vs. if they buy, vs. if they sell is at the core of whether they should buy or sell. We don't have insight into their process for determining this (and yes, they're making these estimates one way or another), so we turn to projection systems that are the best understanding we have given the information available to us. They're also quite explicit about what information they're using and what reliability they give. This allows a framework for a discussion.

Any conversation about buyers vs. sellers is implicitly using a forecast for how likely given teams are to make the playoffs.

There, I highlighted where you went wrong.

 

[SirPsychoSquints]

There, I highlighted where you went wrong.

So - we should throw up our hands? Or what is your approach to estimating how likely a team is to make the playoffs and therefore whether they should be sellers or not?

Meanwhile, of course I disagree with your implication, that the projections are useless. The best arguments anyone has posted in here that they are useless are anecdotes of when the less likely outcome has come to pass. It's like saying there's no point in using polling because the 2016 election came out on the less likely end of things, or saying a coin is biased because it went heads four times in a row.

 

[Adrian's Dome]

So - we should throw up our hands? Or what is your approach to estimating how likely a team is to make the playoffs and therefore whether they should be sellers or not?

Meanwhile, of course I disagree with your implication, that the projections are useless. The best arguments anyone has posted in here that they are useless are anecdotes of when the less likely outcome has come to pass. It's like saying there's no point in using polling because the 2016 election came out on the less likely end of things, or saying a coin is biased because it went heads four times in a row.

Again, I highlighted where you went wrong. Those are 50% outcomes.

We have tons of information to work with, given context. Things like manager tendencies, player tendencies, scheduling, the trade deadline, run differentials, SoS, injuries, attitudes, hot streaks...your projection systems are worthless because they lack to account for all those (very real) variables, and they are not "the best" indicator of anything. One of the systems you constantly cite pegged the Sox for 88 fucking wins before the season began. One right now is telling me the Nats have a 50% playoff odds, the other 20%. Why are we putting weight behind this garbage, let alone trying to push it as anything definitive? It's guesswork, and it's not particularly good.

You continually cite these systems' numbers as if they are meaningful, truthful, or predicative. If you do so, I'll continue to point out that they're worthless, just like the opinions of the "analysts" on sports broadcasts.

 

[SirPsychoSquints]

Again, I highlighted where you went wrong. Those are 50% outcomes.

We have tons of information to work with, given context. Things like manager tendencies, player tendencies, scheduling, the trade deadline, run differentials, SoS, injuries, attitudes, hot streaks...your projection systems are worthless because they lack to account for all those (very real) variables, and they are not "the best" indicator of anything. One of the systems you constantly cite pegged the Sox for 88 fucking wins before the season began. One right now is telling me the Nats have a 50% playoff odds, the other 20%. Why are we putting weight behind this garbage, let alone trying to push it as anything definitive? It's guesswork, and it's not particularly good.

You continually cite these systems' numbers as if they are meaningful, truthful, or predicative. If you do so, I'll continue to point out that they're worthless, just like the opinions of the "analysts" on sports broadcasts.

Scheduling, run differentials, SoS and injuries are all explicitly in the BP & Fangraphs systems.

Manager & player tendencies should be accounted for in the underlying performances, as should attitudes.

The trade deadline is definitely not accounted for, because there is no reasonable way to project that, and hot streaks are not accounted for because most data we have says they're not valuable to predicting the future.

To the extent anyone wants to make adjustments for what's not accounted for, that's awesome. To the extent we want to tease out why two systems have different outcomes, that's a fun discussion to dig into. If someone wants to talk about what's wrong with their method, I'm very interested to hear it.

Separately, on the Nats point, BP is giving the Nats a 17% chance of making the playoffs, and FG is saying 43%. They have a large disagreement about how good each of the NL East contenders are going to be the rest of the year - my guess is that FG is weighting past performance more and BP is weighting current performance more. I think that's pretty interesting, personally, and I think others also do. If you don't, that's cool - but your mission seems to be telling other people not to discuss it.

I don't think they're definitive - I think they're the best estimates we have, which add color and context to many discussions. I think they have some meaning, have some truth, and are as predictive as we have at our disposal.

 

[AB in DC]

Separately, on the Nats point, BP is giving the Nats a 17% chance of making the playoffs, and FG is saying 43%. They have a large disagreement about how good each of the NL East contenders are going to be the rest of the year - my guess is that FG is weighting past performance more and BP is weighting current performance more.

See, that's the only interesting question here. Is there something that Fangraphs is seeing that we're not? If so, what is it? If not, then we should disregard.

 

[jablo1312]

I imagine that BBRef and Fangraphs use similar methodologies to arrive at their playoff percentages AFTER determining each team's strength (expressed as "expected rest of season winning percentage"). That is, they feed the current standings and team strengths into a "remaining schedule" simulation and repeat it until they have a large enough sample size to feel confident in the output. It seems tough for their simulation methodology to be different enough to be driving the large discrepancy in some teams playoff odds.

The key difference, though, is their estimate of team strength. Fangraphs (who I have assumed had the better of the two models here, just based on them updating it more recently) has the Nats rest of season win percentage at .578%. BBRef has it at .533%. They also differ on the Phillies (.552% vs .512%) and Braves (.476 vs .509).

Those are significant enough difference in expected winning percentage to drive a lot of the difference in playoff odds imo. I'd be interested in learning whats creating that difference.

Edit: somehow missed that SPS & AB hit on this point as well. I will say that I think both forecasts have some value. Also (this is a tad anecdotal) neither seems to mirror the betting markets for Divsion winners better than the other- I can remember instances where Fangraphs was very similar to the market and BBRef was completely different, and vice versa.

 

[SirPsychoSquints]

See, that's the only interesting question here. Is there something that Fangraphs is seeing that we're not? If so, what is it? If not, then we should disregard.

Jay Jaffe wrote a post on Fangraphs that includes his skepticism of their number.

https://www.fangraphs.com/blogs/selling-is-the-right-move-for-the-nationals/

Entering Monday, our playoff odds showed the Nationals with a 33.7% chance of winning the NL East and an 8.8% chance at the Wild Card, largely because of their players’ strong projections and the relatively weak ones for the Phillies and Braves, both of whom are seen as likely to play sub-.500 ball for the remainder of the season:

A 42.5% chance at the playoffs seems overly optimistic, to say the least. But by how much?

 

[moondog80]

The Yankees are currently 4 ahead of Oakland and 5.5 ahead of Seattle in the WC race. BP has them at 99.9% to make the payoffs; they quite likely still make it, but 1 out of 1000 for them to miss out has to be a low estimate, right?

 

[grimshaw]

The Yankees are currently 4 ahead of Oakland and 5.5 ahead of Seattle in the WC race. BP has them at 99.9% to make the payoffs; they quite likely still make it, but 1 out of 1000 for them to miss out has to be a low estimate, right?

Yes. Especially since they still have 3 head to head meetings with both of them.
I'm fully on board the fangraphs/BP estimates are (near) meaningless train.

 

[moondog80]

Yes. Especially since they still have 3 head to head meetings with both of them.
I'm fully on board the fangraphs/BP estimates are (near) meaningless train.

OTOH, the M's and A's play each other 10 times the rest of the season, so it's going to be a bit more difficult for *both* of them to gain a bunch of ground. Best case scenario is that they both automatically get saddled with a 5-5 stretch.

 

[SirPsychoSquints]

OTOH, the M's and A's play each other 10 times the rest of the season, so it's going to be a bit more difficult for *both* of them to gain a bunch of ground. Best case scenario is that they both automatically get saddled with a 5-5 stretch.

This is the one. It would take a very putrid stretch from the Yankees to lose to both.

 

[moondog80]

Astros, with a 1 game lead, are 90% likely to win the division.
Red Sox, with a 10 game lead, are 96% likely to win the division.

The Astros have a +124 run diff over the A's, the Red Sox +71 over the Yanks. So there's that. But still, the Sox probability seems about right, while the Astros' number looks really high.

https://www.fangraphs.com/standings/playoff-odds

 

[jon abbey]

Astros, with a 1 game lead, are 90% likely to win the division.
Red Sox, with a 10 game lead, are 96% likely to win the division.

The Astros have a +124 run over the A's, the Red Sox +71 over the Yanks. So there's that. But still, the Sox probability seems about right, while the Astros' number looks really high.

https://www.fangraphs.com/standings/playoff-odds

Not defending those numbers but HOU is expecting Altuve back very soon, I believe. They are 6-11 without him and obviously much much better with him.

 

[Murderer's Crow]

I was listening to MLB today and AJ Hinch was talking about Houston's current struggles and it sounded just like something Boone would be saying. He basically said "we're on track for at least as many wins as last year, we have a team who knows the grind of a long season and when I look at our wins, we're right up there with the elite teams. I can't worry about what the teams around us are doing when we're not playing them." Houston are losers of 5 in a row.

 

[SirPsychoSquints]

Astros, with a 1 game lead, are 90% likely to win the division.
Red Sox, with a 10 game lead, are 96% likely to win the division.

The Astros have a +124 run diff over the A's, the Red Sox +71 over the Yanks. So there's that. But still, the Sox probability seems about right, while the Astros' number looks really high.

https://www.fangraphs.com/standings/playoff-odds

BP is suggesting 89% and 98%. I think you're understating the difference between these amounts. The projections are saying there's ~10% chance of the Astros losing the division and ~3% chance of the Sox losing the division - the Astros are in 3 to 6 times as much danger as the Sox. That seems pretty reasonable.

Also - we all agree the Astros are a meaningfully better team than the A's, right?

 

[AB in DC]

Another data point:

As of September 14, the Cubs were 1.5 games ahead of the Brewers with about two weeks left. Fangraphs listed the Cubs odds of making the ALDS at 90%, with Milwaukee at 60%.

Is it truly hindsight to say that that the 90% was a bit high? It's not like the Cubs had any kind of epic collapse, nor did Milwaukee go on a bonkers winning streak.

At the time, they gave the Cubs a 77% chance at winning the division. So implicitly a 13/23 (57%) chance of winning the wild card. The latter number seems reasonable. But a 77% chance of the division, 1.5 games up in mid-September? That seems awfully strong. Looking at their "expected wins" graph, Fangraphs never really considered Milwaukee to be a serious contender until September, even though they were neck-and-neck with the Cubs for most of the year.

When you think about the earlier posts about the Nats' playoff odds being awfully high, it really sounds like Fangraphs was over-weighting their preseason projections and under-weighting the team's performance to date. At the All-Star break, Milwaukee had the second-best winning percentage in the NL, while the Nats were at .500. But Fangraphs gave the Nats a higher playoff odds probability (60%) than the Brewers (50%).

By contrast, BP had Milwaukee at about 67% with the Nats around 24%, which seems much more reasonable. So this seems like a clear flaw in the FanGraphs algorithm.

 

[John DiFool]

They also had this past weekend the Dodgers w/ the best NL World Series odds-even when Colorado was leading the division. Presumably having a fairly high chance of going to the play-in game and risking a one and done would have significantly cut their odds, eh wot?

Meanhoo while the Dodgers still had those 15% WS odds, the already-clinched Braves were mired at 3%.