This is mostly true. Nobody is buying into breaks thinking they have a realistic shot at the 1/1 Wander. There are a lot of other cards of his that still would turn a huge profit, and are more likely to be hit. As you note in your last paragraph, the idea that everyone knows the Wander 1/1 got hit is foolish. I would bet maybe less than 10% of the sports card market will even realize that card got pulled for a while.I think it moves the needle to some extent. Not an expert on this, but my impression is that people Wander hunting are doing it for short-term gains and likely aiming for parallels - the base are moving but the sheer number of them and ungradeds going for like 10-15 on Ebay isn't making up for these slots valuable in the long run. Most I imagine would know the 1/1 is a complete absurd long-shot but it's also likely the most expensive card in the set, so losing that top prize lottery ticket is going to take it down. We've also seen some of the other biggest hits pulled and gone to market (2 of the #10 acetates, several 1/1s (sketch, autos, etc.), a decent number of the /25 and /71) - basically the cards that make the whole thing back and then some.
I still think the prices will be kind of dumb on these but not having those biggest chase cards feels like it has to hurt the overall value of the slot...if the buyer knows it.
And in sports cards, "a while" is about a month, and by that time, you won't even see many Topps 2022 being broken, because by then, you'll have another half dozen products released by Panini or Topps/Bowman. Basically, a new product is released by these companies on a weekly basis, and the new products dominate the breaking market for about a week, and then the next one starts. Prices on these cards, or any cards, are at their peak in the 1st couple of weeks of the product release. After that, and as cards hit the market (Ebay), prices will come down, particularly on the base stuff (again, unless your name is Trout, Brady, Mahomes, Lebron, Luka, etc.), so the Wander 1/1 getting pulled will have no impact on pricing until pricing inevitably falls as it normally would.