It really was the anti-perfect storm of timing for the Patriots. Coming off of 3 SBs in 4 years not long ago, there was Patriots fatigue. The same way people were sick of the Cowboys in the 90's, people were sick of the Patriots. So any story with a negative edge was going to get huge traction. Naturally, it blew up and people clung to the worst of all possible interpretations of what happened.
Then, on top of that, the 2007 season happened. Bullies. Poor sportsmanship. Runner-Uppers. So now you had cheating, poor sportsmanship, and this face:
...and congratulations, you've got yourself a narrative! They're the bad guys! Once that started seeping into every column, report, interview...it just became fact. And since people have grown to hate the Patriots, they're in no rush to go out and find out if it's true or how valid it is, it's just accepted. Then it just feeds on itself and compounds.
Brady didn't shake hands? Well, they're the bad guys, so it must have been because he's a baby, not because he's competitive and takes losses hard. Belichick didn't answer a stupid question? Well, they're the bad guys, so it must have been because he's an asshole, not because he doesn't discuss injuries. The Patriots offense is very complex? Well, they're the bad guys, so it must be because they're arrogant, not because it's been wildly successful. Rinse. Repeat. Ad infinitum.