What's amazing is Tom Brady had a better passer rating last year, at age 45, than he did in his first three full seasons. But Brady wasn't a top QB until his 5th season. Those all star votes in 2003 were for the SB wins. He threw for 3600 yards and 23 TDs and 12 INTs. Obviously the NFL was different but those are pedestrian. Brady was 6th in the NFL in yards that year.
A solid year, but not a no doubt top QB. Until 2005. And even then, his numbers in his first 6 seasons look absolutely nothing like the numbers in his career from 2007 on. They are a different player altogether. It's like looking at Rafael Palmiero's stats in his career. He was one player, a good one, and then a total monster with eye popping numbers, a totally different player.
Of course, Brady did it with sunburn resistant water and bands and Palmiero did it with steroids. But looking at his career stats is a trip.
In 2002, Brady's second year as a starter, his 28 TD passes led the NFL.
In 2003, Brady's 23 TDs were 10th in the NFL. The league leader was Favre, with 32.
In 2004, Brady's 28 TDs were 8th in the NFL. Manning went bonkers that year with 49.
In 2005, Brady's 4,110 passing yards led the NFL.
Brady's pre-2007 numbers seem pedestrian to us because of what the league has become, but back then those were excellent passing stats. He also finished top 3 in MVP voting twice - in 2003 and 2005.
But yeah, his crazy leap statistically was mainly due to rule changes that altered the whole NFL passing landscape, plus he had some elite receivers in 2007-2009. By then the offense was built around Brady and he had become a stat monster, and he added guys like Gronk and Hernandez and such.
The leap in stats though was mainly reflective of an entire change in the NFL.