Lol, wut? Fans can, and in fact do, react differently when different caliber of players leave and (shock) it does depend on where they go. I don't know anyone who started following the Dodgers once Nick Punto left, but I know many Red Sox fans who, like me, still like to see Mookie Betts do well given the caliber of player (and person) he was while here, as well as for his role in 2018. Also, I hated to see Mo Vaughn leave but when he did I wanted to see him do well with the Angels, but Wade Boggs was dead to me the second he signed with the MFYs.
I don't think it should be that hard to wrap your head around why many Patriot fans who generally don't care that much when players leave might feel differently about Tom Brady. I will never not be a Patriots fan, but I'll always love Tom Brady and there is nothing he could do (short of awful, criminal stuff) that could ever cause me not to love him. Since he went to a team that I couldn't give two shits about, I find it pretty easy to continue to root for him so long as he's not playing against the Patriots. If he had gone to a division rival I would still love him, but it would have hurt a little, and I could not have continued to root for him and would have been hoping for him to move on or retire.
I respect the fact that you don't care once they are gone, but your "pick a position and stick to it" thing is silly.