I've had better luck when I stream from NESN.com rather than the NESN android app (same log-in). Still not great, skips here and there, some weird distortions, but largely it's a cohesive stream. My App experience, casting to the TV, was consistently horrendous.
On a slight tangent, the thing that gets me is that YTTV originally was $40/month. It didn't have everything, like nickelodeon and comedy central or whatever, but it did include all the sports you could conceivably want. Their entire market campaign was "watch like a fan" and it was a total no brainer. Then it incrementally went up twice to where it is now as they added new channels (ones honestly I don't really care about). Now, as adoption is up, they cut the one service that got me to buy in originally, and don't pass those cost savings onto the consumer. Kind of like how my experience with Project FI became more like my experience with the cell phone companies as their adoption grew and they rebranded to Google Fi, the experience with YTTV is slowly becoming more like the experience with the major cable providers. I'm largely glad in the ways google moves these markets forward, but it's disappointing the way they've evolved the business model. You could say that, I'm not a fan.
/end rant