1. The walk rate isn't in a 5-year steady decline - your graph makes it seem as though it is, but it isn't. BB% over 10% is excellent, and the fluctuation from 2004-2009 was negligible - more than covered by statistical error.. There was a slight dropoff in 2010-11, then a big drop off this season. The dropoff in OBP from 2011-2012 was less than the drop of of SLG.
The slight drop off was 1.2% from the year prior and .9% from the average of the previous 5 years where they were over 10%. More importantly, that drop was sustained. And even if the amount of the drop off this year was an outlier, they Sox would have continued their BB% decline into this year as well. There was only 1 month were their walk rate was above 8%, and that was March and April. After that the next two months were at 7.4% and 7.9%, respectively, before dropping off into the 6% range and below. So even before the trade and bad September, this team was continuing to walk at a decreasing rate.
But, as an aside, I think you're right that declining from 10.7% to 10.1% is not a big deal because those are both really good, but declining from a peak of 10.7% to 9% is more troubling. While still above league average, it shows that something has happened. That something is what I'm trying to investigate.
2. There is no need to normalize BB% because there are no mitigating factors that make it necessary. Walks have nothing to do with era or ballpark dimensions or climatalogical factors, league talent dilution etc., As Cliff mentioned earlier, one can certainly remove IBB's from the equation if one wanted to, and they could be weighted similarly based on game situation, batting order position, pitcher BB propensity etc, but that would require grainer adjustments than adjusting to league averages.
I normalized the data to look at the differences between the Red Sox and league average. There are many reasons to normalize the data, and I agree that walking has nothing to do with park factors or climatological factors. I think era is a different story as walk rates were consistently higher from about 1935-1960, before dropping, and then rising again. But one useful way of comparing the Red Sox to the rest of the AL is to normalize the data. Like I said before, if the Sox are walking at the same rate, and the league is getting better, then the Sox are losing ground against their opponents. Also, the normalized graphs show a visual representation of this. If the line is sloped upwards, then the Sox are doing better than the league, if it is flat then they are doing the same as the league, and if it is declining, then they are doing worse. Its a nice way at looking a trends.
However, I have gotten feedback on this from many people. And the consensus is that we're both right
. Normalizing the data is not necessary for BB% because there is not a lot of variance from year to year; but it provides value when trying to evaluate performance against peers. From now on, I'll post both the raw BB% and the normalized BB%.
And I think for BB% it matters because the normalized OBP show a 5 year decline, whereas the normalized BB% show a different decline:
Because the AL, starting in 2005, had rising BB%, the Sox BB% showed a slight decline. The normalized BB% show a decline that happens in 'shelves,' with the more dramatic declines happening in between 2009-2010 and onward. There is that uptick in 2011 because although the Sox showed a negligible decline from 9.2% to 9.0%, the league saw a drop of 8.5% to 8.0%.
All this should be with context too. From when the ownership took over to 2007, the Sox enjoyed an increase in BB% followed by two years at that peak of over 10%, and also saw a rise on OBP from 2002-2007. In 2008 and 2009 the Sox saw stabilized on base production, albeit at slightly lower levels, but still elite. And from 2010-2012, we've seen declining on base production from elite to good, and then to awful this year. And like I said earlier, I think 2012 is an outlier in terms of how bad it was, but even without the big drop off in September, this team would have added to the decline.
It was the winter after the 2009 season where the Sox started to vocalize shifting from on base production to preventing runs via improved defense. If you look at the data for 2010-2012, you'll see on base skills have likewise declined from 2010 forward.
True, the 2011 team was a historically good slugging team, but 2007 was one of the lowest of the ownership's tenure (but still good by historical standards). The difference in SLG this year is due to a drop in home runs and a historically low number of triples hit. And if you exclude September, the slugging average was the same as 2006 (.435), which isn't great, but not an extreme outlier that turned out to be with September included. The low slugging this year definitely is a factor in the poor offensive production this year, but over the long term, it was actually improving from 2006-2011. To me, its a concern, but not in the same way as on base skills.
3. For all the talk early in the season of record pace doubles rates, in the end it was a fairly ordinary Red Sox 2B rate (.055 per PA, same as 2011 and pretty much the same since 2007. Park factor has a lot to do with Red Sox high doubles rates and there is no reason to believe the 2012 Sox lineup was especially proficient as a power unit.