63 replies to this topic

[ScubaSteveAvery]

One thing I noticed when comparing the 2012 Red Sox to previous Red Sox teams is the 2012 team was the worst team in terms of getting on base in the post-WWII era. I was curious if this was an outlier or a general trend. So to start, I decided to look at 4 stats over the course of the current ownership from 2002 to present. The four stats are OBP, BB%, Swing%, and P/PA. The purpose of this thread is to use SoSH's collective awesomeness to figure how what is wrong with the Red Sox and getting on base? This post is only an initial look at the data so please add to it with your thoughts and ideas.

OBP


As you can see, the Sox peaked in 2007 and have been on a fairly steep decline since, with a slight uptick in 2011, before continuing a downward trend. It also looks like in the years the Sox seriously contended for a title, 2006 notwithstanding, the Sox OBP was above .350. The two championship years saw .360 or above. It'd be interesting to see if .360 is a general elite shelf, with drop offs after.

BB%


Like OBP, BB% peaked in 2007 and has been in a steady decline since. It was on the rise until 2007 as well, as if always getting better under Theo's tutelage. The drop off from 2009 to 2010 was severe, only to be bested by the serious drop off from 2011 to 2012. Unlike OBP, there was no uptick in 2011, just more decline (although slightly less decline than the year prior, and after. Again, the elite seasons saw BB% above 10%, so it could be another elite shelf in team batting performance.

Swing%


I found this graph to be interesting, as unlike the previous two charts above, the Sox have not gotten worse over the current ownership's tenure. Higher swinging percentages didn't stop the 2003 and 2004 teams from becoming elite hitting machines, but it could indicate that they were more aggressive and just good at being aggressive. The swing% declined until 2009, and have been going up since, with a big jump between 2010 and 2011.

P/PA


P/PA over the last 10 years shows a general upward trend, peaking in 2010 at over 4 pitches a plate appearance. It has been on a small decline the last two years to levels seen in 2008 and 2005. I'm not sure this really tells us much, other than this team didn't see an uncharacteristically low number of pitches per plate appearance that would show less discipline at the plate. Although, in the 10 year sample, it is the only time there was decline two years in a row, but I doubt that is significant.

There probably needs to be more data to hone in on the problem(s), but trends are clear: steep decline in OBP and BB%, higher swinging percentages, and a small, but potentially meaningless decline in P/PA. I find most troubling the steep drops in OPB and BB%. Did the front office abandon the high OBP mantra that served it well for a majority of the last 10 years? It sure seems like it. Also of note, is that Dave Magadan became the hitting coach in October 2006. Since he came on, we saw a peak in 2007, but declines since. Does anyone know more about his philosophy?

Edited by ScubaSteveAvery, 10 October 2012 - 08:51 AM.

[Paul M]

I think this skill is one the Sox used to own--getting on base, working counts, and command of the strike zone-- and it's slipped as there has been more attention paid to preventing runs than creating them and some players have seemed to regress some and. My only data question is have you explored ways to tease out the macro trends in baseball that explain part of this decline, at least as far as OBP is concerned. Perhaps dividing the decline in OBP over the league average or something to neutralize the effect?

[ScubaSteveAvery]

I think this skill is one the Sox used to own--getting on base, working counts, and command of the strike zone-- and it's slipped as there has been more attention paid to preventing runs than creating them and some players have seemed to regress some and. My only data question is have you explored ways to tease out the macro trends in baseball that explain part of this decline, at least as far as OBP is concerned. Perhaps dividing the decline in OBP over the league average or something to neutralize the effect?

I didn't do that, but I just calculated it simply by dividing the Sox OBP over the MLB OBP average for 2002-2012. This might be a little noisy due to NL vs AL, park effects, etc., but its quick and dirty. 1 would obviously equal league average:



So it does appear that from 2007, there is a general decline downward, notwithstanding that big uptick in 2011. I've also included a chart showing Red Sox OBP vs. MLB OBP average from 2002-2012:



Interesting that the decline in MLB OBP started in 2006 and has dropped every year since, with 2008 and 2009 staying flat. The neutralized chart confirms that the slope of the Sox decline has outpaced MLB decline since 2007, again barring 2011's outlier year.

Edit: Made the neutralized chart bigger so as to read the table at the bottom better.

Edited by ScubaSteveAvery, 09 October 2012 - 11:31 PM.

[curly2]

OBP has been dropping around MLB, perhaps because pitchers are more willing to throw strikes in the age of drug testing. But now matter the trend, this season the Sox were 29th out of 30 teams in walks. Only the Royals drew fewer walks. Even with injuries, that's unacceptable.

Paul is right, the Red Sox used to own it because it seemed like the stressed it throughout the system. In recent years, for whatever reason, that has changed, with guys such as Reddick and Middlebrooks continuing to be promoted despite poor plate discipline. Shouldn't you have to prove you can do more that just HIT to be promoted? Don't you have to show you can work counts?

In addition, the organization went out and gave a huge deal to Crawford, who never walked a lot, and traded for Avilies, who hardly walks at all. Even Adrian Gonzalez mysteriously stopped walking.

Don't get me wrong. I don't want guys LOOKING for walks. Ted Williams didn't say, "Go up and draw a walk," he said, "Get a good ball to hit." The only Sox player I remember who I think went up looking to walk was Jeremy Giambi, and he let a ton of hittable pitches go by. They need to focus on working pitchers but also knowing WHEN to let it fly.

And it has to be re-emphasized from the Gulf Coast League to Boston.

[JakeRae]

I didn't do that, but I just calculated it simply by dividing the Sox OBP over the MLB OBP average for 2002-2012. This might be a little noisy due to NL vs AL, park effects, etc., but its quick and dirty. 1 would obviously equal league average:



So it does appear that from 2007, there is a general decline downward, notwithstanding that big uptick in 2011. I've also included a chart showing Red Sox OBP vs. MLB OBP average from 2002-2012:



Interesting that the decline in MLB OBP started in 2006 and has dropped every year since, with 2008 and 2009 staying flat. The neutralized chart confirms that the slope of the Sox decline has outpaced MLB decline since 2007, again barring 2011's outlier year.

Edit: Made the neutralized chart bigger so as to read the table at the bottom better.

It's not at all clear that 2011 is the outlier rather than 2012. If you look at 2007-2012, there is a clear downward slope with 2011 looking like an outlier, but if you look at 2002-2012, there is no real pattern from 2002-2011, just an oscillation from 4% to 8% above average, followed by a precipitous drop in 2012. It's an interesting hypothesis and I actually expected to see a downward trend, but I'm not buying it based on the data.

[smastroyin]

The Red Sox have clearly not been elite in this are for a couple of years. The question I suppose is whether this is a trend of design or something else. The real question is whether 2012 is instructive to the future (as it's just one year). Previous years we can talk about but I'll be honest, a quick perusal and I don't see a problem in identifying talent (when compared to available talent of course). The big outlier to me in the Theo years is the money they paid Crawford - way too much money for a guy with less than elite OBP skill (and one that was largely reliant on BA as well). And yes they signed Beltre which people thought was indicative of an abandonment of OBP, but he was great for the Red Sox and has been just as great for Texas the last two years, so sure, maybe not an elite OBP player but the best player available at the time (and as it turns out, the Sox would likely have been better off keeping him even at his contract than what they ended up doing, but water under the bridge and all). In contrast to this, they did spend a lot of resources on Adrian Gonzalez, a high OBP slugger. They gave Scutaro, an OBP biased player, the SS job in 2010. They signed Mike Cameron, whose despite a low BA and high number of K's was traditionally a higher than average guy for walk rate and OBP.

In 2012 even they did jettison Reddick, a low OBP slugger. Despite his hot start he ended up basically in the same area of player he was with the Red Sox in 2011, albeit trading a bunch of singles for more HR.

However, there are some disturbing things from 2012:
- Choosing Aviles over Scutaro. I realize there are some who think Scutaro was unable to play SS anymore and that's fine. But in terms of the guy you would want around as a MI Scutaro is clearly more in the philosophy of getting on base and working counts. Aviles derives nearly all of his value from AVG and hitting HR at a decent rate for a SS. This year he was unable to keep his BA above 250 and he was a hole in the lineup.
- Dispensing of Kevin Youkilis like he was trash. I don't want to re-hash this yet again, but from the team's statements (or Valentine's) it seems there was a lot of focus on his batting average. He was not an elite player with the White Sox and his slump at the end (his OPS dropped 40 points in September) was part of the reason they faded. But he was still good at getting on base (beyond the BA) and in terms of valuing skill sets, they chose WMB who projects more as a low OBP slugger.
- Salty over Lavarnway. I have been one of Salty's biggest fans and Lavarnway slumped as well so it's hard to really put much here. But, just in terms of team philosophy they continued to run Salty out there with his sub .300 OBP for the last two years. I don't know if Lavarnway is the answer but they don't seem to feel like giving him the chance.
- Adrian Gonzalez walk rate. Guys have bad years all the time, but it seemed that the Sox approach with Gonzalez was to get him hacking more to find his stroke. He didn't do much better in LA in the aggregate, so maybe this is on him.
- Who the fuck knows what is happening with Jacoby Ellsbury.

There are obviously extenuating circumstances for these things but I hope they are not going to continue too much of this trend.

All of that said, put 2011 Gonzalez and Ellsbury on the team and have Ortiz play the full year and you are right there with the 2011 team. And it's hard to suss out what was really going on with Gonzalez and Ellsbury (obviously he had the injury, but he looked awful coming back).

[sachilles]

I just did a quick chart limiting it to the AL east. Certainly seems like a downward trend from 2009 for the AL east.

[ShaneTrot]

I have no statistical evidence for this and I don't know if it is available but this team always seemed to be playing from behind. I wonder if that effects OBP because guys may be pressing to put the ball in play.

[ScubaSteveAvery]

It's not at all clear that 2011 is the outlier rather than 2012. If you look at 2007-2012, there is a clear downward slope with 2011 looking like an outlier, but if you look at 2002-2012, there is no real pattern from 2002-2011, just an oscillation from 4% to 8% above average, followed by a precipitous drop in 2012. It's an interesting hypothesis and I actually expected to see a downward trend, but I'm not buying it based on the data.

Your point made me think because it does look like there is little clear directional movement until 2012. So I split the data 3 ways on the neutralized OBP and ran three linear trend lines for each. I looked at the whole 2002-2012 period, and because I think the peak of the Sox OBP skills occurred in 2007, I ran a 2002-2007 and 2007-20012 line. I took the equations and did a simple graph to illustrate the slopes of the lines from 0-10 in order to get a clear picture of each:


I found that from 2002-2012, the Sox' neutralized OBP showed an overall downward trend, but only slightly. When I isolated the two periods around the peak, I found, somewhat surprisingly, that the neutralized OBP actually improved from 2002-2007, bucking the general trend of slight decline. This is even more impressive considering the Sox were improving against a MLB wide downward OBP trend.

And from 2007-2012, I found that, as expected, the decline in the Sox OBP outpaced the general downward trend of the whole period by quite a bit, and this includes that big jump in 2011. Without 2011's outlier, the slope would be much steeper. And this holds for the 2002-2007 period too. Without the dip in 2006, the improvement would be even better. And the fact that the 2007-2012 moved the data line from positive to negative, shows how strongly the decline from 2007-2012 affects the whole period (if that makes sense).

I think this shows that there has been a material negative difference in the Sox on-base skills since 2007. And, that from 2002-2007, the Red Sox as a team were defying league wide trends of declining OBP by improving against it.

Edit: Fixed the grammatical error in graph title.

Edited by ScubaSteveAvery, 10 October 2012 - 10:38 AM.

[Rough Carrigan]

I have no statistical evidence for this and I don't know if it is available but this team always seemed to be playing from behind. I wonder if that effects OBP because guys may be pressing to put the ball in play.

I was going to suggest that, too. Up 4-1 in the third inning, it's a bit easier to just watch that tough pitch on the corner. Down 4-1, players might feel some urgency to do something.

I would also suggest that they haven't had as good a lineup the last couple years. The more the really good hitters feel like the burden is on them to make things happen the more they might swing at borderline pitches. Whereas in a 2003-2005 lineup, there were so many good hitters that everyone knew the job would get done even if they didn't do it that particular night.

[JakeRae]

Scuba, what happens if you look at 2002-2011 as a period? Is there a decline during the Theo era or are you simply seeing a trend because of the inclusion of 2012, which clearly looks like an outlier. 2012, as the first post Theo year, may also mark the start of a new era, so I'm not arguing that we should not be concerned that the organization has shifted away from properly emphasizing getting on base, just that I don't see anything in the 2002-2011 data set to indicate that this was really happening. What we had seen under Theo was a fair bit of stability with sharp upward corrections after ever "low" OBP year.

It's unsurprising that the full data set shows a downward slope. If you take a small data set and throw an extreme outlier at the end of it, you are always going to see a trend in that direction for the full data set.

[Rudy Pemberton]

I don't think it's really anything to do with coaching; it's personnel based. Part of it may be related to focusing on good defensively players, who often don't get on base at the kind of clip as guys who aren't as good defensively, but it's mostly losing the guys who were strong at getting on base and replacing them with guys who don't (duh).

Look at that '07 team; you had Tek (367), Youks (390), Lowell (378), Manny (388), Drew (373), Pedroia (380) and Ortiz (445!). Lugo (294) and Crisp (330) didn't get on base a ton; but only Lugo was really well below average.

The '12 team, by comparison had Salty (289), Gonzalez (343), and older and hobbled Youks (315) & WMB (325), Crawford (306), Ross (326), Pedroia (347), and Ortiz (415). Aviles (282), and Ellsbury (313) were actually worse than Lugo and Crisp in '07.

So, the Sox have basically downgraded every position in 5 years (and to be fair; looked at expected regulars). Their best OBP guy in '12 would have been their 3rd worst in '07 (and now, he's been traded). I think that having guys who don't get on base has put more pressure on guys like Pedroia, who is likely pressing now that he's counted on to do more. I don't really know how much you can pin on Magadan, though....the guys they've acquired are generally guys who don't get on base. What do you expect?

[ScubaSteveAvery]

Scuba, what happens if you look at 2002-2011 as a period? Is there a decline during the Theo era or are you simply seeing a trend because of the inclusion of 2012, which clearly looks like an outlier. 2012, as the first post Theo year, may also mark the start of a new era, so I'm not arguing that we should not be concerned that the organization has shifted away from properly emphasizing getting on base, just that I don't see anything in the 2002-2011 data set to indicate that this was really happening. What we had seen under Theo was a fair bit of stability with sharp upward corrections after ever "low" OBP year.

It's unsurprising that the full data set shows a downward slope. If you take a small data set and throw an extreme outlier at the end of it, you are always going to see a trend in that direction for the full data set.

It cuts both ways though, since 2011 was the best overall year during the current ownership, and highest since 2006. And if there is a downward trend starting after 2007, and ending on a really high year (2011) then it would skew the overall line upward. And the data bears this out:



If we look at 2002-2010, we see a decline, although not quite as steep as the 2002-2012 line. But, if we go one year further, 2002-2011 shows a small positive slope, so the excellent 2011 year definitely skewed the data, but not as much as 2012. Although, it does show that the two outlier years on the opposite ends of the spectrum really skew the data in 2011 and 2012. This, to me, suggests that there was a downward trend in OBP, that was reversed for a single season in 2011, before resuming at a much steeper pace in 2012. The amount of decline in 2012 may be an outlier ,but the data still shows an overall downward trend.

Edit: and I think Smas and Rudy have the right idea by looking at the players and personnel decisions that caused a decline.

Edited by ScubaSteveAvery, 10 October 2012 - 12:20 PM.

[C4CRVT]

1. This is interesting data that illustrates that the Red Sox have strayed from a previously sucessful organizational strategy and are paying the price. The general trend within the league is a lower run scoring/OBP due to better pitching/ worse hitting/ less steroids whatver but the de-emphasis within the team has hurt the team's ability to be an elite team.

2. Great data that is kind of re-stating the obvious: the talent level of the 2012 team was much, much worse than previous years, especially after the trade, the Ortiz injury and the promotion of Iglesias. The last third of the year makes the whole "2012" year not truly indicitiave of the way that the team was built.

I think it's 38% #1 and 62% #2. Having players like Middlebrooks and Reddick come up through the system kind of underscores how difficult it can be to adhere to some sort of strict organizational philosophy about high OBP guys. Do people really think that they weren't getting coached to be more selective? I have a hard time believing that that's not just the kind of hitter those guys are. Are we having this conversation if we keep Reddick, Middlebrooks doesn't get hurt and the RS lead the league in scoring?

Besides, the starting pitching's the real problem with this team anyway.

[Paul M]

The Red Sox were able to buck a global decline in OBP that's partly due to PEDs being eliminated but also because almost all teams are using an extra reliever or two and managing them aggressively which is leading to more strike-outs and lower OBP. I think to me you can run a ttest and I'm sure you'd find the adjusted mean for 2002-2007 is significantly > than the next 5 year period. Diagnosing that hopefully is the kind of thing some of the staff can do as they game-plan the off-season. The 2003-2004 teams may never be with us again in terms of sheer ability to work counts, pitchers, and score. And I do think there needs to be a good balance between creating and preventing runs. One last theory is that the cost of OBP in the market has gone up so much since the early low-hanging fruit days that unless you develop some of it it's hard to get. OBP is no longer mispriced.

[sachilles]

Hasn't the league also pressured teams to play quicker, resulting in pitchers delivering pitches quicker? One would think that would be an advantage to the hitters, but perhaps it is having the opposite effect.

Edited by sachilles, 10 October 2012 - 01:05 PM.

[Savin Hillbilly]

Hasn't the league also pressured teams to play quicker, resulting in pitchers delivering pitches quicker? One would think that would be an advantage to the hitters, but perhaps it is having the opposite effect.

It may be having another effect as well.

One thing that's kind of fascinating in terms of this discussion is that according to Fangraphs, pitchers have been throwing fewer and fewer strikes over the past five years:

Zone %

2008: 51.1%
2009: 49.3%
2010: 46.5%
2011: 45.3%
2012: 44.9%

That's a pretty remarkable sustained trend, and it seems counterintuitive in light of the downward BB rate trend:

BB rate

2008: 8.7%
2009: 8.9%
2010: 8.5%
2011: 8.1%
2012: 8.0%

If pitchers are throwing fewer strikes, why are they also throwing fewer walks?

The answer is that they aren't throwing fewer strikes--they're throwing fewer pitches in the strike zone, while the percentage of strikes remains dead flat:

Strike %

2008: 63%
2009: 62%
2010: 63%
2011: 63%
2012: 63%

What could account for a flat strike % despite a falling in-zone%? Either the umpires are calling more of a pitchers' zone, the hitters are swinging at more pitches outside the zone, or both.

To zoom in on the issue we can subtract the zone% from the strike% to get the percentage of pitches that were strikes without being thrown in the zone, then divide that by the overall percentage of OOZ pitches (100-Zone%), to get a strike rate on OOZ pitches:

Strike % on OOZ Pitches

2008: 24.3%
2009: 25.0%
2010: 30.8%
2011: 32.4%
2012: 32.8%

Now, this could just be the league's hitters collectively taking a big step backward in plate discipline, particularly in 2010. Let's look at the O-Swing% numbers:

O-Swing%

2008: 25.4%
2009: 25.1%
2010: 29.3%
2011: 30.6%
2012: 30.8%

These numbers do appear at first glance to track the overall OOZ strike numbers, though a bit more narrowly.

However, if you subtract the O-Swing numbers from the OOZ strike numbers to get what we might call the "false positive" called strike rate, here's what happens:

Called Strike% on OOZ Pitches

2008: -1%
2009: -0%
2010: 1.5%
2011: 1.8%
2012: 2.0%

The trend here is clear, though not huge, and it can't be accounted for by batter behavior. It seems reasonable to assume that a similar trend is in play on the OOZ pitches that batters *do* swing at; an increasing number of them are pitches that would have been called strikes by the umps. Batters are just adapting.

This is borne out by the fact that the percentage of strikes on swings, whether contact is made or not (AS/Str in BBref's "Pitching Pitches" section) has remained dead flat at 72% over the five years in question (well, OK, it was 73% in 2008, went down to 72% in 2009 and stayed there). If the increase in OOZ strikes were mostly due to a change in hitter behavior rather than umpire behavior, we should have seen an increase in the AS/Str number (because if pitchers are throwing more pitches out of the zone, and umpires aren't expanding *their* zone, then the called strike rate should be going down; therefore, if the overall strike rate is stable, then the percentage of strikes on swings should be rising).

So the decline in BB rate (and consequently in OBP) is an artifact of umpiring. Hitters overall aren't less willing to take a ball; umps are less willing to call one. And it seems plausible (though hard to prove) that this is related to MLB's desire to shorten games.

Sorry for the long post....and please feel free to correct my math and my logic. I'm an English major.

[JMDurron]

1. This is interesting data that illustrates that the Red Sox have strayed from a previously sucessful organizational strategy and are paying the price. The general trend within the league is a lower run scoring/OBP due to better pitching/ worse hitting/ less steroids whatver but the de-emphasis within the team has hurt the team's ability to be an elite team.

2. Great data that is kind of re-stating the obvious: the talent level of the 2012 team was much, much worse than previous years, especially after the trade, the Ortiz injury and the promotion of Iglesias. The last third of the year makes the whole "2012" year not truly indicitiave of the way that the team was built.

Regarding point #2, it seems potentially helpful to look at the 2012 team's OBP by month:

Mar/Apr: 336
May: 327
Jun: 328
July: 311
Aug: 324
Sep/Oct: 267

Or, using the b-ref game logs, we can divide the season into before/after the August 25th Nick Punto Trade.

Opening Day-Aug 24: 324
Aug 25 - End of Season: 283

I think it's clear that the post-trade lack of talent in the lineup was clearly an issue, but the pre-trade mark of .324 is still easily the lowest since this ownership group has been in place, so it was still a problem beyond the end of season roster issues.

EDIT - To put a finer point on it, the single best month for the 2012's squad's OBP still was worse than the worst other season's OBP from 2002-2011 (2010).

Edited by JMDurron, 11 October 2012 - 08:08 AM.

[C4CRVT]

Regarding point #2, it seems potentially helpful to look at the 2012 team's OBP by month:

Mar/Apr: 336
May: 327
Jun: 328
July: 311
Aug: 324
Sep/Oct: 267

Or, using the b-ref game logs, we can divide the season into before/after the August 25th Nick Punto Trade.

Opening Day-Aug 24: 324
Aug 25 - End of Season: 283

I think it's clear that the post-trade lack of talent in the lineup was clearly an issue, but the pre-trade mark of .324 is still easily the lowest since this ownership group has been in place, so it was still a problem beyond the end of season roster issues.

EDIT - To put a finer point on it, the single best month for the 2012's squad's OBP still was worse than the worst other season's OBP from 2002-2011 (2010).

I'm in the habit of logging on to the ESPN redsox homepage to check the previous day's box score/ look at the upcoming schedule etc. On that page, they post the team's rank for Runs, OBP, Average and SLG.

Hopefully I'm not mis-remembering but for the first several months of the season, the Red Sox were typically ranked top 3 in runs scored and OBP. .324 would have ranked 11th (MLB) so maybe I'm delusional.

I really don't disagree with the larger point that the FO could really stand to re-emphasize OBP as part of their evaluation process/ coaching strategy. I'm just not sure how much of this year's steep drop in OBP can be attributed to the lack of intent to have a high OBP team. Here's a look at how the lineup was initially assembled (2009-2011 OBP):

1. Ellsbury .348
2. Crawford .324
3. Pedroia .368
4. Gonzalez .382
5. Youkilis .372
6. Ortiz .391
7. Sweeney/Ross .333/.324
8. Saltalamacchia .289
9. Aviles .302

The starting lineup was supposed to have an average OBP of roughly.345 which would have led the league by 7 points. Of course there's no bench or injuries factored into that and of course things didn't go according to plan in many ways... but there's a case to be made that the FO thought they were in fine shape WRT OBP.

Edited by C4CRVT, 11 October 2012 - 10:52 AM.

[ScubaSteveAvery]

So the decline in BB rate (and consequently in OBP) is an artifact of umpiring. Hitters overall aren't less willing to take a ball; umps are less willing to call one. And it seems plausible (though hard to prove) that this is related to MLB's desire to shorten games.

Sorry for the long post....and please feel free to correct my math and my logic. I'm an English major.

Savin, It is important to know the problems with using BIS plate discipline data for Zone% and O-Swing% (and I'm guessing the OOZ numbers where you just subtracted Zone% from 1). The data for the "Plate Discipline" section from Fangraphs comes for Baseball Info Solutions (BIS), and there has been noted bias and problems with BIS plate discipline data. They collect information on pitch location primarily using "video scouts" who watch the same video feeds that we do as fans and place balls inside and outside the strikezone based upon the video feeds. Colin Wyers at Baseball Prospectus has written about this quite a bit the last two years (here, here, and here). In 2010, they said that they also used PitchFX to help with the calls, and then there is some evidence that when the bat is not swung the "video scout" uses the umpire as help. There are a lot of problems with this and it comes out in the data. The first is that there are park biases depending on where the main video feed camera is placed. The cameras are almost always placed off center, with zoom, which distorts how the viewer sees the ball. The second is just human depth perception problems, especially on television (turning a 3D image into 2D). The third is frame rate of the cameras. Because the frame rate cannot keep up with the rapid movement of the ball, a video scout may not get to see an image of the ball as it crosses the plate. And finally, humans just aren't as good as PitchFX at determining when something is, or is not, in the strikezone, especially when the ball is swung at and contact is made. Also, there is evidence that the BIS data has different strike zones from year to year, compromising the ability to compare things like Zone% and Oswing% across multiple years.

PitchFX has a similar Plate Discipline section at Fangraphs, although it is kind of hidden in the "PitchFX" section. PitchFX uses a standard strike zone across all years, allowing us to compare year to year, and simply calculates if a ball falls inside or outside the strike zone when looked at, or swung at. The data is much more standardized and subject to a lot less bias. This doesn't mean PitchFX is perfect, but I think its a lot more accurate than the BIS data. I did use the BIS data for Swing%, but only because it went back to 2002, whereas PitchFX only goes back to 2007 (and really 2008 because not every stadium had PitchFX cameras installed in 2007). I also made sure the PitchFX data showed a similar trend from 2008 to 2012, and it did.

A comparison between the two show much more volatility in the BIS data than the Pitch FX data, and in some cases, one could draw completely different opinions looking at the two.

Comparison of Zone% Using BIS and PitchFX

Year	BIS	PitchFX	Difference
2008	51.10%	49.90%	1.20%
2009	49.30%	50.10%	-0.80%
2010	46.50%	50.10%	-3.60%
2011	45.30%	49.80%	-4.50%
2012	44.90%	49.20%	-4.30%

The PitchFX tells a different story than BIS. From 2008-2011 the data moved a tad up, then back to where it started, only to show a slight drop in 2012. No significant downward trend here.

The same goes with O-Swing%

Comparison of O-Swing% Using BIS and PitchFX

Year	BIS	PitchFX	Difference
2008	25.40%	27.60%	-2.20%
2009	25.10%	27.60%	-2.50%
2010	29.30%	28.00%	1.30%
2011	30.60%	28.60%	2.00%
2012	30.80%	29.00%	1.80%

Like BIS, PitchFX shows a trend upward, but much less dramatic and more linear. The BIS has that big jump between 2009 and 2010, and then that jump sustained itself. There has to be an explanation for that. Over at Tango's site, there was a thread in 2011 where he, Colin Wyers, and Mike Fast hypothesized that BIS changed their algorithm, but didn't say anything about it (the BIS data is hard to check for quality because they don't release the raw data). But you can see in the two tables above how much the BIS data moves between 2009 and 2010 relative to the PitchFX data.


I think it is also tricky to compare the data from BIS with the Baseball-Reference Strike% data. B-R uses Retrosheet, and I think the Strike% stat is strikes called by the umpire and those swung at outside the zone. I'm not sure what the correlation of Retrosheet data is BIS or PitchFx. I'm not sure that Retrosheet is looking at pitch location data like BIS and PitchFX. Subtracting Zone% from Strike% to get OOZ Strike% overstates things since Strike% already includes strikes that were outside the zone (either called or swung at).

Overall, I think the main problem is using the BIS Zone% data. If we use the PitchFX data that shows a relatively flat Zone%, and look at the upward trend in O-Swing% (PitchFX), and then look at the decreasing BB%, things start to make more sense. Pitchers are throwing a very similar number of pitches in the zone, but hitters are swinging at more pitches outside the zone, which would correlate to a lower BB rate since plate discipline seems to be declining. It is just really hard to buy the fact that pitchers have decreased the amount of balls they have thrown in the zone by 6.2% over the last 5 years, yet batters continue to draw less walks. And I would also think if umpires are controlling this much of the pitching and hitting games, we would see it in other stats, as well as just by our own observation.

Additional reading on the BIS/PitchFX Plate Discipline
The Hardball Times - "Is the BIS Data Right?"
The Book - "BIS Bias, Bizz Buzz"

Edit: One more thing to note. It is not entirely clear what the extent umpire's role is,if at all, with these plate discipline stats. I emailed Fangraphs about it and will post the reply when I get it.

Edited by ScubaSteveAvery, 11 October 2012 - 10:03 AM.

• Back to top

[ScubaSteveAvery]

Regarding the pace of the game. I don't know of any recent major study on this (correct me if I'm wrong though), but Fangraphs has a simple "Pace" stat that Dave Appleman introduced in 2010 using PitchFX time stamps. Here is his definition:

The way I calculate Pace, is by taking the difference between the start time of the first pitch in the plate appearance, and the end time of the last pitch in the plate appearance. I then divide by the number of pitches in that plate appearance (minus 1). Pickoff attempts are considered just another pitch, since they don’t have time stamps of their own. Anything that looks like a game delay between pitches is thrown out. The average pace is about 21.5 seconds.

As noted in the original article, pace seems like an organizational thing, which could certainly be the result of coaching. Buehrle, Danks, and Richard all started their major league careers with the White Sox and pitching coach Don Cooper. Even Jake Peavy trimmed 1.5 seconds off his pace with the White Sox this year. I’d say it’s worth researching further.

Here are the stats for pitcher pace from 2008-2012:

Pitcher Pace (in seconds) 2008-2012

Year	Pace
2008	21.60%
2009	21.40%
2010	21.50%
2011	21.60%
2012	22.20%

This data probably isn't 100% correct because he counts pickoffs as a pitch, and throws out abnormal delays by hand (but didn't tell us what his cut off is). He also noted team differences in teaching pitchers to speed up and slow down. And finally, I would think that a pitcher's pace would fluctuate within the game to try and speed up and slow down the tempo. Really short games or really long games might skew the data some. Anyways, it is a quick and dirty stat, and shows no major changes over the last 5 years, other than a medium sized jump this year indicating that pitchers actually took longer this year in between pitches. But it shouldn't be treated as gospel.

[Savin Hillbilly]

SSA, thanks for the very informative response. It does look as if the difference between the BIS and PitchFX data pretty much moots my point--I'll take a look later and see if I can salvage any of it! Oh well.

I did want to respond to this, though--

I think it is also tricky to compare the data from BIS with the Baseball-Reference Strike% data. B-R uses Retrosheet, and I think the Strike% stat is strikes called by the umpire and those swung at outside the zone. I'm not sure what the correlation of Retrosheet data is BIS or PitchFx.

My assumption was that BBref's Strike% number includes all (actual, not ideal) strikes, including those called, those swung at and missed or fouled, and those put in play. In short, every pitch that isn't called a ball. So if the umps were calling pitches with 100% accuracy, then Strike% should equal Zone% plus (O-Swing% * (100-Zone%)). I.e., if Zone%=50 and O-Swing%=30, Strike% should equal 65. If it's more than that, umps are expanding the strike zone. If it's less than that, they're squeezing pitchers.

I tried to assess that by working backward from a known Strike%, Zone% and O-Swing%. But obviously if the Zone% number is unreliable, that may not yield a useful result.

Subtracting Zone% from Strike% to get OOZ Strike% overstates things since Strike% already includes strikes that were outside the zone (either called or swung at).

I don't get this. Of course Strike% already includes strikes outside the zone. That's the whole point of subtracting Zone% from it--to isolate those out-of-zone strikes. Then you look at the difference between out-of-zone strikes and out-of-zone swings to see how many out-of-zone pitches are being *called* strikes. Why doesn't this work? (Again, aside from the Zone% reliability issue.)

[zenter]

I didn't see this mentioned in this thread, but apologies if it has already. Also, apologies for spitballing, but I'm trying to take this to an organizational philosophy level.

Since the triumverate took over, the Red Sox philosophy has been quite loud about identifying undervalued statistics and using them in personnel decisions. Up until around 2007, there were still vocal opponents to use of "Moneyball / sabermetrics", including things like OBP (or its relative OPS+). Since 2007, as the charts above show, OBP is valued pretty highly by most of the MLB - BA seems less important than ever even to national broadcasters.

I mean, could Theo have been trying to push the Red Sox beyond OBP, saying "OBP is too-highly valued, but X (defensive metrics, traditional scouting metrics, bunts/SBs, etc.) are woefully undervalued."? Viewed in this context (and assuming it's true), we'd predictably see a decline - though not a cliff-like dropoff - in Sox OBP, greater than that of the rest of the league.

[Paul M]

If the Sox made the decision to reallocate resources toward defense, speed, etc. I wonder if this is where they were not communicating and collaborating as well with James as they used to as perhaps this relies on more soft information. Signing Carl Crawford to that kind of contract screams a shift away from guys whose value is tied up more with their bat than glove/legs. On a smaller scale, the investment in Iglesias and some of the draft picks and other signings suggest it as well. Economically, it does makes sense to move to the next mispriced sector, but I think it's clear the execution was not great and not sure about the process either. Probably not a clean story on this and the implosion on the pitching side from September 2011 to now is the biggest problem. I also imagine if they field a lot of younger players it will take a year or two to ge the OBP back though I guess that also depends on other factors. It will be interesting to see what kinds of players are placed into the openings they have.

[ScubaSteveAvery]

My assumption was that BBref's Strike% number includes all (actual, not ideal) strikes, including those called, those swung at and missed or fouled, and those put in play. In short, every pitch that isn't called a ball. So if the umps were calling pitches with 100% accuracy, then Strike% should equal Zone% plus (O-Swing% * (100-Zone%)). I.e., if Zone%=50 and O-Swing%=30, Strike% should equal 65. If it's more than that, umps are expanding the strike zone. If it's less than that, they're squeezing pitchers.

I tried to assess that by working backward from a known Strike%, Zone% and O-Swing%. But obviously if the Zone% number is unreliable, that may not yield a useful result.
....

I don't get this. Of course Strike% already includes strikes outside the zone. That's the whole point of subtracting Zone% from it--to isolate those out-of-zone strikes. Then you look at the difference between out-of-zone strikes and out-of-zone swings to see how many out-of-zone pitches are being *called* strikes. Why doesn't this work? (Again, aside from the Zone% reliability issue.)

I think you're right on what Strike% is, but not what Zone% is. Zone% is the amount of pitches a batter sees thrown in a particular strike zone. For PitchFX it is standardized, and nobody really knows what the zone is for BIS. They are both measuring strikes, albeit a bit differently. The Retrosheet data at BBref is measuring outcomes, not location. So if an ump calls a ball a strike, then the Strike% goes up, even though Zone% would (hopefully) decline. As far as I know, Zone% doesn't take into account the ump's call in PitchFX, and only sometimes in BIS. The two measurements are not recording strikes the same way, which makes using them interchangeably tricky, since there are different standards being used. I've emailed Fangraphs about this to see if they know how umpire neutral the BIS and PitchFX data sets are. I'm guessing PitchFX is umpire neutral, but BIS is still unclear. And it would make sense to use an umpire neutral approach to determine how disciplined a hitter is. Just because an ump screws over somebody like JD Drew, doesn't mean that Drew's superior strike recognition skills are bad. It would be unfair to Drew to penalize him for times when a ball is called a strike and he watched it go by.

[ScubaSteveAvery]

Zenter and PaulM, I think you are both on to something. When did the FO start discussing out loud that they were going to focus on defensive market inefficiencies? Although, it raises the question, why abandon something that has proven to work to search for a new market inefficiency? Wouldn't it make sense to try and blend the two? Find Relatively good OBP who have above average fielding metrics? There might be a scarcity issue, but the steady decline in OBP is just weird for a team that was so outspokenly in favor it.

[sachilles]

I don't see where they've abandoned it. This year had some special circumstances that caused a more drastic drop. However, it appears everyone in the AL east dropped substantially over the last few years. Execution may be an issue, but I don't see a change in philosophy.

[zenter]

... Although, it raises the question, why abandon something that has proven to work to search for a new market inefficiency? Wouldn't it make sense to try and blend the two? Find Relatively good OBP who have above average fielding metrics? There might be a scarcity issue, but the steady decline in OBP is just weird for a team that was so outspokenly in favor it.

I, like sachilles, would suspect that this wasn't "abandonment" of OBP any more than putting priority on OBP was abandonment of valuing fundamentals. In essence, if something is in high demand, its value would be higher. If you identify a difference-making aspect that's in low demand, you can add that to your equation. I mean, despite his below-career-average performance this year, Adrian is definitely a hybrid of OBP/OPS+ and defense.

Now it may be that Cherrington disagreed with some of the execution (Crawford), but not the philosophy. There's also a big messy hole in terms of what the Sox did for OBP before the trade (before Sept 1: .325) and after (.267). This team led the AL in RS up to September 1, too. Between and among the loss of Adrian and the Ortiz injury and the lack of Ellsubury resurrgence, there was ample unique personnel reasons to chalk up the hugeness of the dropoff to a bad end of year.

While OBP is of some concern, the lack of pitching quality all year is why the RS stat didn't translate to wins.

[Paul M]

What's also apparent is outside of Ortiz not one player with more than 300 ABs posted an OBP above .350. Including Nava, only those two have walks/AB > 10%. So, even in the first half Ortiz had a very big impact on the team OBP and the line-up dynamics changed. Ortiz had more walks that strikeouts. The rest of the team had 3 times Ks to walks. Gotta pay the man.

[ScubaSteveAvery]

I, like sachilles, would suspect that this wasn't "abandonment" of OBP any more than putting priority on OBP was abandonment of valuing fundamentals. In essence, if something is in high demand, its value would be higher. If you identify a difference-making aspect that's in low demand, you can add that to your equation. I mean, despite his below-career-average performance this year, Adrian is definitely a hybrid of OBP/OPS+ and defense.

Now it may be that Cherrington disagreed with some of the execution (Crawford), but not the philosophy. There's also a big messy hole in terms of what the Sox did for OBP before the trade (before Sept 1: .325) and after (.267). This team led the AL in RS up to September 1, too. Between and among the loss of Adrian and the Ortiz injury and the lack of Ellsubury resurrgence, there was ample unique personnel reasons to chalk up the hugeness of the dropoff to a bad end of year.

While OBP is of some concern, the lack of pitching quality all year is why the RS stat didn't translate to wins.

I'd agree that abandoned is too strong. But I don't agree that 2012 was a one-off event. Like I showed earlier, 2012 continued a trend of decline starting from the peak in 2007. I do think that the amount of decline is probably an outlier, based on the September. But, like JMDurron in post #18, the pre-Punto trade batting average would have been the lowest since the ownership took over, and continues the downward trend. Adrian fit the model of hybrid player, but we have no clue what happened that caused his ability to walk to plummet significantly.

I think the downward trend is attributable to a multitude of factors, and I hope that is what we try to figure out. The magnitude of the problem may not be as big or important as figuring out the pitching issues, but I think its worth figuring out. Its possible that a change in what the what the front office values could help explain the decline. Its not the whole picture, but the factors are worth exploring, yeah? I'm guessing natural regression of aging players is another factor. As are injuries, and the lack of depth on bench to handle injuries to major players. As Smas mentioned earlier, trading Scutaro and then plugging in Aviles dropped OBP, as was trading Youkilis and replacing him with Middlebrooks. Middlebrooks was great, but contributed to a drop in OBP. There are many factors, some good and some bad, some the FO has control of, and some that don't. But its worth looking into each one to figure out why. We may never get the causation, but the point of this thread is to figure out why.

[Paul M]

The reailty is guys like Bill Mueller, Kevin Millar, Jeremy Giambi, David Ortiz, and Mark Bellhorn are more expensive today but at the same time none of these were known for their gloves (it was all about the bats and they fit their mold) and all were acquired in the first two years of the Theo era. What's funny is they traded Nomar for clubhouse/glove reasons and acquired Pokey, Minky, and Roberts for the types of skills they presumably grew to value later, but all along they had some balance with places for glove-heavy players.

Stating the obvious, but they had a damn near perfect record of adding talent and all of these guys came for relatively short deals and pretty reasonable contracts. The 2004 team had a K/BB ratio of less than 2; 2012 over 3. Yes, macro trends in the game explain part of this but they just simply would not play guys like Aviles or even Middlebrooks. They dealt Shea Hillenbrand afterall to address a need and to remove a clubhouse issue--did they deal with those quicker then?--but I think Shea's approach was universally loathed. I guess I/we should come to grips with the fact it may be another generation before we see another offense like that group. And I think the run of success in bringing in talent may also be a once in a 25-year period. But they don't need to be that proficient to field a winner.

[ScubaSteveAvery]

Good post Paul. I've put together a sheet looking at normalized OBP and BB% per position for the Sox from 2002-2012, and then I'll put together a list of who held what position that year, how many plate appearances they appeared in, and their OBP and BB% from those individual seasons. I'm hoping that will clarify the picture of who replaced who, and what impact, if any, that had the Sox' OBP and BB% numbers. This also gives us the ability to analyze on macro levels the FO's strategy and success in player evaluation, as well as attempting to explain the impact particular injuries had. It also might help explain why the Sox are underperforming macro trends over the last few years. You're probably right that the old scrap heap pile is now the treasure pile and those players are no longer available cheaply. I'm interested to see how the data looks when compared to the individual player level.

Edited by ScubaSteveAvery, 12 October 2012 - 11:38 AM.

[ScubaSteveAvery]

So this table is huge, and all this positional stuff may have to go in another thread. Sorry to those reading on mobile. But here is a breakdown of Red Sox catchers from 2002-2012. I ordered the players based on plate appearances they received in a year. The minimum number of plate appearances to qualify for the table was 20 per year. I figured that 20 PAs is roughly 5 games of starting, or 10-20 of pinch hitting (sorry, no Corky Miller sightings here). The sample is obviously tiny for those with around 20 PAs and the stats subject to SSS bias, but it gives you an idea of the depth the Sox had that year.

The purpose of the chart is both to show the evolution of the position over the past 10 years, and the depth the Red Sox had when it came to normal backups, or those brought on because of injury. Lots of different conclusions or points can be drawn from it, so hopefully it won't be too intimidating so we can get good discussion. I have the information ready in a spreadsheet to do one of these for every position, so if there is a position anyone wants to see next, then please let me know. Also, if you think gridlines are necessary, I can do that too.

Anyways, here goes nothing...

Red Sox Catchers OBP & BB% 2002-2012

Year	Metrics
2002		Jason Varitek	Doug Mirabelli
	PA	519	173
	OBP	0.332	0.312
	BB%	7.90%	9.80%
2003		Jason Varitek	Doug Mirabelli
	PA	521	176
	OBP	0.351	0.307
	BB%	9.80%	6.30%
2004		Jason Varitek	Doug Mirabelli
	PA	536	182
	OBP	0.390	0.368
	BB%	11.60%	10.40%
2005		Jason Varitek	Doug Mirabelli
	PA	539	152
	OBP	0.366	0.309
	BB%	11.50%	9.20%
2006		Jason Varitek	Doug Mirabelli	Javy Lopez	Josh Bard
	PA	416	176	65	21
	OBP	0.325	0.261	0.215	0.381
	BB%	11.10%	6.30%	3.10%	14.30%
2007		Jason Varitek	Doug Mirabelli	Kevin Cash
	PA	518	127	33
	OBP	0.367	0.278	0.242
	BB%	13.70%	8.70%	12.10%
2008		Jason Varitek	Kevin Cash
	PA	483	162
	OBP	0.313	0.309
	BB%	10.80%	11.10%
2009		Jason Varitek	Victor Martinez	George Kottaras
	PA	425	237	107
	OBP	0.313	0.405	0.308
	BB%	12.70%	10.10%	10.30%
2010		Victor Martinez	Jason Varitek	Kevin Cash	J. Saltalamacchia
	PA	538	123	68	25
	OBP	0.351	0.293	0.224	0.360
	BB%	7.40%	8.10%	8.80%	24.00%
2011		J. Saltalamacchia	Jason Varitek	Ryan Lavarnway
	PA	386	250	43
	OBP	0.288	0.300	0.302
	BB%	6.20%	8.40%	9.30%
2012		J. Saltalamacchia	Ryan Lavarnway	Kelly Shoppach
	PA	448	166	158
	OBP	0.288	0.211	0.327
	BB%	8.50%	6.60%	7.00%

[ScubaSteveAvery]

Also, these two graphs are to accompany the table above. They are measuring the normalized walk rate of the Red Sox catchers from 2002-2012. I normalized the data to the AL BB%s because I want to compare the DH position too and wanted the normalized data to be consistent across the board.

The first is all catchers per year:



The second splits out the starters and the backups. The catcher with the post plate appearances during the year I designated as the starter, and all the rest were backups (this might be problematic for 2009 since Victor came on at the trade deadline and I think served as the primary starter the rest of the season). This one gives you a good idea at how good, or bad, the catching depth was at drawing walks. I figured BB% would be more fair than OBP, since if OBP includes batting average, so naturally the bench players would have depressed OBPs,or skewed ones due to SSS concerns.

[Vermonter At Large]

First of all, there's a lot of noise in OBP and no real need to normalize walk rates. In fact the Sox maintained a very consistent walk rate of between .102-.107 BB/PA from 2004-2009. There was a decline to .092 and .090 in 2010-11, then a steep drop off to .069 this year. I would classify the .090-.099 range as very good, the .100+ as excellent, and the .069 as below average.

I think the bigger story in 2012 was the poor quality of contact that was reflected in nearly every metric available, including a .046 drop in SLG to an anemic .415 this year. Even with a higher walk rate, this team would have had a difficult time converting those walks into extra runs.

The offense was just terrible except for a few hot stretches in the middle of the season.

[Paul M]

Slugging ranks by month:

April: 2nd
May: 4th
June: 11th
July: 18th
August: 11th
September: 30th

Clearly one month stands alone.

Just to complete the thought same ranks by OBP:

April: 7th
May: 12th
June: 11th
July: 20th
August: 11th
September: 30th (.270 which is hard to even fathom to be honest)

Based on this--I know the park played smaller, but still--their OBP skill was not close to what we have seen in the last 10 years. Outside the top 10 in all but 1 month.

[OttoC]

Almost 10% of the walks in the AL in this century were intentional and perhaps should be taken out of the "equation."

[Sprowl]

Salty gets further and further away from his high-BB years.

If one wants to be optimistic (and I do, for lack of a better attitude), it's worth observing that Salty's BB% is likely to rise into the low teens, based on his minor league years.

Otherwise, he's close to what the early years predicted: feast or famine. I think he suffered in 2012 from early bad luck on line drives. Part of that was true luck, so I expect a small uptick next year. The other part was the shift, which is more like earned bad luck. Is there a word for anti-karma?

Salty's hitting is good enough to keep for a catcher if his catching is good enough to keep on defense.

[ScubaSteveAvery]

First of all, there's a lot of noise in OBP and no real need to normalize walk rates. In fact the Sox maintained a very consistent walk rate of between .102-.107 BB/PA from 2004-2009. There was a decline to .092 and .090 in 2010-11, then a steep drop off to .069 this year. I would classify the .090-.099 range as very good, the .100+ as excellent, and the .069 as below average.

There may be noise in OBP, but that is why it is important to look at both OBP and BB% together. There are different factors to look at when comparing. One is the ability to not make an out and the other is the ability to walk. They don't always move together (OBP drops in 2008 and 2009, but BB% remain steady). Also, the question is not whether they are still good at getting on base, but whether they are declining in a way that would indicate something larger at play that simply macro league trends or random variance. This year's amount of decline is clearly an outlier, fueled by a horrible September, but previous to that, the team's on base skills would have continued a 5 year downward trend. I'm curious why BB% shouldn't be normalized? I think it is necessary to show how the team is walking relative to the league, which provides some context. For instance, if the Sox BB% stayed flat, but the league was collectively getting better, then the Sox lose ground. What worries me is that the Sox are no longer an elite on base team. And if the pitching staff is going to be awful, then the team needs to score more runs and make less outs.

I think the bigger story in 2012 was the poor quality of contact that was reflected in nearly every metric available, including a .046 drop in SLG to an anemic .415 this year. Even with a higher walk rate, this team would have had a difficult time converting those walks into extra runs.

The offense was just terrible except for a few hot stretches in the middle of the season.

The team still hit a ton of doubles this year, so its not like they turned into a singles hitting club. Slugging has showed no downward trends for the Sox at all in the period I'm looking at, but on base skills have. Its more likely that the slugging this year is a one-off event than on base skills, which continued a decline.

Edited by ScubaSteveAvery, 13 October 2012 - 12:10 PM.

[ScubaSteveAvery]

Here is the position chart for first base. I've added some extras, like how many games started. I'm going to have to re-do the catcher chart I think since PAs per position might not be totally correct, since Mirabelli played a few innings at first, as did Victor Martinez. But it should be close. I also realized that doing the normalized BB% chart for starters and backup players wasn't quite fair, since they were both normalized to AL average for all players. It would be more instructive to normalize them against starters and backups. But I don't have time to do that for all teams in the AL.

Red Sox 1B OBP & BB% 2002-2012

Year	Metrics
2002		Tony Clark (70 GS)	Brian Daubach (54 GS)	Jose Offerman (36 GS)
	PA	288	202	189
	OBP	0.269	0.342	0.330
	BB%	6.90%	8.40%	11.60%
2003		Kevin Millar (96 GS)	David Ortiz (44 GS)	Shea Hillenbrand (17 GS)
	PA	420	185	76
	OBP	0.343	0.346	0.395
	BB%	9.00%	12.40%	3.90%
2004		Kevin Millar (66 GS)	David Ortiz (31 GS)	Doug Mientkiewicz (26 GS)	Dave McCarty (25 GS)	Brian Daubach (13 GS)
	PA	271	143	114	115	53
	OBP	0.492	0.336	0.298	0.304	0.302
	BB%	8.90%	9.10%	8.80%	10.40%	7.50%
2005		Kevin Millar (102 GS)	John Olerud (38 GS)	David Ortiz (10 GS)	Roberto Petagine (7 GS)
	PA	409	178	46	26
	OBP	0.350	0.348	0.370	0.423
	BB%	10.80%	7.90%	6.50%	11.50%
2006		Kevin Youkilis (117 GS)	Eric Hinske (11 GS)	David Ortiz (10 GS)	Mark Loretta (9 GS)	Carlos Pena (8 GS)	J.T. Snowe (7 GS)
	PA	554	47	65	39	32	41
	OBP	0.372	0.340	0.419	0.179	0.375	0.317
	BB%	12.40%	10.60%	9.30%	5.10%	12.50%	14.60%
2007		Kevin Youkilis (124 GS)	Eric Hinske (29 GS)	David Ortiz (7 GS)
	PA	568	120	29
	OBP	0.391	0.292	0.345
	BB%	12.50%	11.70%	17.20%
2008		Kevin Youkilis (110 GS)	Sean Casey (40 GS)	Jeff Bailey (7 GS)
	PA	482	168	31
	OBP	0.384	0.387	0.452
	BB%	9.50%	8.30%	12.90%
2009		Kevin Youkilis (77 GS)	Victor Martinez (22 GS)	Casey Kotchman (19 GS)	Jeff Bailey (19 GS)	Mark Kotsay (12 GS)	David Ortiz (6 GS)
	PA	333	93	84	82	54	21
	OBP	0.441	0.387	0.310	0.341	0.241	0.381
	BB%	14.40%	7.50%	7.10%	12.20%	3.70%	9.50%
2010		Kevin Youkilis (97 GS)	Mike Lowell (38 GS)	Victor Martinez (12 GS)	Lars Anderson (11 GS)
	PA	426	152	55	37
	OBP	0.411	0.303	0.364	0.326
	BB%	13.40%	8.60%	7.30%	13.50%
2011		Adrian Gonzalez (155 GS)	Kevin Youkilis (5 GS)
	PA	699	21
	OBP	0.409	0.286
	BB%	10.40%	4.80%
2012		Adrian Gonzalez (103 GS)	James Loney (26 GS)	Mauro Gomez (13 GS)	Kevin Youkilis (12 GS)	David Ortiz (7 GS)
	PA	455	104	58	44	30
	OBP	0.354	0.269	0.293	0.364	0.400
	BB%	5.90%	4.80%	6.90%	9.10%	13.30%

[ScubaSteveAvery]

Also, as a companion chart to the table above, here is a chart of normalized OBP and BB% for Red Sox first baseman. Like the catchers, I normalized the data to AL only, so all the data is consistent when comparing DHs.



I also did one for catchers too:

[Vermonter At Large]

1. The walk rate isn't in a 5-year steady decline - your graph makes it seem as though it is, but it isn't. BB% over 10% is excellent, and the fluctuation from 2004-2009 was negligible - more than covered by statistical error.. There was a slight dropoff in 2010-11, then a big drop off this season. The dropoff in OBP from 2011-2012 was less than the drop of of SLG.

2. There is no need to normalize BB% because there are no mitigating factors that make it necessary. Walks have nothing to do with era or ballpark dimensions or climatalogical factors, league talent dilution etc., As Cliff mentioned earlier, one can certainly remove IBB's from the equation if one wanted to, and they could be weighted similarly based on game situation, batting order position, pitcher BB propensity etc, but that would require grainer adjustments than adjusting to league averages.

3. For all the talk early in the season of record pace doubles rates, in the end it was a fairly ordinary Red Sox 2B rate (.055 per PA, same as 2011 and pretty much the same since 2007. Park factor has a lot to do with Red Sox high doubles rates and there is no reason to believe the 2012 Sox lineup was especially proficient as a power unit.

I am glad this thread is here, and I think there is much of value to be gleaned here. I hope that there will be others that look at the decline of offense - perhaps in different aspects. 2012 was a very difficult season to look at in many ways, given the personnel availability at different parts of the season and what appeared to be a very streaky run accumulation pattern. I think that for most of the season the pitching was so god awful that we tended to overestimate the suckiness of the offense. The overall decline in offense throughout MLB also makes it very difficult, since it is very difficult to compare contact events from the past season to previous norms.

[ScubaSteveAvery]

1. The walk rate isn't in a 5-year steady decline - your graph makes it seem as though it is, but it isn't. BB% over 10% is excellent, and the fluctuation from 2004-2009 was negligible - more than covered by statistical error.. There was a slight dropoff in 2010-11, then a big drop off this season. The dropoff in OBP from 2011-2012 was less than the drop of of SLG.

The slight drop off was 1.2% from the year prior and .9% from the average of the previous 5 years where they were over 10%. More importantly, that drop was sustained. And even if the amount of the drop off this year was an outlier, they Sox would have continued their BB% decline into this year as well. There was only 1 month were their walk rate was above 8%, and that was March and April. After that the next two months were at 7.4% and 7.9%, respectively, before dropping off into the 6% range and below. So even before the trade and bad September, this team was continuing to walk at a decreasing rate.

But, as an aside, I think you're right that declining from 10.7% to 10.1% is not a big deal because those are both really good, but declining from a peak of 10.7% to 9% is more troubling. While still above league average, it shows that something has happened. That something is what I'm trying to investigate.

2. There is no need to normalize BB% because there are no mitigating factors that make it necessary. Walks have nothing to do with era or ballpark dimensions or climatalogical factors, league talent dilution etc., As Cliff mentioned earlier, one can certainly remove IBB's from the equation if one wanted to, and they could be weighted similarly based on game situation, batting order position, pitcher BB propensity etc, but that would require grainer adjustments than adjusting to league averages.

I normalized the data to look at the differences between the Red Sox and league average. There are many reasons to normalize the data, and I agree that walking has nothing to do with park factors or climatological factors. I think era is a different story as walk rates were consistently higher from about 1935-1960, before dropping, and then rising again. But one useful way of comparing the Red Sox to the rest of the AL is to normalize the data. Like I said before, if the Sox are walking at the same rate, and the league is getting better, then the Sox are losing ground against their opponents. Also, the normalized graphs show a visual representation of this. If the line is sloped upwards, then the Sox are doing better than the league, if it is flat then they are doing the same as the league, and if it is declining, then they are doing worse. Its a nice way at looking a trends.

However, I have gotten feedback on this from many people. And the consensus is that we're both right . Normalizing the data is not necessary for BB% because there is not a lot of variance from year to year; but it provides value when trying to evaluate performance against peers. From now on, I'll post both the raw BB% and the normalized BB%.

And I think for BB% it matters because the normalized OBP show a 5 year decline, whereas the normalized BB% show a different decline:





Because the AL, starting in 2005, had rising BB%, the Sox BB% showed a slight decline. The normalized BB% show a decline that happens in 'shelves,' with the more dramatic declines happening in between 2009-2010 and onward. There is that uptick in 2011 because although the Sox showed a negligible decline from 9.2% to 9.0%, the league saw a drop of 8.5% to 8.0%.

All this should be with context too. From when the ownership took over to 2007, the Sox enjoyed an increase in BB% followed by two years at that peak of over 10%, and also saw a rise on OBP from 2002-2007. In 2008 and 2009 the Sox saw stabilized on base production, albeit at slightly lower levels, but still elite. And from 2010-2012, we've seen declining on base production from elite to good, and then to awful this year. And like I said earlier, I think 2012 is an outlier in terms of how bad it was, but even without the big drop off in September, this team would have added to the decline.

It was the winter after the 2009 season where the Sox started to vocalize shifting from on base production to preventing runs via improved defense. If you look at the data for 2010-2012, you'll see on base skills have likewise declined from 2010 forward.

3. For all the talk early in the season of record pace doubles rates, in the end it was a fairly ordinary Red Sox 2B rate (.055 per PA, same as 2011 and pretty much the same since 2007. Park factor has a lot to do with Red Sox high doubles rates and there is no reason to believe the 2012 Sox lineup was especially proficient as a power unit.

True, the 2011 team was a historically good slugging team, but 2007 was one of the lowest of the ownership's tenure (but still good by historical standards). The difference in SLG this year is due to a drop in home runs and a historically low number of triples hit. And if you exclude September, the slugging average was the same as 2006 (.435), which isn't great, but not an extreme outlier that turned out to be with September included. The low slugging this year definitely is a factor in the poor offensive production this year, but over the long term, it was actually improving from 2006-2011. To me, its a concern, but not in the same way as on base skills.

[Vermonter At Large]

True, the 2011 team was a historically good slugging team, but 2007 was one of the lowest of the ownership's tenure (but still good by historical standards). The difference in SLG this year is due to a drop in home runs and a historically low number of triples hit. And if you exclude September, the slugging average was the same as 2006 (.435), which isn't great, but not an extreme outlier that turned out to be with September included. The low slugging this year definitely is a factor in the poor offensive production this year, but over the long term, it was actually improving from 2006-2011. To me, its a concern, but not in the same way as on base skills.

Looking back at that 2007 season is interesting ... that team's contact rates were almost identical to the 2012 season, but the 2007 team scored 133 more runs (and no, it wasn't just because that team walked 261 more times). If one could unlock the reason that 2007 team was so much more successful than this team we might have an idea of where this team needs to go. What you're doing might provide some answers to that question.

[ScubaSteveAvery]

So before I do 2B, I figured I would do 3B, since how the Sox handled 1B and 3B with regards to Youkilis is important. In some ways, looking back, one could argue that the decision to not sign Beltre, but move Youkilis over to 3B again, and then trade for Gonzalez actually hurt the Sox long term. I was a fan of the Gonzalez trade at the time, but looking back, one can see how managing that third base transition better would have benefited the Sox more, especially when Youkilis was throwing up insane walk rates and high OBPs at first base, but proved to no longer have the body to play third full time.

Also, I question Will Middlebrooks as a long term option at third base, considering his awful walk rate. If he can get it around the 7% range like he did in that 24 game sample of AAA, then he can maybe stick, but he's shown no other ability to walk on a consistent basis. If Bogaerts develops well next year, then Middlebrooks could probably be made a trade chip.

Red Sox 3B OBP & BB% 2002-2012

Year	Metrics
2002		Shea Hillenbrand (154 GS)	Lou Merloni (6 GS)
	PA	675	25
	OBP	0.329	0.360
	BB%	3.70%	0.00%
2003		Bill Mueller (124 GS)	Shea Hillenbrad (28 GS)	Lou Merloni (4 GS)
	PA	548	117	20
	OBP	0.393	0.291	0.150
	BB%	10.60%	3.40%	5.00%
2004		Bill Mueller (94 GS)	Kevin Youkilis (54 GS)	Mark Bellhorn (13 GS)
	PA	403	238	57
	OBP	0.345	0.370	0.411
	BB%	10.90%	12.60%	19.30%
2005		Bill Mueller (140 GS)	Kevin Youkilis (14 GS)	Ramon Vazquez (6 GS)
	PA	563	65	25
	OBP	0.369	0.431	0.261
	BB%	9.80%	13.80%	4.00%
2006		Mike Lowell (148 GS)	Kevin Youkilis (10 GS)	Alex Cora (4 GS)
	PA	629	54	21
	OBP	0.337	0.426	0.250
	BB%	7.50%	11.10%	0.00%
2007		Mike Lowell (150 GS)	Kevin Youkilis (12 GS)
	PA	650	53
	OBP	0.378	0.396
	BB%	8.20%	11.30%
2008		Mike Lowell (108 GS)	Kevin Youkilis (32 GS)	Jed Lowrie (22 GS)
	PA	462	135	108
	OBP	0.342	0.415	0.318
	BB%	8.20%	11.80%	11.10%
2009		Mike Lowell (105 GS)	Kevin Youkilis (56 GS)
	PA	445	246
	OBP	0.330	0.378
	BB%	6.30%	11.00%
2010		Adrian Beltre (151 GS)
	PA	641
	OBP	0.365
	BB%	6.20%
2011		Kevin Youkilis (111 GS)	Jed Lowrie (29 GS)	Mike Aviles (6 GS)	Drew Sutton (4 GS)
	PA	480	133	59	20
	OBP	0.375	0.326	0.379	0.450
	BB%	14.00%	6.80%	3.40%	5.00%
2012		Will Middlebrooks (69 GS)	Pedro Ciriaco (35 GS)	Kevin Youkilis (29 GS)	Nick Punto (15 GS)	Danny Valencia (7 GS)	Mauro Gomez (6 GS)
	PA	279	136	120	70	28	24
	OBP	0.326	0.293	0.336	0.271	0.143	0.375
	BB%	4.30%	3.70%	7.50%	12.90%	0%	0.00%

[ScubaSteveAvery]

Here are the charts for 3B

[ScubaSteveAvery]

I'll hold off on 2B for now because like 3B, shortstop is a really interesting position for the current ownership, and a position which shows the front office's commitment toward on base skilled players. There were many reasons for getting rid of Nomar, but I'm sure that his approach to the plate was one of them. He was consistently below average in BB%, despite putting up respectable OBP, which was largely driven by higher averages. After obtaining Orlando Cabrera, who proved valuable down the stretch, they decided not to bring him back to take a stab at Edgar Renteria who was a superior player in his prime. After he proved to be a mess, they decided to play defensive whiz, Alex Gonzalez, until signing Lugo to his ill-fated contract. Outside of 2007, Lugo actually wasn't a bad player when it came to getting on base his remaining two seasons. After Lugo, the Sox moved on to Marco Scutaro, who historically was an above average short stop when it came to walking and getting on base.

Lugo's problem was staying healthy in 2008 and 2009. Luckily in 2008, Jed Lowrie and Alex Cora both had career best years in terms of getting on base. But in 2009, we were all subject to the Nick Green experience, as well as the return of Sea Bass, and a much lesser Jed Lowrie. It was only in 2012, when the Sox traded Scutaro and replaced him with Mike Aviles and the pu pu platter of Jose Iglesias, Pedro Ciriaco, and Nick Punto, did the front office lack a shortstop that had on-base skills.

One thing I found interesting, was the front office obtained players that were historically above average at getting on base, but they all never played up to previous performance. Renteria, Lugo, and Scutaro all played below average once coming to the Sox. Lugo had an awesome 79 game stretch where he started and played great, but then he was injured and returned to playing below his potential. Scutaro, while effective in his two years here, never found that 9% walk rate again, although in 2011, his OBP was great, boosted by a red hot month of September (1.019 OPS!). It is hard to say what really went wrong, but the Red Sox troubles at SS are not for want of trying.

Red Sox SS OBP and BB% 2002-2012

Year	Metrics
2002		Nomar Garciaparra (153 GS)	Rey Sanchez (7 GS)
	PA	690	27
	OBP	0.354	0.333
	BB%	5.90%	0.00%
2003		Nomar Garciaparra (155 GS)	Damian Jackson (6 GS)
	PA	718	31
	OBP	0.346	0.355
	BB%	5.40%	6.50%
2004		Orlando Cabrera (58 GS)	Pokey Reese (56 GS)	Nomar Garciaparra (37 GS)	Cesar Crespo (7 GS)
	PA	248	207	163	34
	OBP	0.320	0.279	0.350	0.176
	BB%	4.40%	5.30%	4.30%	0%
2005		Edgar Renteria (150 GS)	Ramon Vazquez (6 GS)
	PA	689	28
	OBP	0.334	0.286
	BB%	8.00%	7.10%
2006		Alex Gonzalez (110 GS)	Alex Cora (47 GS)
	PA	429	189
	OBP	0.299	0.326
	BB%	5.10%	8.50%
2007		Julio Lugo (139 GS)	Alex Cora (22 GS)
	PA	625	89
	OBP	0.295	0.253
	BB%	7.70%	2.20%
2008		Julio Lugo (79 GS)	Jed Lowrie (45 GS)	Alex Cora (38 GS)
	PA	306	187	162
	OBP	0.356	0.360	0.360
	BB%	11.10%	12.30%	8.00%
2009		Nick Green (74 GS)	Alex Gonzalez (43 GS)	Julio Lugo (27 GS)	Jed Lowrie (18 GS)
	PA	272	159	114	70
	OBP	0.296	0.316	0.333	0.200
	BB%	6.30%	3.10%	10.50%	8.60%
2010		Marco Scutaro (131 GS)	Jed Lowrie (21 GS)	Yamaico Navarro (6 GS)
	PA	619	83	28
	OBP	0.337	0.422	0.107
	BB%	7.70%	13.30%	3.60%
2011		Marco Scutaro (102 GS)	Jed Lowrie (47 GS)	Mike Aviles (6 GS)
	PA	435	187	25
	OBP	0.360	0.278	0.280
	BB%	8.50%	5.30%	0.00%
2012		Mike Aviles (123 GS)	Jose Iglesias (23 GS)	Pedro Ciriaco (11 GS)	Nick Punto (5 GS)
	PA	517	76	40	20
	OBP	0.280	0.203	0.400	0.250
	BB%	4.30%	5.30%	2.50%	5.00%

[ScubaSteveAvery]

[ScubaSteveAvery]

Second base has been a fairly cut and dry position for the current ownership. They were lucky at times (Mark Bellhorn), unlucky at others (Mark Loretta), and scored with Pedroia as a long term option with elite on base skills. One of Theo's first moves was to trade for Todd Walker. In the three years prior to Walker joining the Sox, he had an OBP above .350, and had a BB% of 9.5%, 8.4%, and 7.4% respectively. He never recovered his previous OBP, or over 8% BB% that he had consistently put up on his whole career. If I remember correctly, he was allowed to be a free agent because his defense was questionable and his average performance at the plate.

The Sox were really lucky that Mark Bellhorn, who started the season as the utility guy, stepped up due to injuries to Nomar and Pokey and put up one of his career best seasons. Although 45% of his plate appearances in 2004 ended in a walk, home run, or strikeout, he put up elite walk numbers that year. He regressed back to utility player level in 2005, yet still put elite walk numbers until he was DFA'd after a thumb injury. Theo traded for Graffanino, who tagged teamed with Cora to fill in for the rest of the year. On Theo's Pearl Jam Tour, the GM quartet traded Mirabelli for Loretta, who was coming off an injury shortened season. Historically, Loretta had elite on-base skills, who before he was injured in 2005 had a 9.7% BB% and .360 OBP. He struggled all year to return to his previous elite on-base skills, but was still a serviceable one year stop gap. A .355 OBP is still is really good, even if it was more average driven.

Pedroia's BB% jumped from 8.1% and 6.9% in his first two years as a starter into the elite 10% range starting in 2009 until this year, where he posted sub par walk numbers for him. Its unclear if this was an off year, or if the drop off in performance was due to the nagging thumb injuries. That might explain the OBP, but it could be that his walk rate was lower because he was pressing to get hits and get the team in the game (this seemed to be a team wide problem this year). Hopefully next year he will return to his usual self, as he's one of the best second baseman in the game.

Red Sox 2B OBP & BB% 2002-2012

Year	Metrics
2002		Rey Sanchez (97 GS)	Lou Merloni (47 GS)	Carlos Baerga (10 GS)
	PA	357	186	43
	OBP	0.315	0.337	0.372
	BB%	4.80%	10.80%	7.00%
2003		Todd Walker (134 GS)	Damian Jackson (14 GS)	Bill Mueller (10 GS)
	PA	630	54	44
	OBP	0.322	0.321	0.477
	BB%	7.10%	3.70%	2.30%
2004		Mark Bellhorn (118 GS)	Pokey Reese (18 GS)	Bill Mueller (14 GS)	Cesar Crespo (6 GS)	Ricky Gutierrez (5 GS)
	PA	557	61	56	28	26
	OBP	0.372	0.246	0.500	0.179	0.308
	BB%	13.80%	9.80%	12.50%	0%	3.80%
2005		Mark Bellhorn (82 GS)	Tony Graffanino (50 GS)	Alex Cora (31 GS)	Bill Mueller (5 GS)
	PA	330	200	143	22
	OBP	0.327	0.355	0.270	0.409
	BB%	14.80%	4.50%	4%	18.20%
2006		Mark Loretta (133 GS)	Dustin Pedroia (19 GS)	Alex Cora (10 GS)
	PA	631	78	45
	OBP	0.355	0.234	0.289
	BB%	6.70%	7.70%	6.70%
2007		Dustin Pedroia (132 GS)	Alex Cora (30 GS)
	PA	578	130
	OBP	0.380	0.333
	BB%	8.10%	3.10%
2008		Dustin Pedroia (150 GS)
	PA	724
	OBP	0.374
	BB%	6.90%
2009		Dustin Pedroia (154 GS)	Nick Green (6 GS)
	PA	714	26
	OBP	0.371	0.346
	BB%	10.40%	3.80%
2010		Dustin Pedroia (75 GS)	Bill Hall (38 GS)	Jed Lowrie (24 GS)	Marco Scutaro (16 GS)	Eric Patterson (5 GS)
	PA	351	157	98	73	33
	OBP	0.367	0.287	0.337	0.315	0.303
	BB%	10.50%	7.00%	11.20%	6.80%	9.00%
2011		Dustin Pedroia (158 GS)
	PA	730
	OBP	0.387
	BB%	11.80%
2012		Dustin Pedroia (139 GS)	Pedro Ciriaco (11 GS)	Nick Punto (10 GS)
	PA	615	47	72
	OBP	0.345	0.304	0.366
	BB%	7.80%	2.10%	12.50%

[ScubaSteveAvery]

Both graphs confirm that the team had always above average OBP, but the walk rates suffered during the Loretta year, and then was down in 2007 (but mostly because Cora's awful 3% walk rate brought down the team average below AL average. From 2009 on, the team has been above average, driven by Pedroia.





Edit: edited title of second graph

Edited by ScubaSteveAvery, 17 October 2012 - 08:09 PM.