Well, the record (since 1956) belongs to none other than the Yankees, who went 6 games without an XBH in 1967. Amazingly, they logged 43 singles over that span (more than 7 per game). But they still only batted at a .197 clip. Two other teams (1964 Angels, 1985 Pirates) had streaks of 5 games.
The full list is
here. Again, these are full games without an XBH, so the Sox aren't in the group of teams with 4-game futility streaks. They are, however, among dozens of teams with 3-game streaks.