A couple of things.
First, in an effort to be self congratulatory, this thread is fucking awesome. Thank you everyone including Heyman who has proven himself to be a below replacement level baseball writer. There is no "you had to be there" when evaluating his work. Its lazy, intellectually dishonest and a its also a scathing indictment of Sports Illustrated for giving him a platform to publish what effectively amounts to garbage. On the flip side, this thread goes a long way toward framing how people should look at a baseball career.
That said, ivanvamp and others deserve credit here for pointing out that observation can enhance or detract from an otherwise objective player's performance. The crux of the argument for most non-SABR types is that the stat heads simply reduce the game to statistics. HSB and others make a valid point that its simply impossible to observe every event in a player's career and your eyes can indeed fool you.
However, imagine a scenario where you have two players who play the same position during the same era and who are statistically identical in every single way (e.g. games played, PAs, defensive metrics and WAR). They are the same player, yet people may clearly prefer one over the other for a multitude of reasons that are not captured in any valued statistical measures. Perhaps its simply that one gives maximum effort all the time while the other does not. They arrive at the same results, however its entirely conceivable that the former is valued more than the latter.
Now I know some will question how to define what giving maximum effort means but everyday life, let alone sports, is rich with examples of people who are constantly "hustling" versus those that do enough to get by. I imagine most people reading this thread can immediately think of family members, friends or coworkers who fit these descriptions. And maybe those perceptions are incorrect...but then again, maybe they aren't.
The point is that observations, while subject to bias, should have some role in evaluating performance.
Finally, PrestonBroadus Lives deserves a membership if only for an awesome screen-name.
I think one other issue to consider, and yes it's an intangible, is leadership. There's no metric for it, no way to measure it, but only the totally clueless person would deny its reality. Anyone who has served in the military, been involved in sports at any level, been in business...leadership is crucial to the success of a team or organization. Most of us are not around these athletes to know what kind of leadership they bring to a team. Sportswriters tend, due to the nature of their business, to be around them more. It's maybe not enough to give a really fair evaluation, but it's something that should be thrown into the mix.
Consider this scenario. Player X is having a rough stretch and is getting down on himself (it happens). Player Y is a guy who is a real leader. He pulls X aside and says, ok, let's talk about it. He listens, gives some advice, even spends extra time working with player X on a few things. Scouting reports, hitting tips, whatever. Things that, yeah, the coaches should do, but sometimes - as anyone who has ever played sports can tell you - it's received better coming from a peer. Player X then turns it around and attributes it to player Y's help.
Now, there is no stat that covers player Y's contribution there. In fact, the turnaround seems to be totally to the credit of player X. But player X knows that most of the credit should go to player Y. It's a leadership thing and there is no metric for it.
But it is very, very real.
How much should that play into the HOF consideration? Who knows. But put it this way: if I had two players with equivalent stats, and Jones is, by all accounts, a quality leader who goes out of his way to help others on his team, and Smith is, by all accounts, a pretty self-centered "it's all about me" kind of guy, and I had one HOF vote, I'd go with Jones. And it would be very fair to do so.
That said, we fans can't really know who does what. Maybe guys that seem to not be leaders (b/c they're quiet, etc.) really are, and guys that seem to be leaders (b/c they're outspoken) aren't. But sportswriters, as a profession, probably are more in tune with that than the average fan. We kind of have to rely on the anecdotal evidence for that. But that doesn't make it any less real.