[Tangotiger]

It’s the fifth annual fielding survey. I’m hoping for another great turnout this year. Please take a few moments, and participate!
http://www.tangotiger.net/scouting/

{This message was approved last year by Moderator, and not considered spam. Previous version here: http://sonsofsamhorn...showtopic=10953 }

[philly sox fan]

This is really an excellent project and one that is well worth your time.

[Tangotiger]

philly sox fan, on Aug 21 2007, 10:08 PM, said:

This is really an excellent project and one that is well worth your time.

Thanks for the support. I've already reached over 100 submissions, which is a fantastic turnout.

[Tangotiger]

Prelim results. I have 143 ballots and counting. A very boring fielding team if you ask me. None of them close to the best at their positions according to you. Overall average. Manny low as usual (below average, but not horrible).

Wily Mo Pena is once again challenging for the worst fielder in baseball. You also hate Hinske.

[David Laurila]

For the second year running, I enjoyed filling out this ballot. Whatever my opinion is worth.

[Tangotiger]

Results:

http://www.tangotige...esults2007.html

[PedroKsBambino]

Always a great thing to read, and it's terrific that Tango puts it together each year.

I think all Sox fans deserve a pat on the back for giving Wily Mo Pena a well-deserved zero for instincts!

[OttoC]

It certainly is an interesting undertaking but the sample sizes are quite small for the most part. The only two sets of fans who responded in any volume were Red Sox and Mariners. Blue Jays' fans were a distant third. For example, Varitek had an overall (position neutral) rating of 54 based on 220 ballots while the leader at that position (75), Joe Mauer, was named on 16 ballots.

[Tangotiger]

OttoC, on Oct 15 2007, 11:17 PM, said:

It certainly is an interesting undertaking but the sample sizes are quite small for the most part. The only two sets of fans who responded in any volume were Red Sox and Mariners. Blue Jays' fans were a distant third. For example, Varitek had an overall (position neutral) rating of 54 based on 220 ballots while the leader at that position (75), Joe Mauer, was named on 16 ballots.

16 ballots is sufficient. If I were to select 16 random Redsox ballots, and then select 16 other random Redsox ballots, the correlation between the two groups of ballots will be something like r=.90. Those are results that anyone would love to have. Therefore, all I need is 16 ballots for it to be representative of how fans truly feel.

[OttoC]

Tangotiger, on Oct 16 2007, 12:01 PM, said:

16 ballots is sufficient. If I were to select 16 random Redsox ballots, and then select 16 other random Redsox ballots, the correlation between the two groups of ballots will be something like r=.90. Those are results that anyone would love to have. Therefore, all I need is 16 ballots for it to be representative of how fans truly feel.

I may be missing something, here, but if there are only sixteen ballots, isn't it hard to take repeated samples of 16 random ballots? Florida is an extreme example. Their players appear on an average of 6.5 ballots with only one player being on as many as ten ballots.

[Rice4HOF]

OttoC, on Oct 16 2007, 11:46 AM, said:

I may be missing something, here, but if there are only sixteen ballots, isn't it hard to take repeated samples of 16 random ballots? Florida is an extreme example. Their players appear on an average of 6.5 ballots with only one player being on as many as ten ballots.

I think what Tom is saying is using the example of a team with several ballots (i.e the Red Sox with over 200) and if you take any random group of 16 of those 200, you always get essentially the same results. Therefore a sample of 16 is statistically almost as good as a sample of 200.

If you find that difficult to believe, let me give you a simple (albeit extreme) example:
You are given a coin and asked to flip it 200 times. If you get heads 190 times, you might (with very good reason!) begin to suspect it's not a fair coin.
Now if you were asked to flip it only 16 times, and came up with heads 15 times you should also have very good reason to suspect it's not fair.
While with the 200 case example, the odds of a fair coin landing heads 190 times is (skipping the math...) about one in a bajillion, in the second case it's maybe one in 20,000. While technically, flipping it 200 times makes you somewhat more sure it's a rigged coin, flipping it 16 times should be enough to make you pretty confident in your suspicion.

[OttoC]

Rice4HOF, on Oct 16 2007, 03:23 PM, said:

I think what Tom is saying is using the example of a team with several ballots (i.e the Red Sox with over 200) and if you take any random group of 16 of those 200, you always get essentially the same results. Therefore a sample of 16 is statistically almost as good as a sample of 200.

If you find that difficult to believe, let me give you a simple (albeit extreme) example:
You are given a coin and asked to flip it 200 times. If you get heads 190 times, you might (with very good reason!) begin to suspect it's not a fair coin.
Now if you were asked to flip it only 16 times, and came up with heads 15 times you should also have very good reason to suspect it's not fair.
While with the 200 case example, the odds of a fair coin landing heads 190 times is (skipping the math...) about one in a bajillion, in the second case it's maybe one in 20,000. While technically, flipping it 200 times makes you somewhat more sure it's a rigged coin, flipping it 16 times should be enough to make you pretty confident in your suspicion.

Yes, but if you flip a coin "6.5" times and get 5 heads, you probably won't blink.

It has been a long, long time since I've dealt with this part of statistics and maybe my uneasiness stems from insufficient understanding of Tango's methodology or my ignorance of the discipline, but there are some sample-size issues that I can think of.

Derek Jeter. Jeter was placed on more ballots than any other Yankee; indeed, the other 12 Yankees averaged just a little more than half as many ballots (a couple of his teammates were on about 85% of his total). This makes me think that fans of other teams rated him, and if they were Red Sox fans, he undoubtedly received lower marks from them. It wouldn't take to many to reduce his scores by 10%. Maybe that is just quibbling because he may have been overrated by Yankee fans. However, there were entire teams that averaged 6, 10-12 ballots, and the small sample size can throw ratings off. The average fan is going to see Johnny Shortstop rated at 93 and call him the best shortstop even though he was only rated on six ballots.

I still think this exercise is a good idea; I'd just like to see larger sample sizes.

[Tangotiger]

ARod was on 62 ballots and Jeter on 70. We're talking about 8 Yankee ballots with Jeter, without ARod. Also note that I do do quality checks. As an example, any ballot that is entirely all "1" or all "5" are thrown out. I do other quality checks that aren't so straightforward. (I won't reveal the algorithm.) Overall, I throw out about 1% to 2% junk ballots every year. Half of those usually involve Derek Jeter or Barry Bonds. Derek Jeter did have a very low agreement level (0.49), as he does every year.

Increase the sample size isn't necessarily a good thing. Quality is usually preferred to quantity.

The fact of the matter is that 16 random ballots correlates at r=.90 with 16 other random ballots. You need two thousand plate appearances from hitters in one group to correlate at r=.90 with 2,000 PA from the same hitters in another group.

Bias and selective sampling exists in all samples. All you can do is try to reduce it as much as you can. And getting an r=.90 does that very well. I understand the theoretical objection about wanting more ballots to feel comfortable. But, you need to apply that same objection to any other sample that has an r less than .90, if that's your desired threshhold. Which means objecting to just about all sample data.