Jump to content

Yo! You're not logged in. Why am I seeing this ad?


I'm working on moving the site to a new server.  If it's down for an hour or two, it's just me testing.


Re-valuing 1st rd draft picks

This topic has been archived. This means that you cannot reply to this topic.
2 replies to this topic

#1 philly sox fan

  • SoSH Member

  • 9752 posts

Posted 22 October 2005 - 08:31 AM

Last winter Nate Silver wrote an article using PECOTA to determine valuations of the 04/05 free agent contracts. He determined that an expected WARP of production was worth 2.14M. He then proceeded to use that simple 2.14M multiplier in a few articles.

The original valuation article:

We're so clever Christian Guzman is a bargain

His method, or at least the 2.14M multiplier, didn’t work very well and actually had little chance of working well because it wildly overvalued low end players. I had planned to write up an appropriately snarky review of the BP articles using Silver’s method. Unfortunately, in response to many e-mails mocking his initial proclamation that Christian Guzman was the bargain signing of the free agent period he wrote a follow-up with a new method to determine player values. Sadly, that took the wind out of my snarky sail.

The revised method is here:

Ok, maybe Guzman stinks

His new formula is:

Salary = (WARP^2 * $212,730) + (WARP * $402,530)

As we’ll see that does a much better job of handling low end player valuations. If you’re a subscriber it’s worth checking out the graph presentation of the data used to generate that equation. All of the data points are between 0-7 WARP. That’s an important consideration when looking at how the new methodology handles star players beyond that production range.

One of his conclusions from that article is:

The pattern here is non-linear. Teams are willing to pay a premium for star talent, above and beyond the number of wins that they can be expected to contribute; signing one 6-win player is more costly than signing two 3-win players.

The non-linearity is the important change, but I wish he had made it clearer that the new equation fit was most applicable to players within that 0-7 WARP range. He seems to think it’s an improvement because of the way it handles “star talent”, but most of the improvement is in the very low end of the player range and arguably there simply isn’t enough data to make the very high end valuations very meaningful.

One of the areas that Silver studied using the blunt tool that was his initial methodology was the valuations of first rd draft picks. Since draft research is of a lot of interest to me (no kidding) I thought it was worthwhile to re-evaluate Silver’s draft research with his new player valuation method.

Silver’s original draft article is here:

It really is called: Lies, Damned Lies - Valuing Draft Picks

In a nutshell, his problem is that he started his analysis with artificially high baseline valuations because his FA methodology was wrong and as a result his shocking revelations about the true high value of draft picks is simply an artifact of that initial mistake. The draft was a particularly bad research area because the nature of “average” draft production is such that it exacerbated the weakness of his methodology. Every draft slot produces a small number of good players and a very large number of bad players. Those bad players add a lot of zeros into the denominator of the “average” calculations and those zeros drive the average draft production down to negligible values right where Silver’s methodology breaks down.

This table shows the value of a player of X WARP under three systems. “Old BP” is Silver’s original multiply everything by 2.14M method. “New BP” uses the formula from his most recent article. POOMA stands for Pulled Out Of My Ass. For something else I was working on, I was just screwing around with valuations along a gradient that might better handle low end player valuations. Considering their origin, my ass, the POOMA valuations work pretty well at the low end WARP player. Prior to Silver’s recent method correction article I re-did all of the calculations from his draft article with the POOMA values. That a bunch of numbers I made up so clearly trounced Silver’s original PECOTA derived valuations amused me so I’m going to keep them in this piece.

WARP       old BP     new BP               POOMA
  1         2.14       0.62          0.5    0.5
  2         4.28       1.66          0.5    1.0
  3         6.42       3.12          1.0    2.0
  4         8.56       5.01          1.0    3.0
  5        10.7        7.33          1.5    4.5
  6        12.84      10.07          1.5    6.0
  7        14.98      13.24          2.0    8.0
  8        17.12      16.83          2.0   10.0
  9        19.26      20.85          3.0   13.0
 10        21.4       25.30          3.0   16.0

The average draft production we’ll be looking at is all in the 1-3 WARP range. “Old BP” put valuations in the 2-6M range for these players. In reality, players in this range generally have a hard time holding jobs as regulars. For example, Bellhorn produced 1.7 WARP in roughly 2/3 of a season. Silver would claim his performance would be fairly valued at ~5M over the course of a season. Despite getting a “bargain” – Bellhorn was only being paid 2.75M - the Sox released him. Silver’s original article was undermined by the fact that he valued a lot of 2005 Bellhorn type players as 4+M values. His “new BP” methodology values these players at a more reasonable 0.6-3M. As we’ll see that makes a huge difference. FWIW, my ass was even harsher on these players with a 0.5-2M value range.

If you look at the differences across the upper mid-range players – 4-7 WARP – you see that “old BP” was very high on the lower end 4-5 WARP players, but by 7 WARP both BP methods converge. POOMA was much lower across the range. I think “new BP” is reasonable in the 4-5 range, but I think it’s probably a bit too high at the 6-7 range.
There are pretty big differences across the three systems for the high end 8-10 WARP players. I liked that POOMA topped out at 16M which seems to be close to the actual high end FA market the last couple of years. “Old BP” places a 10 WARP player over 20M. If you think the recent market correction has artificially held down player salaries, then that might be a reasonable valuation. However, in this player range “new BP” skyrockets. Unless you’re Scott Boras claiming that ARod would still get a 25M/yr contract today, then you probably don’t buy these high end valuations. I believe Derrek Lee had the highest WARP in baseball this year at 12.3 WARP. The “new BP” methodology claims that performance has a free market value of 37M. Yes, the scarcity of truly great performers should accelerate high end performance valuations, but I think the lack of data inputs in that range make the “new BP” valuations unreliable.

Silver’s draft article is broken into four tables and I’m just going to re-run each table through the three different systems. Instead of trying to re-state each line of the table in my own words, I’m just going to use Silver’s explanation.

I've described this methodology in some detail before, so I'll discuss it just briefly now. Market Value is a player's WARP score translated into dollars, using the $2.14 million/win estimate. Marginal Cost represents the salary and related costs to be paid to that player, which we estimate as a fixed sum of $500,000 in his first service year, $750,000 in his second, 31% of his Market Value in Year Three, 44% of his Market Value in Year Four, and 61 and 64% of his Market Value in Years Five and Six, respectively.  [Edit: those percentages were derived from an academic research article Silver sited]
Net Value is simply the player's Market Value less his Marginal Cost. The next row, Gross Present Value, takes his Net Value and discounts it at a rate of 5% a year, assuming that his first year of service time begins 2 1/2 years after he is drafted.

Silver also included a stock figure for Singing Bonus. It looks like an estimate more than an exact number derived from any data. Silver derived a Net Present Value of these picks by subtracting the Signing Bonus from the Gross Present Value. He used the Gross or Net values to make different conclusions.
I should also point out that the WARP totals were derived from the 1989-1994 drafts. I’ve never tried to systematically tease out pre-FA and post-FA service time production, but these certainly seem reasonable. They’re low, anyway, and anything low is pretty reasonable.

In his tables the column after Y6 was a simple “Total” for each row using his valuation methodology. I’ve used three different valuation systems to derive three different “Totals”. “New BP” is Silver’s value update. “Old BP” is the original multiply everything by 2.14M. POOMA is POOMA. The values in the individual year columns are all derived from the “new BP” formula (Edit: the “old BP” values are all in his original article of course).

Silver split the first round into 4 tiers. The numbers in parentheses represent draft slots.
Tier 1(1-7)  Y1    Y2    Y3    Y4    Y5    Y6     New BP  Old BP  POOMA
WARP        2.11  2.37  2.42  2.99  2.56  1.94    14.39    14.38  14.39
MarketVal   1.80  2.15  2.22  3.11  2.42  1.58    13.28    30.78   8.42
MargCost    0.50  0.75  0.69  1.37  1.48  1.01     5.80    11.66   4.14
NetVal      1.30  1.40  1.53  1.74  0.95  0.57     7.48    19.12   4.28
GrosPresVal 1.23  1.33  1.46  1.65  0.90  0.54     7.11    15.39   4.07
SignBonus                                          3.00     3.00   3.00
NetPresVal                                         4.11    12.39   1.07

On average these premium picks produced 2-3 WARP per year for six years for a pre-FA total of 14.4 WARP. The “Old BP” method claimed that a 14.4 WARP player has a Market Value of 30.78M. In theory that means that if random team X signed a marginal scrub that was likely to produce 14.4 WARP over six years to a 6yr/31M contract that would be a fair market value contract. In reality, that would be a hideous contract to an obviously bad player. Right from that first line Silver should have realized that his Market Valuations were outrageously inflated. He didn’t and as a result of his inflated starting point his endpoint valuations were equally inflated.

The “New BP” method dramatically reduces the Market Value from 30.78M to 13.28M. My ass knocked Market Value all the way down to 8.42M. Those very different starting points are carried through by the simple calculations in the next three lines and we end up with a “new BP” Gross Present Value of 7.11M and an “old BP” Gross Present Value of 15.39M.

I think that new Gross Present Value of 7.11M makes sense (and is better than the too low POOMA 4.07M value). Teams generally try to keep the premium prospects to contracts worth about 4-5M. I think it’s reasonable to believe that the draft process keeps those actual bonuses somewhat below the players true market value. The “new BP” difference of 2-3M seems appropriate.

The “new BP” methodology reduces the Net Present Value of these picks by roughly 8M (or 67%) from 12.4M to 4.1M. That takes a sizable bite out of some of Silver’s original conclusions:

MLB bigwigs are fond of complaining about the escalating cost of signing bonuses paid to amateur draft picks (and have done a remarkably good job of controlling them), but a typical top-tier draft pick can be expected to contribute about $15 million worth of value to his club above and beyond what comparable production would cost in the free-agent market. That is a large margin of error to cover for the Williamsons and Brien Taylors of the world, and a team that punts on a first-round pick for the sake of saving a few bucks of signing bonus is almost certainly making a large mistake.

If a bad player was worth 31M, then this conclusion might actually be reasonable.

We’ll take quick looks at his next three tiers.
Tier 2(8-15) Y1    Y2    Y3    Y4    Y5    Y6     New BP  Old BP  POOMA
WARP        1.53  1.94  2.44  1.94  2.73  1.80    12.38    12.37  12.38
MarketVal   1.11  1.58  2.25  1.58  2.68  1.41    10.62    26.48   7.78
MargCost    0.50  0.75  0.70  0.70  1.64  0.90     5.19    10.71   4.36
NetVal      0.61  0.83  1.55  0.89  1.05  0.51     5.44    15.77   3.42
GrosPresVal 0.58  0.79  1.47  0.84  0.99  0.48     5.17    12.61   3.24
SignBonus                                          1.90     1.90   1.90                                
NetPresVal                                         3.27    10.71   1.34

“Old BP” claimed a 12.4 WARP player was worth 26.5M and therefore draft slots #8-15 have a Net Present Value of 10.71M. “New BP” makes a much more reasonable claim that these players are worth 10.6M and therefore these draft slots have a Net Present Value of 3.3M. POOMA is again much lower at 1.3M. In this case, splitting the difference between the “new BP” and POOMA values probably puts you in the right ballpark.

Tier 3(16-25)Y1    Y2    Y3    Y4    Y5    Y6     New BP  Old BP  POOMA
WARP        1.37  1.92  1.85  1.67  1.70  1.32     9.83     9.84   9.83
MarketVal   0.95  1.56  1.47  1.27  1.30  0.90     7.45    21.05   4.93
MargCost    0.50  0.75  0.46  0.56  0.79  0.58     3.63     8.09   2.85
NetVal      0.45  0.81  1.02  0.71  0.51  0.32     3.81    12.96   2.08
GrosPresVal 0.43  0.77  0.97  0.67  0.48  0.31     3.62    10.45   1.98
SignBonus                                          1.50     1.50   1.50
NetPresVal                                         2.12     8.95   0.48

Silver put a break between #15 and #16 because the picks in the second half of the first round can be lost as FA compensation. The Net Present Value of these picks should be considered as part of the cost of any FA signed. Silver spends a lot of time in the original article discussing the effects of these conclusions on the FA compensation system, but I’m not really going to go into that much.

The “old BP” claimed that a 9.8 WARP player was worth 21.05M and therefore these draft slots have a Net Present Value of 8.95M. The “New BP” drops the Market Value to 7.45M and the Net Present Value to 2.12M. POOMA is all the way down to a measly 480k. That may be too low, but we’re dealing with such low WARP totals that the whole exercise is more theoretical then anything else at this point. These Marginal Costs are determined by arbitration based cost estimates for players who do not go to arbitration. There’s just no such thing as a player who sails along producing 1.3-1.9 WARP per year and keeps his job much less goes to arbitration.

Silver never really makes it clear why he put a break between #25 and #26, but this tier also includes all sandwich round picks.
Tier 4(26+)  Y1    Y2    Y3    Y4    Y5    Y6     New BP  Old BP  POOMA
WARP        0.87  0.90  0.80  0.63  0.53  0.35     4.08     4.08   4.08
MarketVal   0.51  0.53  0.46  0.34  0.27  0.17     2.28     8.73   2.07
MargCost    0.50  0.75  0.14  0.15  0.17  0.11     1.81     3.54   1.79
NetVal      0.01 –0.22  0.32  0.19  0.11  0.06     0.47     5.19   0.28
GrosPresVal 0.01 -0.20  0.30  0.18  0.10  0.06     0.44     4.24   0.26 
SignBonus                                          1.00     1.00   1.00
NetPresVal                                        -0.56     3.24  -0.74

The “old BP” claimed that these 4.1 WARP players that produced fractions of WARP per year were actually worth 8.73M and therefore these draft slots have a Net Present Value of 3.24M. The “new BP” drops the Market Value to 2.28M and the Net Present Value to negative 0.56M. By Silver’s new methodology, late first rd draft picks have less Gross Present Value than their signing bonus. On average, the instant you sign a sandwich pick for 1M you’ve just thrown away 560k.

That negative valuation (also present in POOMA) makes sense to me. There is a sharp drop-off in expected production towards the end of the first round and that drop-off is steeper than the decline in signing bonuses. I think it’s reasonable to think that teams come out ahead by 2-3M at the top of the draft, but that edge disappears relatively quickly and somewhere in the second half of the draft the advantage shifts slightly to the players. And it wouldn’t surprise me if that slight advantage is maintained for a couple rounds until the bonuses bottom out.

If it’s true that these picks have negative valuations, then it makes the controversial decisions by Brian Sabean to intentionally forfeit late first round picks much more reasonable than is commonly believed. These “new BP” valuations provide some empirical support for that philosophy. And if you really wanted to give Sabean a BP/Silver/sabremetric embrace, you could reference Silver’s recent work on the important economic drivers for trying to push a high 80s competitive team to become a low 90s playoff team. If someone like Michael Tucker is worth a couple of wins and those couple of wins get your team into the playoffs, then arguably the Giants should be reallocating their budget dollars away from the draft.

You can make that case, but I still don’t like it very much. While this kind of economic analysis may indicate that individual picks aren’t worth their bonuses, it’s not true that a team will systematically benefit from not signing these players. In fact, teams that routinely lose lots of top draft picks to the FA compensation system – a process somewhat analogous to not signing overvalued draft picks – have a difficult time maintaining productive farm systems. And because productive farm systems are the greatest market inefficiency in baseball any team that thinks its saving money by avoiding giving slot money to draft picks is being penny wise, pound foolish. Those individual negative valuations should probably be considered the cost of having a (hopefully) productive farm system.

However, these low valuation late first rd draft picks provide great opportunities for large revenue team to invest money to make even more money by drafting and signing premium players who fall in the draft. Let’s look at the Sox signing of Hansen and Dodgers pick of Hochevar. At #24 the Sox would have been expected to pay a draft slot appropriate player 1.3M. That player’s Gross Present Value might be roughly 2M. In purely financial terms, the Sox would clear 700K. Instead the Sox will pay Hansen 4M, but his Gross Present Value might be 6M. By dramatically increasing their expenditure the Sox could end up clearing 2M or nearly 3 times as much value as if they had taken a draft slot player.

The numbers are even more dramatic for the Dodgers and Hochevar. At #40 the Dodgers would be expected to pay 850K and that player might only have a Gross Present Value of 500k. On average, they would expect to lose 350k from that player. Hochevar is a premium talent who might have a Gross Present Value of 6M. Thanks to some confusion with Hochevar’s representation, the Dodgers almost signed him for 3M. So a slot that should have cost them a few hundred thousand would have instead returned to them 3M. As the old saying goes, it’s a heck of a lot easier to make money when you have money. That the Yankees almost never take advantage of these opportunities has been colossally stupid.

And if this can’t be as long and snarky as originally intended, at least I can end with the Yankees being colossally stupid.

#2 mr guido

  • 3112 posts

Posted 22 October 2005 - 09:20 AM

Great writeup. A question that strikes me...

Even once the values drop into the negatives it is still by a relatively slim margin. This means that a relatively modest improvement in avg production out of a particular slot may be able to swing the values to a net positive. What's your take on the level of variance for WARP from a tier 4 pick from organization to organization? That is, how much better should we expect a "smart" organization to be at drafting? Is it negligible? Or would it be enough to make a sandwich pick a net positive (even before considering implications of building a farm system)? If it's the latter, any team's decision to purposely skimp on these types of picks is an admission that they just suck at drafting relative to the rest of the orgs out there.

#3 wade boggs chicken dinner

  • SoSH Member

  • 7849 posts

Posted 23 October 2005 - 09:55 PM

It would be interesting (and far beyond my ability) to do this analysis not from the spot in which a player is drafted, which might include signability issues, but from how the player is rated prior to the draft. As Mr. Guido noted, it seems that when players like Hansen fall because of signability concerns, it skews how much future WARP will be obtained from the slot.

Maybe I'm wrong about this.