DawgTalkers.net Forums The Archives 2013 NFL Season NFL Draft (2013) The NFL Draft : How We Decide

The NFL Draft

Posted on: April 8, 2010 11:22 AM, by Jonah Lehrer

The 2010 NFL Draft is later this month, and there is already plenty of speculation about which QB will go first, and which DT is a better choice, and which teams will trade up for a higher draft pick. The stakes for the teams are huge, as a failed draft pick will not only waste millions of dollars in salary but will also come with a high opportunity cost. So there is a strong incentive to get the decision right, and to have a decision-making system that leads to the right personnel pick.

And yet, that hasn't happened. Instead, NFL teams remain tethered to useless metrics. Just look at the NFL scouting combine, which is a big job fair for prospective NFL players. (The combine includes everything from the 40 yard dash to a batter of psychological tests.) In recent years, the combine has become a major press event, and teams regularly cite combine results when justifying draft picks. But this is a mistake, as the combine is a big waste of time:

Combine measures examined in this study include 10-, 20-, and 40-yard dashes, bench press, vertical jump, broad jump, 20- and 60-yard shuttles, three-cone drill, and the Wonderlic Personnel Test. Performance criteria include 10 variables: draft order; 3 years each of salary received and games played; and position-specific data. Using correlation analysis, we find no consistent statistical relationship between combine tests and professional football performance, with the notable exception of sprint tests for running backs. From a practical standpoint, the results of the study should encourage NFL team personnel to reevaluate the usefulness of the combine's physical tests and exercises as predictors of player performance. This study should encourage team personnel to consider the weighting and importance of various combine measures and the potential benefits of overhauling the combine process, with the goal of creating a more valid system for predicting player success.

In How We Decide, I devote a few pages to critiquing the Wonderlic, which is an intelligence test given to players at the combine. While Wonderlic scores are seen as important for prospective QB's, there is scant evidence that the test predicts QB success. The underlying reason, I think, is that the kind of logical, abstract intelligence measured by the Wonderlic has little to do with the kind of decisions made on the football field; knowing pre-algebra doesn't help you find the open man.

The larger lesson of the NFL draft is that it illustrates the astonishing difficulty of predicting success. Think about all the advantages that NFL teams have: years of detailed performance data from college and high school games; quantitative results from the combine; the ability to conduct personal interviews; thorough medical reports and drug tests; press coverage, etc. In other words, they know more about the people they're drafting than just about any other employers on the planet. Nevertheless, they still can't consistently predict which 350 pound offensive linemen will be able to block, and which QB's will be able to endure the rigors of a blitzing defense, and which wide receivers will stay motivated even when they don't get the ball. If these teams can't get it right, with all this data and expertise, then how the hell can the rest of us? If we can't even evaluate a narrow and specific athletic skill, then how can we ever expect to retain the best teachers, or pick the best politicians, or fund the best scientists, or hire the best executives?

But wait - it gets worse. Not only have NFL teams failed to find relevant and reliable variables for predicting future player performance, but they frequently pretend as though they know exactly what they're doing. Instead of embracing their ignorance, most NFL teams default into overconfidence, which is why they regularly trade up draft picks. (The teams are so convinced that they've the identified the perfect player that they are willing to pay much higher salary and relinquish future draft picks.) Cade Massey and Richard Thaler have a new paper that looks at the "return on value" from early draft picks. They conclude that, on average, the first pick in the draft is the least valuable in the entire first round. Here's Thaler explaining this research:

If the market for draft picks were "efficient," meaning that the prices reflected intrinsic value, the resulting value for a team that trades up for a higher pick should be equal to the value of the picks it gives up. The price of moving up is steep: to move from the 11th pick to the 5th pick, for example, a team would have to forfeit its second-round pick as well. To be worth it, the player taken just six picks earlier would have to be a whole lot better -- because both of the players given up could have become stars, too.

How confident should a team be that this early pick is better? Suppose we rank all the players at a given position -- running back, linebacker, etc. -- in the order they were picked in the draft, then compare any two in consecutive order on the list. What do you think is the chance that the player picked higher will turn out to be better -- as judged, say, by number of games started in his first five years in the league?

If teams knew nothing, the answer would be 50 percent, as it would be for flipping a coin. If they had perfect knowledge, the answer would be 100 percent. Go ahead, make your guess.

The answer is 52 percent -- an outcome that is barely better than that of a coin flip.

What this research makes clear is that powerful incentives don't automatically lead to better decisions or effective models. (Not that we needed the NFL as proof - just look at Wall Street.) Even when our biases are penalized - and the sin of overconfidence has cost NFL teams a lot of money - we still cling to them. The powers of market forces are no substitute for a little self-awareness.

Link

Another article referencing the same paper.

When a First Pick Isn’t the Best Pick
By RICHARD H. THALER
Published: April 2, 2010

AS March Madness winds down, it’s time to turn our attention to the next major sports event on the calendar. That’s right, the National Football League draft, now a three-day extravaganza with the first round broadcast in prime time on April 22.
Enlarge This Image
David G. Klein

Spring is a fitting time for the draft, because hope springs eternal for fans of downtrodden teams. The rules of the draft specify that teams can choose according to their won-lost records from the previous year, with the worst teams choosing first. The idea is to give the bad teams the early picks to help them become winners.

As it turns out, however, the draft does not play the Robin Hood role particularly well. Indeed, I have written a paper, recently revised, on this subject with Cade Massey, a professor at the Yale School of Management. We found that the teams choosing early in the draft generally don’t, in fact, get the players that provide the most value per dollar. Our paper is titled “The Loser’s Curse” because we discovered that the first pick in the draft is, on average, the least valuable in the entire first round.

That surprising result has implications not only for football, but also for any domain where organizations try to select talent, whether C.E.O.’s or their own “rookies” — newly minted graduates.

Our analysis pivots around the idea that while there is an active trading market for N.F.L. draft picks, the market places too high a value on picking early. A team that wants to select a player before its turn can offer another team a deal in order to “trade up.” Over time, teams have come to agree on the price of such trades by resorting to a table now universally known as the Chart, which assigns a value to each pick in the draft. Alas, the Chart has the “wrong” prices.

If the market for draft picks were “efficient,” meaning that the prices reflected intrinsic value, the resulting value for a team that trades up for a higher pick should be equal to the value of the picks it gives up. The price of moving up is steep: to move from the 11th pick to the 5th pick, for example, a team would have to forfeit its second-round pick as well. To be worth it, the player taken just six picks earlier would have to be a whole lot better — because both of the players given up could have become stars, too.

How confident should a team be that this early pick is better? Suppose we rank all the players at a given position — running back, linebacker, etc. — in the order they were picked in the draft, then compare any two in consecutive order on the list. What do you think is the chance that the player picked higher will turn out to be better — as judged, say, by number of games started in his first five years in the league?

If teams knew nothing, the answer would be 50 percent, as it would be for flipping a coin. If they had perfect knowledge, the answer would be 100 percent. Go ahead, make your guess.

The answer is 52 percent — an outcome that is barely better than that of a coin flip. This means that although the value of players declines throughout the draft, quality declines more slowly than compensation — players picked early are very highly paid. As a result, the first pick in the draft has often provided less value to his team, in performance per dollar, than the last pick in the first round (the one awarded to the Super Bowl winner). In other words, in the world of the N.F.L. draft, the rich get richer.

The league has helped to create this problem in the way it pays rookies. A special salary cap applies to rookies and, unlike the overall salary cap the league uses, it varies across teams. The teams with the first picks get more money to spend on signing their draftees. Since agents know how much a team has been allocated to sign the first-round pick, they demand that amount for their player. As a result, compensation for draft picks declines almost in lock step; the first player gets the most, then the second pick, and so on, with the last player taken in the first round getting only 25 percent of the amount awarded to the first pick.

It turns out that the N.F.L. and the players’ union are trying to renegotiate their collective bargaining agreement. One topic reported to be on the bargaining table is shifting some of the pay away from early draft picks.

The owners and players should find common ground on this issue, because it makes absolutely no sense to be giving so much money to unproven rookies, many of whom turn out to be busts. By reducing the premium paid to the highest draft choices, the league could restore the redistributive goal that the draft was created to achieve. And veteran players would probably agree with the principle that eight-figure salaries should be reserved for players who have already proved themselves on the field.

SO if teams’ ability to select players is only slightly better than flipping coins, should we expect that corporations can do any better in picking their chief executives?

After all, it’s probably easier to predict the performance of football players than of C.E.O.’s. Athletes perform the same job in a very public forum for years, and all aspects of their job are subject to wide critical evaluation. They are also given extensive physical and mental tests. (Yes, it is important for a football player to be smart — and several years of college don’t assure that.)

On the other hand, chief executives hired from outside a particular company have been performing mostly in private. And I’ve never heard of a prospective C.E.O. being given an I.Q. test — or having to run the business version of the 40-yard dash (perhaps a press conference?).

So maybe companies shouldn’t pay big bucks in the desperate hope of getting the equivalent of a Peyton Manning, who was the first overall pick in 1998 and, of course, has proved his superstar value. Instead, maybe they should dig around for a replica of Tom Brady, who was the 199th pick in the 2000 draft and has gone on to play in four Super Bowls, winning three.

Richard H. Thaler is a professor of economics and behavioral science at the Booth School of Business at the University of Chicago.
Kink

I read the title and thought to myself............. a coin flip?

I think where darfts get "goofed up" is the "ratings" of players. I think it is who do you really like. Do you pass on someone you like because of some arbitrary rating that says you are taking them "too high". I like berry and Rolando mcClain no matter where their number is. they will be a big impact...why let mcclain fall to the giants because of some smhecksperts have him rated in the 20s. he is a great player in college and will be a great pro.

Yeah, I think you have to pick the guy you want whenever you pick. The only exception would be if you think you could get him a round later and therefore could get 2 guys you want by picking the other one first.