Which I suppose is money. I don’t read Forbes on a regular basis, and I probably won’t anytime soon after their latest foray into the world of sports analysis.
Forbes annually publishes a list of the most valuable franchises in sports, which is usually a pretty interesting list, even if the Yankees are always at the top. That’s well within their realm of expertise, but this one, ranking the top general managers in major pro sports (MLB, NBA, NFL, and *cough* the NHL), is completely off-base. As they said in the article, “our rankings will surely raise some eyebrows.” Indeed.
The list uses two criteria to determine a GM’s ranking:
- Improvement in winning percentage vs. the GM’s predecessor
- Improvement in cutting payroll relative to the GM’s predecessor
The first criterion is weighted twice so as not to give a boost to GMs who slash payroll but don’t improve in the standings, so they say.
Here are the top 10 in the rankings, which fail to even pass the initial smell test once you read the name at the top:
- Kevin McHale (Minnesota Timberwolves)
- Jay Feaster (Tampa Bay Lightning)
- Billy King (Philadelphia 76ers)
- A.J. Smith (San Diego Chargers)
- Lou Lamoriello (New Jersey Devils)
- Don Waddell (Atlanta Thrashers)
- Marty Humey (Carolina Panthers)
- Jerry Angelo (Chicago Bears)
- Bill Polian (Indianapolis Colts)
- John Paxson (Chicago Bulls)
McHale’s top rank is laughable, since he has done little to help his team after drafting Kevin Garnett years ago. Obviously KG was not consulted in the creation of this list. Here are some of the main problems I have with the methodology here, aside from the ludicrous results above:
- Winning percentage is not an adequate measure across all leagues. There’s a reason that Billy Beane was the #1-rated baseball GM and only #26, because winning percentages are much closer together in baseball. The best baseball teams in a given year have a winning percentage around .600, while the worst are around .400. That’s not a lot of room for improvement using this method.
- The major pro sports have different rules affecting team salaries, from a fairly strict salary cap in the NFL, to a loose one in the NBA, to merely a luxury tax in MLB. These rules go way over my head, and Forbes didn’t even consider them.
- The amount of time it takes to rebuild a lost franchise is greatly different between the sports. NFL GMs have to build rosters of more than 50 players, while NBA rosters max out at 15. Baseball players are on a different development timeline, with most spending several years in the minor leagues, while few top draft picks pan out.
- Sustained success is difficult in any league, and it is not inherently the same degree of difficulty across leagues. A GM should receive credit for any period of sustained success, regardless of the performance of his predecessor.
- Some teams have no incentive to cut costs. The Yankees are among the teams whose GMs should not be punished because of their deep-pocketed ownership. Brian Cashman faces challenges that are unique to him and a few other select GMs.
There are just too many things to consider in order to adequately rank GMs with such a simple formula. This task is immeasurably harder than, say, ranking the best outfielders in the National League, because it’s so hard to determine what really makes a successful GM, and it’s even harder to show how much of a team’s success can be attributed to the GM’s wisdom in drafting and deal-making.
This is the type of analysis that requires much more thought than the writers at Forbes put into their piece. Their list shouldn’t even really be a starting point for the discussion. It’s merely a distraction, taking away the spotlight from the research of real analysts in the sports world. Plus, it makes those of us who write about numbers that are grounded in actual theory look like we generally don’t know what we’re talking about.
Here are a few links to further discussion on this topic:
jfwiii.net 
Nevermind the fact that evaluating GMs compared to their predecessors is a pretty severe problem too (although it’s still clearly not the worst problem here). Anybody following a poor GM will have their score automatically inflated, and vice versa.
I think this might actually top the RPI for the award of “Most Idiotic Rating System Ever Conceived for Any Purpose.” And yes, the RPI is worse than the BCS rankings.
The “VBDI,” Vitale Bald Dome Index, would challenge the RPI for that award. I believe he created it when he saw Southern Illinois and UNLV ahead of Duke in the RPI.