I have written several articles detailing the flawed methodologies of "advanced basketball statistics," including Economics is Not a Science, Nor is Basketball Statistical Analysis and Economics is Not a Science, Nor is Basketball Statistical Analysis, Part II. Phil Birnbaum is a "stat guru" who primarily focuses on baseball, a sport whose discrete, one on one encounters between pitchers and batters lends itself much more readily to accurate statistical analysis than a free flowing five on five sport like basketball. Birnbaum has taken a look at "advanced basketball statistics" and he is not impressed by what he found:
You know all those player evaluation statistics in basketball, like "Wins Produced," "Player Evaluation Rating," and so forth? I don't think they work. I've been thinking about it, and I don't think I trust any of them enough put much faith in their results.
That's the opposite of how I feel about baseball. For baseball, if the sportswriter consensus is that player A is an excellent offensive player, but it turns out his OPS is a mediocre .700, I'm going to trust OPS. But, for basketball, if the sportswriters say a guy's good, but his "Wins Produced" is just average, I might be inclined to trust the sportswriters.
I don't think the stats work well enough to be useful.
Please click on the above link and read Birnbaum's article in its entirety, because he does an excellent job of explaining exactly how difficult it is to correctly assign individual credit for team success in basketball--and Birnbaum does not even address an issue that I have brought up several times: the raw box score numbers themselves are very subjective (I have mainly focused on assists but the same could be said for blocked shots, steals and, to some degree, even rebounds, depending on how the official scorekeepers define tips, etc.).
Birnbaum cites a study by David Lewin and Dan T. Rosenbaum that shows that minutes played by players in a preceding season is at least as good of a predictor of team performance in the subsequent season as the so-called "advanced basketball statistics" are. Birnbaum notes that minutes played "is probably the closest representation you can get to what the coach thinks of a player's skill," so this is an indication that--contrary to the constant bleating by "stat gurus" like Dave Berri and their media sycophants like Henry Abbott--NBA coaches actually do have some idea about what they are doing.
Birnbaum expresses some hope that plus/minus statistics could be useful if the sample sizes are large enough but, as I previously reported, "stat guru" Ken Pomeroy has studied plus/minus stats and is very skeptical of their usefulness. Birnbaum concludes, "But, just picking up a box score or looking up standard player stats online, and trying [to figure out] from that which players are how much better than others (the approach that 'Wins Produced' and other stats take)...well, I don't think you're ever going to be able to make that work."
MLB "Stat Guru" Phil Birnbaum Explains Why "Advanced Basketball Statistics" Don't Work
