View Single Post
07-30-2012, 02:26 PM
Registered User
TheDevilMadeMe's Avatar
Join Date: Aug 2006
Location: Brooklyn
Country: United States
Posts: 45,861
vCash: 500
I'm the only moderator of the History Board who isn't also a mod here (and there's still time for that ).

I think advanced statistics are an invaluable tool, but I think a lot of stats people get so bogged down in numbers that they sometimes loose track of the essence of the game itself. I find instances where the statistician overreaches and claims his stats how much more than they actually do to be unhelpful.

Two specific criticisms of mainstream hockey analytics:

1) The assumption that "conventional hockey wisdom" is worthless if it can't be statistically proven. Countless times, I have seen conventional wisdom tossed aside by the latest man with "The Answer" only to see later statistical work indicate that yes, the conventional wisdom had something to it. Off the top of my head, "goaltenders have no effect on shots against" and "skaters have no effect on save percentage" to be mindbogglingly ignorant statements (especially the second one), yet for a time were (and in same cases still are) accepting as truisms by some in the hockey analytics community.

I think the responsible thing to do would be to start with the assumption that conventional wisdom has a grain of truth to it, and should only be thrown out in the face of convincing evidence to the contrary (which we do have in quite a few cases). The current assumption seems to be that conventional wisdom should be dismissed off the bat unless convincing (statistical) evidence can be found in favor of it.

Taken to the extreme, the collective opinions of paid NHL GMs and coaches are dismissed as those of a bunch of meatheads stuck a past without the newfangled stats.

2) The tendency to dismiss every effect that can't be easily explained statistically as "luck." The easiest example I can think of is the commonly used blanket statement that any increase or decrease in playoff performance is due to random variation. This would make sense of players were simply machines driven by probability engines, but completely ignores the psychological difference between the playoffs and the regular season both in terms of pressure and in terms of playing the same opponent over and over again.

This might be a specific example of #1 (dismissing conventional wisdom as luck out of hand).

Back to me now: In case you haven't noticed, I'm much better at criticizing studies than coming up with my own

TheDevilMadeMe is online now   Reply With Quote