View Single Post
07-29-2012, 10:51 AM
#20
Iain Fyffe
Hockey fact-checker

Join Date: Feb 2009
Location: Fredericton, NB
Country:
Posts: 3,509
vCash: 500
Quote:
 Originally Posted by Smokey McCanucks Would statistical analysis tell you that Dustin Penner would be one of the most key guys in the playoffs this season? The stats would probably tell you Penner sucks. But he came through when it mattered.
The crucial point here is: no one could have told you that. Some people might have predicted that, but then people are often subject to confirmation bias, where they tout their successful predictions and ignore their incorrect ones. Stats couldn't tell you that Penner will perform well in the 2012 playoffs? Well, neither could anything else.

All players have peaks and valleys in performance. Those that have peaks in the playoffs are called "clutch." Those that have valleys are called "chokers." Even if these peaks and valleys are randomly distributed. That's one of the biggest differences in points of view between statistical and traditional analysis. Traditional analysis tries to attach meaning to every variance in performance (AKA building narratives from randomness). Statistical analysis looks at trends in performance level, and sees peaks and valleys everywhere, with no particular pattern. Traditional analysis loves streaks and slumps, statistical analysis sees little value in them.

One reason that statistical analysis cannot define clutchiness is that there is little to no evidence that it actually exists. Players who are clutch one year are suddenly not clutch the next. Cam Ward is so clutch that he usually can't even get his team into the playoffs. If it were a real thing, it would be observable and predictable, and not rely on ex post facto explanation. If clutchiness can only be identified after the fact, chances are you're attaching meaning to patterns that are not necessarily real patterns.

Because that's another thing the human mind is seriously prone to: seeing patterns in random noise. It's often called apophenia or patternicity. Ask a person to create a random series of As and Bs and they will almost always create a series of patterns, whereas an actual random series will include long stretches of As and Bs, which a human brain will see as a pattern, even though there isn't one.

Here's a random sequence of 100 I created with an online tool:

babbabaaabbbaaaabbaaaabbaaaababbbababaaabbaabbabbb abbbbababbabbbabbbaaabbaabababbaaabababaabbbbbbbaa

Note the seven consecutive Bs near the end, which traditional analysis calls a streak, and if it occurred in the playoffs would be seen as clutch performance (if B is good) or choking (if B is bad). There's only one sequence where it goes ABABABA, otherwise it's groups of the same letters for the most part. There are three runs of four consecutive As, very close to each other, resulting in a "streak" of 12 of 16 letters that are A. There's also a run where 16 of 21 letters are Bs.

There are all kinds of streaks and slumps here, and yet the sequence is randomly generated by a computer. Traditional analysis assigns meaning to streaks and slumps. Statistical analysis sees normal variance in performance.

Here's the next one I did (I'm using this tool BTW):

baabababaaaaaababbbabbaababaaaabbbbbbbaaaaaababbbb bbaabaabbabbbbaaabbbbbbbbbbbbbbaabbaaaaabbbbbabaaa

55 Bs and 45 As (the first was 53 and 47, this player must be good at getting Bs), and a series of 14 consecutive Bs at one point. Also four As followed by six Bs then five As. Patterns and streaks everywhere, and yet it's actually random.

There are just the first two series I created. Further ones might "look" a little more random that this last one, but not much more. The fact is, most people don't really comprehend what "random" means, even if they think they do.

Quote:
 Originally Posted by Smokey McCanucks That's a part of hockey, it's intangible and it transcends "Sabremetric" statistical analysis, it exists on another level, another plane.
And so it belongs, perhaps, in another forum?

Quote:
 Originally Posted by Jagorim Jarg Case in point: Can you name me one team that wouldn't have drafted Alexandre Daigle first overall?
Indeed, both statistical analysis and the scouts agreed that Daigle was the best bet that year. But things like drafting are just that: bets. You can find the players most likely to succeed in the NHL, but no more than "most likely".

Was drafting Daigle a bad idea? Probably not. If there were better, consistent, psychological profiling of prospects, maybe there would have been a clue. But then, maybe not. Professional scouts miss all the time on draft picks. There's just no way of ever predicting the future with 100% accuracy.