View Single Post
04-28-2012, 10:46 PM
Student Of The Game
seventieslord's Avatar
Join Date: Mar 2006
Location: Regina, SK
Country: Canada
Posts: 30,940
vCash: 500
Originally Posted by plusandminus View Post
Meeting eye-ball tests is important. But when I post, rather than just pointing out well-known and obvious things, I often write about things that may not be as obvious or well-known. One example was regarding New Jersey's penalty killing in 2002-03. It is very well-known that Scott Stevens is considered one of the best defensive defencemen ever. Yet, their 2nd penalty killing unit had far better "per minute" stats.
(posts 52 and especially 62)
Facts are that S.Stevens was on ice during 51.54 % of the penalty killing icetime. The "adjusted icetime" suggested he was on ice for 81.25 %.
Facts are that S.Niedermayer was on ice during 40.61 % of the penalty killing icetime. The "adjusted icetime" suggested he was on ice for 18.75 %.
The "adjusted stats" shows that "Wow!!! Stevens played an amazing 81 % during PK! Niedermayer played far, far less, just 19 %."
Facts says "Stevens played 51.54 % of the time, while Niedermayer played 40.61 %".
Why manipulate stats to say that a player had 4 times more icetime than another, when in reality he had just 1.3 times more icetime?
The reply I got was - when analyzed - that Stevens faced more than 3 times harder opposition than Niedermayer. (Someone is welcome to show me factual proof that supports that claim.) I think I even researched that, by looking at which opponents that was on the ice during the goals, and found that things were not as black and white as I was told here on the board. It should also be noted again that when New Jersey played on the road, they didn't have the benefit of choosing which opponents Stevens or Niedermayer played against.

The above case is admittedly definitely rather extreme, and it's about penalty killing ice time.
Just to make sure, how sure are you that you executed the estimate formula the same way that the originators did? I know that a factor of 1.2 or 1.3 is applied to first lines at ES, so it wouldn't surprise me if something similar or greater was applied on the PK too. Because it's true, top unit PKers should end up with more GA, typically. the fact that they are facing top PP unity typically outweighs the fact that they are the best PKers, right? I can't imagine they would have done this project without accounting for that, but I could be wrong.

You are right, this is an extreme example either way.

But there are many cases where even strength adjusted icetimest turned out to be rather wrong. I ranked every defenceman within each team based on estimated versus factual icetimes, and looked at the top-5 ones to see if the estimations at least managed to tell if a defenceman was e.g. "2nd defenceman", and I think the estimations produced errors in about 50 % of the cases.
What did you classify as an "error"? How far away from the actual result did it have to be to be classified as such?

Basically I just want it to be clear that estimated icetimes are estimations and not facts. They are fairly accurate, but are unreliable to use to "rank" players based on "who played most". I also find them unreliable as a parameter in larger formulas, like "best scorers per 60 minutes of icetime". I understand it's up to everyone to decide how to use estimated icetimes.
I think the unreliability of these numbers gets overstated by the conservative types around here. Everywhere you look, the players with the reputations of being the best players end up with the highest results and the worst end up with the lowest. What's more, with no change to the methodology used, it ends up with higher totals of top players in the first 20 years post-expansion compared to what we're used to nowadays, which is exactly how everyone remembers it.

seventieslord is online now   Reply With Quote