HFBoards

Go Back   HFBoards > Fantasy Hockey Talk > All Time Draft
All Time Draft Fantasy league where players of the past and present meet.

Modern Hockey Analysis Tools

Reply
 
Thread Tools
Old
03-31-2012, 08:53 AM
  #1
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
Modern Hockey Analysis Tools

I'd like to open up a conversation on the value of modern hockey statistical analysis and the results thereof. This is a subject that I've been delving into a lot lately, and I think there is a lot of value in it, with a few caveats. I'll start by taking a look at my favorite of the modern analytical sites: http://hockeyanalytics.com/

The research behind this site got off the ground in 2004, and goes back to an analysis of the 2002-03 season as the beginning of its data. This page:

http://hockeyanalytics.com/seasons/2003-season/

...contains two of the most important essays on the foundational principles that this Alan Ryder guy uses in doing his analysis - specifically Shot Quality and Player Contribution.

Shot Quality is an attempt to improve upon SV% based analysis of goalie performance by incorporating shots taken data from the NHL into the equation. The NHL generates data for each shot taken in the league - specifically who took it, from what distance, and what kind of shot it was. Mr. Ryder breaks down this data to determine which sorts of shot/distance combinations are likely to produce which sorts of results in order to arrive at a shot quality metric, which he then applies granularly to each goalie in the league as an adjustment to the base SV% statistic. And voila! A modern goaltending performance metric cometh.

Player Performance is more complicated. It spins off of Bill James' Sabremetric-style Win Shares statistical analysis in baseball and applies it to the hockey rink. Specifically, he looks at Marginal Goals Created and Marginal Goals Prevented as his units of "credit" for each player in the NHL. Mr. Ryder breaks the game down into its component parts.
- Each team gets a certain number of credits for performance above marginal (replacement) value, in terms of goals created and goals prevented. This is taken directly from the GF/GA statistics.

- Goaltending is assigned 1/6th the total marginal value in the game, with all of that value coming on the defensive half, so generally speaking (this is a simplification) a total of 1/3rd of each team's total marginal Goals Prevented value, assuming a perfectly average goalie.

- there is a "league wide constant" which states that 58% of the game is actually defense, and thus defensive value is weighted more heavily than offensive value in the calculations.

- Offensive value is calculated and broken up per player (and per game situation) based on stats we all know well.

- ...and what's left over (after we filter out the goalie and the offensive value in the game) is the defensive value. It is a top-down approach to arriving at defensive value, which I think is appropriate, as there are no clear statistics which could be used to drive a more direct statistical analysis.

- from what's there of the team defensive value, the credits are then distributed among the relevant players based on position (defensemen get approximately 60%), an improvement on the +/- statistic (which I don't have the energy to discuss at this point) and Time On Ice
It's a complicated system with a lot of moving parts and a lot of statistical analysis that will go over many people's heads (so be it), but I think in the end it has a lot of merit, and is an excellent start down the road to a more rational form of hockey statistical analysis.

Nits that I would pick:

- Mr. Ryder will admit to a few problems with the Shot Quality data. Specifically, the statkeepers in certain arenas may be high, but he has a built-in arena factor which seeks to remedy this issue.

- Defensive Contribution does a poor job of seperating the actual defensive value of even strength partners. Two guys who skate together on the same pairing throughout the season are going to have nearly identical ES Defensive Contribution values in this metric, even though the eye test tells us that they may have widely different actual values in terms of preventing goals. Mr Ryder's metrics generally hate on Zdeno Chara, and this is one of the reasons why. It's also the reason why Brian Rafalski shows up as one of the best defensemen in the league when he is partnered with Nicklas Lidstrom.

- Defensive Contribution also cannot take into account strength of matchups when distributing credits. Personally, I think this is an overblown factor in actual performance in the post-lockout era where defensive matchups are much more difficult to dictate without resort to simply icing the puck to get a line change. But insofar as this factor is meaningful, it is not accounted for in the numbers.

- Finally, I think the value of goaltending is somewhat overrated in his analysis. It simply doesn't pass the smell test to me that, according to his analysis, the best players in the league year-after-year are goaltenders...and when I say the best I mean almost the entire top-20. Sorry, but Martin Gerber was not the 20th most valuable player in the league in 2007-08. This distortion (in my opinion) is another reason why Mr Ryder's metric hates on Zdeno Chara; it assigns such a huge portion of the defensive credit to Tim Thomas over the past few seasons that there is little left for Boston's defensemen.

At any rate, Chara's standing in the Player Contribution tables is a persistent fly in this guy's ointment and goalie value has to be taken with a grain of salt, but otherwise, I think his analysis is highly enlightening. You can find links to his archived yearly NHL review pages (which have linked PDF summaries and Excel documents) here:

http://hockeyanalytics.com/2004/03/player-contribution/

Happy reading.

Sturminator is offline   Reply With Quote
Old
03-31-2012, 02:14 PM
  #2
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
Here's a sample of the results of hockeyanalytics' Player Contributions analysis...it's the top-20 defensemen from 2002-11 (PCO = Player Contribution Offense; PCD = Player Contribution Defense; PC = Total Player Contribution):

2002-03:

Name PCO PCD PC
Nicklas Lidstrom 53 71 124
Al MacInnis 56 58 114
Sergei Zubov 40 66 106
Sergei Gonchar 59 44 103
Scott Niedermayer 27 72 99
Dan Boyle 43 55 98
Eric Desjardins 24 73 97
Kim Johnsson 29 67 96
Wade Redden 33 63 96
Zdeno Chara 31 61 92
Tomas Kaberle 35 57 92
Brian Rafalski 22 68 90
Toni Lydman 11 73 84
Robert Svehla 31 50 81
Sami Salo 26 53 79
Rob Blake 34 45 79
Derian Hatcher 17 61 78
Alexander Khavanov 25 52 77
Scott Stevens 9 66 75
Filip Kuba 19 55 74

2003-04:

Name PCO PCD PC
Scott Niedermayer 43 71 114
Bryan McCabe 49 57 106
Chris Pronger 49 56 105
Kenny Jonsson 18 76 94
Nicklas Lidstrom 21 72 93
Adrian Aucoin 32 58 90
Sergei Zubov 25 65 90
Kim Johnsson 28 60 88
Mathieu Schneider 38 49 87
Wade Redden 34 53 87
Sami Salo 15 66 81
Zdeno Chara 35 44 79
Dick Tarnstrom 54 25 79
Philippe Boucher 17 61 78
Brent Sopel 31 45 76
Jordan Leopold 21 55 76
Mattias Ohlund 26 49 75
Tomas Kaberle 19 54 73
Kimmo Timonen 33 39 72
Dan Boyle 25 47 72

Sturminator is offline   Reply With Quote
Old
03-31-2012, 02:21 PM
  #3
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
2005-06:

Name PCO PCD PC
Sergei Zubov 70 79 149
Nicklas Lidstrom 58 85 143
Chris Pronger 30 100 130
Tomas Kaberle 40 67 107
Dan Boyle 37 64 101
Lubomir Visnovsky 50 51 101
Mathieu Schneider 49 47 96
Tom Preissing 30 63 93
Brian Rafalski 20 73 93
Dion Phaneuf 40 49 89
Paul Martin 17 71 88
Joe Corvo 32 54 86
Marc-Andre Bergeron 24 60 84
Andrei Markov 33 50 83
Bryan McCabe 50 32 82
Philippe Boucher 35 46 81
Scott Niedermayer 37 38 75
Robyn Regehr 12 63 75
Steve Staios 18 55 73
Joni Pitkanen 40 33 73

2006-07:

Name PCO PCD PC
Nicklas Lidstrom 42 88 130
Sergei Zubov 46 67 113
Tomas Kaberle 42 68 110
Scott Niedermayer 49 52 101
Chris Pronger 39 58 97
Dan Boyle 46 47 93
Philippe Boucher 46 46 92
Brian Rafalski 33 59 92
Lubomir Visnovsky 46 45 91
Sami Salo 35 56 91
Ryan Whitney 41 44 85
Sheldon Souray 62 22 84
Ron Hainsey 17 65 82
Scott Hannan 11 71 82
Jay Bouwmeester 25 56 81
Brian Campbell 29 52 81
Mathieu Schneider 38 41 79
Bryan McCabe 39 40 79
Sergei Gonchar 40 38 78
Marc-Edouard Vlasic 7 69 76

Sturminator is offline   Reply With Quote
Old
03-31-2012, 02:30 PM
  #4
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
2007-08:

Name PCO PCD PC
Nicklas Lidstrom 43 78 121
Brent Burns 51 56 107
Brian Rafalski 36 69 105
Zdeno Chara 55 42 97
Mike Green 53 44 97
Dion Phaneuf 60 33 93
Tomas Kaberle 35 58 93
Adrian Aucoin 23 65 88
Andrei Markov 49 34 83
Duncan Keith 23 60 83
Dennis Wideman 28 55 83
Sergei Gonchar 41 40 81
Marc-Edouard Vlasic -1 79 78
Ron Hainsey 12 66 78
Daniel Girardi 19 59 78
Mathieu Schneider 38 39 77
Paul Martin 15 61 76
Rostislav Klesla 2 72 74
Craig Rivet 19 56 75
Niklas Kronwall 24 50 74

2008-09:

Name PCO PCD PC
Mike Green 84 49 133
Nicklas Lidstrom 47 70 117
Brian Rafalski 39 63 102
Dan Boyle 51 51 102
Scott Niedermayer 37 55 92
Andrei Markov 53 36 89
Dennis Wideman 40 43 83
Marc-Edouard Vlasic 17 65 82
Shea Weber 44 38 82
Duncan Keith 30 49 79
Jan Hejda 12 65 77
Zdeno Chara 42 35 77
Chris Pronger 26 49 75
Joe Corvo 26 48 74
Paul Martin 21 52 73
Kimmo Timonen 26 46 72
Mark Streit 38 33 71
Sheldon Souray 41 30 71
Brian Campbell 28 42 70
Willie Mitchell 16 53 69

Sturminator is offline   Reply With Quote
Old
03-31-2012, 02:36 PM
  #5
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
2009-10:

Name PCO PCD PC
Mike Green 71 33 104
Drew Doughty 49 55 104
Chris Pronger 32 70 102
Nicklas Lidstrom 33 67 100
Matt Carle 22 78 100
Duncan Keith 48 47 95
Dan Boyle 57 34 91
Chris Phillips 19 68 87
Tyler Myers 41 45 86
Andy Greene 19 67 86
Christian Ehrhoff 37 48 85
Brian Rafalski 28 55 83
Keith Yandle 33 49 82
Mark Streit 31 46 77
Shea Weber 36 42 78
Kimmo Timonen 20 53 73
Adrian Aucoin 29 43 72
Fedor Tyutin 17 55 72
Zdeno Chara 31 40 71
Brian Campbell 23 48 71

2010-11:

Name PCO PCD PC
Alex Pietrangelo 32 58 90
John Carlson 30 60 90
Brett Clark 19 70 89
Nicklas Lidstrom 43 44 87
Dan Boyle 46 36 82
Keith Yandle 41 40 81
Niklas Kronwall 28 52 80
Lubomir Visnovsky 54 25 80
Christian Ehrhoff 40 37 77
Brent Burns 41 34 75
Shea Weber 36 39 75
John-Michael Liles 33 41 74
Fedor Tyutin 16 58 74
Trevor Daley 17 56 73
Andy Greene 7 65 72
Dustin Byfuglien 55 17 72
Kris Letang 33 38 71
Mark Giordano 26 42 68
Erik Karlsson 38 30 68
Duncan Keith 22 46 68

Sturminator is offline   Reply With Quote
Old
03-31-2012, 02:57 PM
  #6
TheDevilMadeMe
Global Moderator
 
TheDevilMadeMe's Avatar
 
Join Date: Aug 2006
Location: Brooklyn
Country: United States
Posts: 37,813
vCash: 500
Oh, this is the famous metric that ranks Sergei Zubov first in 2006 because of shootout goals?

The use of shootout statistics on the metric is obviously a big issue post-expansion.

TheDevilMadeMe is offline   Reply With Quote
Old
03-31-2012, 02:59 PM
  #7
overpass
Registered User
 
Join Date: Jun 2007
Posts: 3,511
vCash: 500
Quote:
Originally Posted by Sturminator View Post
Nits that I would pick:

- Mr. Ryder will admit to a few problems with the Shot Quality data. Specifically, the statkeepers in certain arenas may be high, but he has a built-in arena factor which seeks to remedy this issue.
I think there are still a few things that Shot Quality doesn't pick up on.

Basically it needs to identify those shots that NHL coaches and teams want to prevent above all else, and separate them from those shots that they are willing to allow to some degree. The biggest thing it misses, IMO, is lateral movement before the shot. If the offensive team passes the puck laterally or even just has room to make a lateral move with the puck before shooting it makes it a lot more difficult for the goaltender to stop. There's no way to identify this statistically without tracking it during the games.

The 1972 Summit Series is an interesting example of this. The blog Black Dog Hates Skunks did a rewatch of the 1972 Summit Series and counted scoring chances for each team. He found that Canada outchanced the Soviets at even strength by a good margin over the series. But his scoring chances were based primarily on the area the shot was taken from. And there's a quote out there from a Soviet official after the Series saying something like "We had 340 attacks on the move and they had only 270" - showing that the Soviets valued puck movement in the attack as well as the place the attack was taken from. From what I remember, Canada got a lot of chances down low where a forward would beat the defenceman wide and get a shot and a rebound on Tretiak - but Tretiak would have the angle covered and didn't have to move. The Soviets had more shots off passes, and when they got Dryden moving pucks started going through him.

This is why, in part, goaltending is overrated in a lot of statistical analysis. I agree with you that PC assigns too much credit to goaltending. It doesn't pass the smell test, especially when you look at what NHL teams pay goaltenders. Should Ottawa sign Brian Elliott to a big contract based on his save percentage and PC? Not unless Elliott can bring Ken Hitchcock, the St. Louis forwards, and their relentless backchecking and strong possession play along with him.

Quote:
Originally Posted by Sturminator View Post
- Defensive Contribution does a poor job of seperating the actual defensive value of even strength partners. Two guys who skate together on the same pairing throughout the season are going to have nearly identical ES Defensive Contribution values in this metric, even though the eye test tells us that they may have widely different actual values in terms of preventing goals. Mr Ryder's metrics generally hate on Zdeno Chara, and this is one of the reasons why. It's also the reason why Brian Rafalski shows up as one of the best defensemen in the league when he is partnered with Nicklas Lidstrom.
Yes. Available data on outcomes can't separate partners. You'd basically have to design and track your own stats of good plays and bad plays to do so.

Quote:
Originally Posted by Sturminator View Post
- Defensive Contribution also cannot take into account strength of matchups when distributing credits. Personally, I think this is an overblown factor in actual performance in the post-lockout era where defensive matchups are much more difficult to dictate without resort to simply icing the puck to get a line change. But insofar as this factor is meaningful, it is not accounted for in the numbers.
I dunno. To take one example Tom Preissing looked pretty good as Ottawa's 6th defenceman in 06-07, when he got heavily sheltered/offensive minutes. He was +40, and to my eye he was very effective along the offensive blueline, getting shots on net, and pinching to hold the line. When he signed with LA, he was exposed on the second pairing when he had to play defence and win puck battles in his own zone. I think it still makes a big difference when a defenceman has to "step up the ladder" and play against tougher competition. But this is hard to measure statistically because good players tend to play against good players - you need to use multiple iterations instead of single iteration qualcomp numbers.

Quote:
Originally Posted by Sturminator View Post
- Finally, I think the value of goaltending is somewhat overrated in his analysis. It simply doesn't pass the smell test to me that, according to his analysis, the best players in the league year-after-year are goaltenders...and when I say the best I mean almost the entire top-20. Sorry, but Martin Gerber was not the 20th most valuable player in the league in 2007-08. This distortion (in my opinion) is another reason why Mr Ryder's metric hates on Zdeno Chara; it assigns such a huge portion of the defensive credit to Tim Thomas over the past few seasons that there is little left for Boston's defensemen.
Agreed.

overpass is offline   Reply With Quote
Old
03-31-2012, 03:07 PM
  #8
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
Quote:
Originally Posted by TheDevilMadeMe View Post
Oh, this is the famous metric that ranks Sergei Zubov first in 2006 because of shootout goals?

The use of shootout statistics on the metric is obviously a big issue post-expansion.
Yes, I agree that the shootout points are not a great measure of player skill/production, but from a strict Win Shares perspective, he does have it about right. Penalty shots only score about 25% of the time, I believe. Going 4-for-4 or whatever Zubov did in that season is going to bring your team probably 2-3 points in the standings above an average 1-for-4 performance.

But yes, feel free to ignore the points assigned for shootout performance. They make sense in terms of the analytic model (helping your team win), but not really from an overall player performance model that measures who are actually the best hockey players.

There's a good argument that Zubov's performance was quite underrated that season even not counting the shootouts.

Sturminator is offline   Reply With Quote
Old
04-01-2012, 12:04 AM
  #9
seventieslord
Registered User
 
seventieslord's Avatar
 
Join Date: Mar 2006
Location: Regina, SK
Country: Canada
Posts: 23,560
vCash: 500
I like this stuff, I like reading about it, I definitely use whatever pointed and undeniable results come out of it to tweak opinions I hold on players, but when it comes time to have a paragraphs-long conversation about it... I never feel that interested.

It just sort of bores me to talk about, even though I can get lost in a table of numbers.

Interesting to see that Jovanovski never got on any of those lists for PC.

Also, it's clear that it is giving way too much credit to bottom-pairing players who weren't as valuable as the guys who played ahead of them, even if these guys definitely made the most of the icetime they received. THere has to be a built in way, within this model, to recognize "oh, so JM Liles has a better score than Dion Phaneuf? That's what happens when the coach shelters him and gives Dion big minutes against the toughest opposition." But right now there isn't.

having done that study for all those "high GP" defensemen, I think I want to do something for the guys with fewer games now, something that is "apples to apples" for the likes of Keith, Weber, Suter, Jonsson, Phaneuf, - and then throw in a few "stragglers" that I included (somewhat unfairly to them) in the last comparison, like Svehla, Markov, Turnbull, Persson, Beck and Wilson.

What next? Something that attempts to sort out the modern defensive defensemen? Hatcher, Foote, Ramsey, Lowe, Huddy, Steve Smith, Dallas Smith, Hajt, Burrows, Tinordi, Van Impe, The Watsons, Plager, Ohlund, Seiling, Dupont, Svoboda, Langevin, Macoun, Wesley, Rochefort, Regehr, The Samuelssons, Dailey, Maxwell, Gregg, Gusarov - and you could definitely call Russell, Beck, Johansson, Numminen, Svehla, and maybe a couple others from the previous study, guys who "straggle" the line between offense and defense and could go in either study.

...or maybe not. sounds really difficult. TOI would be important for these guys but so is their actual defensive ability, and I don't think adjusted ESGA/GP would do them justice.

seventieslord is offline   Reply With Quote
Old
04-01-2012, 12:40 AM
  #10
nik jr
Registered User
 
Join Date: Sep 2005
Country: Congo-Kinshasa
Posts: 10,526
vCash: 500
Quote:
Originally Posted by overpass View Post
I think there are still a few things that Shot Quality doesn't pick up on.

Basically it needs to identify those shots that NHL coaches and teams want to prevent above all else, and separate them from those shots that they are willing to allow to some degree. The biggest thing it misses, IMO, is lateral movement before the shot. If the offensive team passes the puck laterally or even just has room to make a lateral move with the puck before shooting it makes it a lot more difficult for the goaltender to stop. There's no way to identify this statistically without tracking it during the games.

This is why, in part, goaltending is overrated in a lot of statistical analysis. I agree with you that PC assigns too much credit to goaltending. It doesn't pass the smell test, especially when you look at what NHL teams pay goaltenders. Should Ottawa sign Brian Elliott to a big contract based on his save percentage and PC? Not unless Elliott can bring Ken Hitchcock, the St. Louis forwards, and their relentless backchecking and strong possession play along with him.
agree, and especially about lateral movement

i have seen some posts on this site and on others that ES shot quality does not vary much at all between teams, especially over more than 1 season. but that seems completely wrong to my eyes.

nik jr is offline   Reply With Quote
Old
04-01-2012, 01:06 AM
  #11
Hockey Outsider
Registered User
 
Hockey Outsider's Avatar
 
Join Date: Jan 2005
Country: Canada
Posts: 3,342
vCash: 500
Quote:
Originally Posted by overpass View Post
This is why, in part, goaltending is overrated in a lot of statistical analysis. I agree with you that PC assigns too much credit to goaltending. It doesn't pass the smell test, especially when you look at what NHL teams pay goaltenders.
Although I agree with you point that statistical analysis often overrates goaltending, let me play devil's advocate.

Let's suppose I'm a GM and I have to decide how much to spend on goaltending. Let's also suppose that goaltending is exceedingly important, which is what most advanced statistical analysis suggests. As a GM, I'd still be hesitant to give a huge pay cheque to goalies simply because there's so much variability in their performance - in any season, average goalies can play at a Vezina level and star goalies can be surprisingly mediocre. That can happen with forwards and defensemen too, of course, but I feel that there's far more variability in goalies' performances. (If you disagree with this point, then you'll disagree with the rest of my argument).

Since there's more unpredictability, paying goalies becomes a riskier proposition, therefore their salaries are lower than we'd expect compared to their importance to the team. In other words: a goalie might really be worth $9 million per year, but I wouldn't pay them nearly that much if there's a good chance that they have an unexplained off-year and/or I can replace them with an AHL goaltending on a hot streak for a fraction of the price.

====

One issue with PC (or any purely statistical tool) is that it can't tell when a star player makes his linemate look better than he really is.

For example, in 2003, Hejduk is ranked the 2nd best forward in the NHL (behind Naslund). Forsberg is only 5th. It was obvious to anybody following the NHL that season (or, really, anybody who is knowledgeable about both players' careers) that Forsberg was the catalyst for the Avalanche that year and he made Hejduk look better than he really was. Of course, the calculation can't possibly "know" that Forsberg made Hejduk that year, but it's something that readers of the formula would need to take into account.

Cheechoo-Thornton in 2006 is an even more extreme example. Cheechoo is ranked 4th among forwards, while Thornton is only 12th! Again, it's obvious that Thornton made Cheechoo look better than he really was, even though the formula can't "know" that.

Another issue is that the system assumes that goals are worth more than assists (that's why Thornton '06 is ranked just 23rd among forwards in offensive points, despite winning the Art Ross!). I think that, generally, goals are worth more than assists, but for the truly gifted playmakers who consistently demonstrate the ability to elevate linemates (Thornton, Forsberg, Oates, Gretzky, Lemieux, etc), an assist is worth as much as the goal itself. Again, there's no way that a mathematical formula can "know" if the assist is Thornton taking a teammate to new heights, or Stu Grimson somehow lucking his way into a secondary assist. Thus, I think that PC consistently underrates elite playmakers.

Hockey Outsider is offline   Reply With Quote
Old
04-01-2012, 01:09 AM
  #12
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
Quote:
Originally Posted by nik jr View Post
agree, and especially about lateral movement

i have seen some posts on this site and on others that ES shot quality does not vary much at all between teams, especially over more than 1 season. but that seems completely wrong to my eyes.
Yes, the problem of lateral movement on shots is an excellent insight, and unfortunately something that is extremely difficult to account for the way stats are currently kept.

Sturminator is offline   Reply With Quote
Old
04-01-2012, 04:47 AM
  #13
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
Nice to see you here, HO. I had hoped that this thread might attract guys like you and BM.

Quote:
Originally Posted by Hockey Outsider View Post
Although I agree with you point that statistical analysis often overrates goaltending, let me play devil's advocate.

Let's suppose I'm a GM and I have to decide how much to spend on goaltending. Let's also suppose that goaltending is exceedingly important, which is what most advanced statistical analysis suggests. As a GM, I'd still be hesitant to give a huge pay cheque to goalies simply because there's so much variability in their performance - in any season, average goalies can play at a Vezina level and star goalies can be surprisingly mediocre. That can happen with forwards and defensemen too, of course, but I feel that there's far more variability in goalies' performances. (If you disagree with this point, then you'll disagree with the rest of my argument).

Since there's more unpredictability, paying goalies becomes a riskier proposition, therefore their salaries are lower than we'd expect compared to their importance to the team. In other words: a goalie might really be worth $9 million per year, but I wouldn't pay them nearly that much if there's a good chance that they have an unexplained off-year and/or I can replace them with an AHL goaltending on a hot streak for a fraction of the price.
I think that the main reason goalie performance swings so much from season to season is directly connected to the overrating of goalies in PC-type analysis. Simply put, the defense in front of a goalie has a big say in his ultimate statistics, completely irrespective of his own play and even beyond that which is captured in Shot Quality metrics.

To wit: I have long believed that the upper-crust of #1 defensemen are generally the most valuable players in the league at any given time, ceteris parabis (that is: barring über-dominant goalies or forwards).

Quote:
One issue with PC (or any purely statistical tool) is that it can't tell when a star player makes his linemate look better than he really is.
Yeah, this is a problem, as is the problem of assist value. Mr Ryder has tried, if not to statistically address, then to at least point out this problem on his site. He specifically broke down first assists during Thornton's Hart season and showed that Thornton not only led the league in assists, but was way out ahead in first assists. But obviously the model doesn't capture the value created here relative to players with more 2nd assists. It probably underrates high-end playmakers, yes, but then again, so does normal assist tabulation by the NHL.

Ultimately, these numbers must be informed by what we know of the relative contributions of players on the same unit, and strength of matchup information, insofar as it is relevant.

aside: I have been a critic of strength of matchup arguments for some time, especially as they apply to defensemen, not because I think matching up against bottom units is as valuable as doing the same against top units, but because I think that the frequency at which "soft minutes" occur for star players is a good deal lower than many of the arguments around here seem to assume, espcially in the post-lockout era. This is a particularly pernicious argument when it comes to modern offensive defensemen.

We have historical examples of guys like Housley and Gonchar who we know were weak defensively and were hidden at even strength a great deal of the time by their coaches. But of course, this shows up in their TOI numbers, so it's not that hard to perceive even without the anecdotal evidence. The problem is that The Housley Effect has grown to become more or less a cudgel which is used against all modern offensive defensemen regardless of the actual specifics of their individual situations. At this point, there is pretty much an assumption that if a modern defenseman is good offensively, then he must be playing sheltered minutes. This is utter tosh.

Housley and Gonchar are the most extreme end of the curve when it comes to defensemen. We should not go around judging all players who rememble them in any way stylistically by the standards we use to judge those two. Strength of matchups must obviously be accounted for in how we evaluate player performance, but it must be done so carefully and with some nuance. For some time, we have sadly been drifting back towards caveman hockey philosophies in the ATD.

Sturminator is offline   Reply With Quote
Old
04-01-2012, 08:57 AM
  #14
overpass
Registered User
 
Join Date: Jun 2007
Posts: 3,511
vCash: 500
Quote:
Originally Posted by Hockey Outsider View Post
Although I agree with you point that statistical analysis often overrates goaltending, let me play devil's advocate.

Let's suppose I'm a GM and I have to decide how much to spend on goaltending. Let's also suppose that goaltending is exceedingly important, which is what most advanced statistical analysis suggests. As a GM, I'd still be hesitant to give a huge pay cheque to goalies simply because there's so much variability in their performance - in any season, average goalies can play at a Vezina level and star goalies can be surprisingly mediocre. That can happen with forwards and defensemen too, of course, but I feel that there's far more variability in goalies' performances. (If you disagree with this point, then you'll disagree with the rest of my argument).

Since there's more unpredictability, paying goalies becomes a riskier proposition, therefore their salaries are lower than we'd expect compared to their importance to the team. In other words: a goalie might really be worth $9 million per year, but I wouldn't pay them nearly that much if there's a good chance that they have an unexplained off-year and/or I can replace them with an AHL goaltending on a hot streak for a fraction of the price.
Part of the reason there is more unpredictability in goalie "performance" is that there is a lot of teammate and opponent performance included in goalie performance, if you measure goalie performance by save percentage.

For example, a goalie gets a shutout. He'll be the first star for the night. And obviously he performed well. But the opponents probably did a bad job of making their shots - they hit his shoulder instead if the top corner. And his teammates probably did a good job of limiting high-quality chances. Similarly, when a goalie allows a lot of goals, it's usually the case that his opponents were making good shots and his teammates were allowing high quality opportunities.

Teammate and opponent performance will mostly even out in the long run, like goalie performance, but they are a major source of variance within save percentage in the short run.

Quote:
Originally Posted by Hockey Outsider View Post
One issue with PC (or any purely statistical tool) is that it can't tell when a star player makes his linemate look better than he really is.

For example, in 2003, Hejduk is ranked the 2nd best forward in the NHL (behind Naslund). Forsberg is only 5th. It was obvious to anybody following the NHL that season (or, really, anybody who is knowledgeable about both players' careers) that Forsberg was the catalyst for the Avalanche that year and he made Hejduk look better than he really was. Of course, the calculation can't possibly "know" that Forsberg made Hejduk that year, but it's something that readers of the formula would need to take into account.

Cheechoo-Thornton in 2006 is an even more extreme example. Cheechoo is ranked 4th among forwards, while Thornton is only 12th! Again, it's obvious that Thornton made Cheechoo look better than he really was, even though the formula can't "know" that.

Another issue is that the system assumes that goals are worth more than assists (that's why Thornton '06 is ranked just 23rd among forwards in offensive points, despite winning the Art Ross!). I think that, generally, goals are worth more than assists, but for the truly gifted playmakers who consistently demonstrate the ability to elevate linemates (Thornton, Forsberg, Oates, Gretzky, Lemieux, etc), an assist is worth as much as the goal itself. Again, there's no way that a mathematical formula can "know" if the assist is Thornton taking a teammate to new heights, or Stu Grimson somehow lucking his way into a secondary assist. Thus, I think that PC consistently underrates elite playmakers.
One problem here is that these uberstats are trying to be valid for different lengths of time.

For example, Hockey Prospectus's GVT is published and updated for each game during the season, and is also published for careers.

If you look at one goal in isolation where two assists are awarded, and you don't include any subjective information about how it was created, I think it makes sense to give the goal scorer the most credit and the second assist the least credit. That's probably how it works on average.

But when you look at a full season, the players who consistently get lots of assists should probably get a bit more credit for each assist on average - because it appears they have a skill. And when you look at full careers we can be even more confident that high assist players deserve lots of credit. So at that point I don't think it's correct to value goals higher than assists, especially when evaluating star players.

Basically it's a regression thing - we aren't confident that one assist demonstrates playmaking talent, and it might be luck, so we regress the value of that assist. But the more assists we see from the player, the more we can be confident that there is real playmaking talent there, and the less the value should be regressed.


Last edited by overpass: 04-01-2012 at 10:38 AM.
overpass is offline   Reply With Quote
Old
04-01-2012, 10:43 AM
  #15
overpass
Registered User
 
Join Date: Jun 2007
Posts: 3,511
vCash: 500
A thought about the value of not taking penalties. I haven't studied this because there isn't a good historical database separating out penalties that put the team down a man, but I suspect that, at least for defencemen, taking minor penalties correlates with preventing ES goals to some degree. So I'm not sold on giving full credit for not taking penalties until this is studied.

All else equal it's certainly valuable to avoid penalties. It would be interesting to estimate the value Lidstrom has provided over his career by not taking penalties, compared to, say, Pronger.

overpass is offline   Reply With Quote
Old
04-01-2012, 11:36 AM
  #16
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
Quote:
Originally Posted by overpass View Post
A thought about the value of not taking penalties. I haven't studied this because there isn't a good historical database separating out penalties that put the team down a man, but I suspect that, at least for defencemen, taking minor penalties correlates with preventing ES goals to some degree. So I'm not sold on giving full credit for not taking penalties until this is studied.

All else equal it's certainly valuable to avoid penalties. It would be interesting to estimate the value Lidstrom has provided over his career by not taking penalties, compared to, say, Pronger.
While I agree with you that taking minor penalties almost certainly correlates at some level with ES goal prevention, that goal prevention should already be captured in a PC style defensive value metric, as well as in an analysis of the type seventies did.

The defenseman takes the penalty and prevents a good scoring chance. His line stays clean in terms of ES goals against, but the team is now in jeopardy (roughly 20% chance, historically) of getting scored on by the opposing powerplay. If the team gives up a goal here, it only shows up on the penalty-taker's ledger in the form of 2 PIMs. So yes, I think players should be punished for taking penalties, and quite possibly at "full value" (ie. about 1/5th of a marginal goal allowed).

Sturminator is offline   Reply With Quote
Old
04-01-2012, 01:10 PM
  #17
Hawkman
Moderator
Deacon Blues
 
Hawkman's Avatar
 
Join Date: Dec 2007
Posts: 4,107
vCash: 500
Quote:
Originally Posted by overpass View Post
A thought about the value of not taking penalties. I haven't studied this because there isn't a good historical database separating out penalties that put the team down a man, but I suspect that, at least for defencemen, taking minor penalties correlates with preventing ES goals to some degree. So I'm not sold on giving full credit for not taking penalties until this is studied. All else equal it's certainly valuable to avoid penalties. It would be interesting to estimate the value Lidstrom has provided over his career by not taking penalties, compared to, say, Pronger.
Agreed. I've always thought Lidstrom's style is more beneficial than Pronger's or Stevens'. I want to say something about the value of the "big hit". I've seen guys on this board argue that if Stevens played today he would modify his hits to avoid suspensions. This doesn't make sense for 2 reasons. 1. You're assuming he can modify his personality and style of play and you don't know that is true. 2. I played hockey and if Stevens is not going to give me the "big hit", I'm not going to be intimidated and change my style of play and therefore he is going to be less effective.

Hawkman is offline   Reply With Quote
Old
04-01-2012, 01:53 PM
  #18
Hockey Outsider
Registered User
 
Hockey Outsider's Avatar
 
Join Date: Jan 2005
Country: Canada
Posts: 3,342
vCash: 500
Thanks - unfortunately I no longer have time to participate in the ATD but I still lurk here occasionally.

Good discussion about the value of goaltending - I agree that PC seems to allocate too much value to goaltending and that probably leaves too little for elite defensemen. Look at this comment from page 49 of the 2011 file:

"Chara remains the most over-hyped defenseman in the NHL – second in the Norris balloting but PC ranked him #34 with 60 PC."

Even worse, Chara is only ranked 41st in PCD (defensive contribution). The problem is that Thomas has 278 PC points (2nd in the league), which suggests that he contributed virtually the same amount of value as Corey Perry, Daniel Sedin and Alex Ovechkin combined! That can't be right. Since too many PC points are assigned to Thomas, there aren't enough left for Chara.

Keeping in mind the above disclaimer (i.e. I'm not completely sold on how PC evaluates defensemen), I find it interesting that the PC scores of top defensemen were very low in 2011 (which is consistent with the view that last season was a poor year for elite defensemen). Last year, no defenseman had 100 PC points; in every season from 2003 to 2010, 3-6 defensemen eclipsed the century mark. Last year's PC winners (Pietrangelo and Carlson, with 90 points each) would have ranked 12th, 6th, 10th, 11th, 8th, 6th and 8th over the past seven seasons.

Regarding penalties - everyone realizes that penalties are bad because teams are much more likely to allow goals against while on the penalty kill. However, not enough is said about the other cost of going on the PK - the significant reduction in the team's offense. Non-coincidental penalties are so dangerous because they increase the penalized team's likelihood of allowing a goal against, and it also effectively shuts down their offense for two minutes. Is this (the "offensive opportunity cost") captured in the PC formula?

Hockey Outsider is offline   Reply With Quote
Old
04-01-2012, 02:29 PM
  #19
overpass
Registered User
 
Join Date: Jun 2007
Posts: 3,511
vCash: 500
Quote:
Originally Posted by Sturminator View Post
While I agree with you that taking minor penalties almost certainly correlates at some level with ES goal prevention, that goal prevention should already be captured in a PC style defensive value metric, as well as in an analysis of the type seventies did.

The defenseman takes the penalty and prevents a good scoring chance. His line stays clean in terms of ES goals against, but the team is now in jeopardy (roughly 20% chance, historically) of getting scored on by the opposing powerplay. If the team gives up a goal here, it only shows up on the penalty-taker's ledger in the form of 2 PIMs. So yes, I think players should be punished for taking penalties, and quite possibly at "full value" (ie. about 1/5th of a marginal goal allowed).
Yeah, if the metric is giving full credit for ES goal prevention it should give full credit for the negative effects of the penalty as well.

overpass is offline   Reply With Quote
Old
04-02-2012, 03:25 AM
  #20
Sturminator
I voted for Kodos
 
Sturminator's Avatar
 
Join Date: Feb 2002
Location: West Egg, New York
Country: Ukraine
Posts: 7,386
vCash: 500
Quote:
Originally Posted by Hockey Outsider View Post
Regarding penalties - everyone realizes that penalties are bad because teams are much more likely to allow goals against while on the penalty kill. However, not enough is said about the other cost of going on the PK - the significant reduction in the team's offense. Non-coincidental penalties are so dangerous because they increase the penalized team's likelihood of allowing a goal against, and it also effectively shuts down their offense for two minutes. Is this (the "offensive opportunity cost") captured in the PC formula?
It probably is not. In general, one of the best insights of modern hockey analytic methods is that they show us clearly the high cost of taking penalties, but you're right that even this PC method may underestimate their true cost to the team.

A robust metric would probably subtract from the penalty-taker the difference between the marginal per minute GF/GA of the team as ES and on the PK for each minute that he puts the team down a man. Of course, this also leaves the "partner problem" in the equation. That is, if your partner takes a minor penalty to prevent a good ES scoring chance, it shows up in your ES goals prevented, but he is the only one penalized for it. Accounting for this factor (as for the partner problem, in general) is likely impossible under the current models.

Sturminator is offline   Reply With Quote
Old
04-06-2012, 02:11 PM
  #21
overpass
Registered User
 
Join Date: Jun 2007
Posts: 3,511
vCash: 500
Here's a look at minor penalties taken by Chris Pronger and Nicklas Lidstrom. Starting in 1997-98, when both became Norris contenders, through 2010-11.

I have left out major penalties because many of them are coincidental, and don't put the team down a man. Some of the minor penalties are probably coincidental as well, but I don't have a data source that separates those numbers.

Year Pronger GP Lidstrom GP Pronger MinP Lidstrom MinP Pronger ESGA Lidstrom ESGA
1998 81 80 65 9 55 51
1999 67 81 49 7 59 61
2000 79 81 41 9 43 70
2001 51 82 30 9 26 65
2002 78 78 45 10 50 55
2003 5 82 5 19 7 55
2004 80 81 39 9 50 51
2006 80 80 37 25 47 48
2007 66 80 32 23 27 35
2008 72 76 43 20 34 36
2009 82 78 39 15 56 49
2010 82 82 32 12 45 56
2011 50 82 21 10 33 65
Total 873 1043 478 177 532 698
Per-season 67 80 37 14 41 54
Per-82 82 82 45 14 50 55

MinP=minor penalty taken, ESGA= on-ice even strength goal against.

Pronger has been on the ice for fewer ES goals against than Lidstrom, even on a per-game basis, despite playing similar ice time. This is one reason I've been high on Pronger in the past. But it appears that Pronger takes penalties as a substitute for allowing goals, compared to Lidstrom. So his team ends up allowing goals on the power play instead.

It's hard to quantify the effect without having the coincidental penalties split out, but if Pronger allows 5 fewer ESGA than Lidstrom but takes 30 more minor penalties, that looks pretty similar.

Looking at Lidstrom's career season by season, there is a very high negative correlation (-0.72) between minor penalties taken and ESGA.This correlation would undoubtedly be of a lower magnitude if those numbers were normalized to league average, but it's still interesting.

overpass is offline   Reply With Quote
Old
04-06-2012, 02:15 PM
  #22
Jafar
Keep it logical
 
Jafar's Avatar
 
Join Date: Jul 2010
Location: Canada
Country: Canada
Posts: 8,831
vCash: 50
Maybe , but what are Pronger's minor penalties? His toughness and intimidation factor , as well as his unpredictability might play a role in some forward's mind.Just a thought.

Of course , it seems it doesn't change the EVGA stats , so I guess my point is invalid , but there's other things that might change in crucial moment of the game because of Pronger's intimidation factor.I would like to see the numbers for the playoffs.

Jafar is offline   Reply With Quote
Old
04-06-2012, 02:28 PM
  #23
overpass
Registered User
 
Join Date: Jun 2007
Posts: 3,511
vCash: 500
Quote:
Originally Posted by BenchBrawl View Post
Maybe , but what are Pronger's minor penalties? His toughness and intimidation factor , as well as his unpredictability might play a role in some forward's mind.Just a thought.

Of course , it seems it doesn't change the EVGA stats , so I guess my point is invalid , but there's other things that might change in crucial moment of the game because of Pronger's intimidation factor.I would like to see the numbers for the playoffs.
I'm sure Pronger's toughness and intimidation has its benefits. That's his intention, and his teammates and opponents seem to agree that it has value. But it should show up in the ESGA numbers already. It's possible that he throws players off their game so they are also affected when he is off the ice, I guess. I don't mean to write anything off, just trying to include as much as I can in the numerical analysis.

I'll look at the playoff numbers next.

overpass is offline   Reply With Quote
Old
04-06-2012, 02:38 PM
  #24
Jafar
Keep it logical
 
Jafar's Avatar
 
Join Date: Jul 2010
Location: Canada
Country: Canada
Posts: 8,831
vCash: 50
Quote:
Originally Posted by overpass View Post
I'm sure Pronger's toughness and intimidation has its benefits. That's his intention, and his teammates and opponents seem to agree that it has value. But it should show up in the ESGA numbers already. It's possible that he throws players off their game so they are also affected when he is off the ice, I guess. I don't mean to write anything off, just trying to include as much as I can in the numerical analysis.

I'll look at the playoff numbers next.
EVGA is great but hockey is so tough to analyze with statistics since it's a fluid team game.I think we are getting closer and closer though , and it is good work!

Is it possible Lidstrom's ES numbers are also affected positively by the fact he always had great defensive forwards playing in front of him , from Yzerman to Fedorov to Datsyuk to Zetterberg.Not only those four players were great defensively , they also were part of a great puck possession offensive unit normally , leaving less time in the defensive zone at EV for Detroit.If only we could have the ''Time on Attack'' and ''Time on Defense'' numbers , so many things would become clearer , but I don't think we do and I'm not even sure they do it to this day which is pathetic if it's the case.

Of course , Lidstrom's own play helping the transition is also in the equation as to how many time on attack/defense they spend , but forwards are also a big part of it.

Jafar is offline   Reply With Quote
Old
04-06-2012, 02:54 PM
  #25
overpass
Registered User
 
Join Date: Jun 2007
Posts: 3,511
vCash: 500
Lidstrom and Pronger - playoffs

Year Player GP ESGA MinP Player GP ESGA MinP
1998 Chris Pronger 10 8 13 Nicklas Lidstrom 22 14 4
1999 Chris Pronger 13 11 14 Nicklas Lidstrom 10 6 2
2000 Chris Pronger 7 6 11 Nicklas Lidstrom 9 9 2
2001 Chris Pronger 15 6 16 Nicklas Lidstrom 6 4 0
2002 Chris Pronger 9 3 7 Nicklas Lidstrom 23 15 1
2003 Chris Pronger 7 4 7 Nicklas Lidstrom 4 3 0
2004 Chris Pronger 5 1 8 Nicklas Lidstrom 12 6 2
2006 Chris Pronger 24 16 13 Nicklas Lidstrom 6 6 1
2007 Chris Pronger 19 8 13 Nicklas Lidstrom 18 9 3
2008 Chris Pronger 6 2 6 Nicklas Lidstrom 22 10 7
2009 Chris Pronger 13 8 6 Nicklas Lidstrom 21 10 3
2010 Chris Pronger 23 21 12 Nicklas Lidstrom 12 10 1
Total Chris Pronger 151 93 126 Nicklas Lidstrom 165 102 26
Per-playoff Chris Pronger 12.6 7.7 10.5 Nicklas Lidstrom 13.8 8.5 2.2
Per-82 Chris Pronger 82 50 68 Nicklas Lidstrom 82 51 13

In the playoffs, Pronger has taken even more penalties, and Lidstrom has taken fewer. There's an absolutely massive gap between them in the rate that they take minor penalties. (I would really like to see the coincidental penalties separated out, but don't feel like doing the work now.)

In the HOH top defencemen project, I posted the on-ice and off-ice ES goal ratio for several players in the playoffs, including Lidstrom and Pronger. Here are those numbers for the 1998-2010 playoffs.

Lidstrom - 1.28 R-ON, 1.32 R-OFF
Pronger - 1.50 R-ON, 0.87 R-OFF

It appears that Pronger has had a much larger impact on the ice at even strength...but the fact that Pronger will take 4-5 additional minor penalties in a 7 game series has to be weighted against that. Hard to quantify it exactly, as the on/off numbers have a lot of other influences and we don't have the coincidental penalties separated out.

Basically Pronger became more like Pronger in the playoffs.

overpass is offline   Reply With Quote
Reply

Forum Jump


Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -5. The time now is 07:17 AM.

monitoring_string = "e4251c93e2ba248d29da988d93bf5144"
Contact Us - HFBoards - Archive - Privacy Statement - Terms of Use - Advertise - Top - AdChoices

vBulletin Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
HFBoards.com is a property of CraveOnline Media, LLC, an Evolve Media, LLC company. ©2014 All Rights Reserved.