HFBoards Modern Hockey Analysis Tools
 Register FAQ Members List Search Today's Posts Mark Forums Read
 Mobile Hockey's Future Become a Sponsor Site Rules Support Forum vBookie
 Notices Please do not post or solicit links to illegal game streams.

 All Time Draft Fantasy league where players of the past and present meet.

# Modern Hockey Analysis Tools

 03-31-2012, 08:53 AM #1 Sturminator I voted for Kodos     Join Date: Feb 2002 Location: West Egg, New York Country: Posts: 8,612 vCash: 500 Modern Hockey Analysis Tools I'd like to open up a conversation on the value of modern hockey statistical analysis and the results thereof. This is a subject that I've been delving into a lot lately, and I think there is a lot of value in it, with a few caveats. I'll start by taking a look at my favorite of the modern analytical sites: http://hockeyanalytics.com/ The research behind this site got off the ground in 2004, and goes back to an analysis of the 2002-03 season as the beginning of its data. This page: http://hockeyanalytics.com/seasons/2003-season/ ...contains two of the most important essays on the foundational principles that this Alan Ryder guy uses in doing his analysis - specifically Shot Quality and Player Contribution. Shot Quality is an attempt to improve upon SV% based analysis of goalie performance by incorporating shots taken data from the NHL into the equation. The NHL generates data for each shot taken in the league - specifically who took it, from what distance, and what kind of shot it was. Mr. Ryder breaks down this data to determine which sorts of shot/distance combinations are likely to produce which sorts of results in order to arrive at a shot quality metric, which he then applies granularly to each goalie in the league as an adjustment to the base SV% statistic. And voila! A modern goaltending performance metric cometh. Player Performance is more complicated. It spins off of Bill James' Sabremetric-style Win Shares statistical analysis in baseball and applies it to the hockey rink. Specifically, he looks at Marginal Goals Created and Marginal Goals Prevented as his units of "credit" for each player in the NHL. Mr. Ryder breaks the game down into its component parts. - Each team gets a certain number of credits for performance above marginal (replacement) value, in terms of goals created and goals prevented. This is taken directly from the GF/GA statistics. - Goaltending is assigned 1/6th the total marginal value in the game, with all of that value coming on the defensive half, so generally speaking (this is a simplification) a total of 1/3rd of each team's total marginal Goals Prevented value, assuming a perfectly average goalie. - there is a "league wide constant" which states that 58% of the game is actually defense, and thus defensive value is weighted more heavily than offensive value in the calculations. - Offensive value is calculated and broken up per player (and per game situation) based on stats we all know well. - ...and what's left over (after we filter out the goalie and the offensive value in the game) is the defensive value. It is a top-down approach to arriving at defensive value, which I think is appropriate, as there are no clear statistics which could be used to drive a more direct statistical analysis. - from what's there of the team defensive value, the credits are then distributed among the relevant players based on position (defensemen get approximately 60%), an improvement on the +/- statistic (which I don't have the energy to discuss at this point) and Time On IceIt's a complicated system with a lot of moving parts and a lot of statistical analysis that will go over many people's heads (so be it), but I think in the end it has a lot of merit, and is an excellent start down the road to a more rational form of hockey statistical analysis. Nits that I would pick: - Mr. Ryder will admit to a few problems with the Shot Quality data. Specifically, the statkeepers in certain arenas may be high, but he has a built-in arena factor which seeks to remedy this issue. - Defensive Contribution does a poor job of seperating the actual defensive value of even strength partners. Two guys who skate together on the same pairing throughout the season are going to have nearly identical ES Defensive Contribution values in this metric, even though the eye test tells us that they may have widely different actual values in terms of preventing goals. Mr Ryder's metrics generally hate on Zdeno Chara, and this is one of the reasons why. It's also the reason why Brian Rafalski shows up as one of the best defensemen in the league when he is partnered with Nicklas Lidstrom. - Defensive Contribution also cannot take into account strength of matchups when distributing credits. Personally, I think this is an overblown factor in actual performance in the post-lockout era where defensive matchups are much more difficult to dictate without resort to simply icing the puck to get a line change. But insofar as this factor is meaningful, it is not accounted for in the numbers. - Finally, I think the value of goaltending is somewhat overrated in his analysis. It simply doesn't pass the smell test to me that, according to his analysis, the best players in the league year-after-year are goaltenders...and when I say the best I mean almost the entire top-20. Sorry, but Martin Gerber was not the 20th most valuable player in the league in 2007-08. This distortion (in my opinion) is another reason why Mr Ryder's metric hates on Zdeno Chara; it assigns such a huge portion of the defensive credit to Tim Thomas over the past few seasons that there is little left for Boston's defensemen. At any rate, Chara's standing in the Player Contribution tables is a persistent fly in this guy's ointment and goalie value has to be taken with a grain of salt, but otherwise, I think his analysis is highly enlightening. You can find links to his archived yearly NHL review pages (which have linked PDF summaries and Excel documents) here: http://hockeyanalytics.com/2004/03/player-contribution/ Happy reading.
03-31-2012, 02:14 PM
#2
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
Here's a sample of the results of hockeyanalytics' Player Contributions analysis...it's the top-20 defensemen from 2002-11 (PCO = Player Contribution Offense; PCD = Player Contribution Defense; PC = Total Player Contribution):

2002-03:

 Name PCO PCD PC Nicklas Lidstrom 53 71 124 Al MacInnis 56 58 114 Sergei Zubov 40 66 106 Sergei Gonchar 59 44 103 Scott Niedermayer 27 72 99 Dan Boyle 43 55 98 Eric Desjardins 24 73 97 Kim Johnsson 29 67 96 Wade Redden 33 63 96 Zdeno Chara 31 61 92 Tomas Kaberle 35 57 92 Brian Rafalski 22 68 90 Toni Lydman 11 73 84 Robert Svehla 31 50 81 Sami Salo 26 53 79 Rob Blake 34 45 79 Derian Hatcher 17 61 78 Alexander Khavanov 25 52 77 Scott Stevens 9 66 75 Filip Kuba 19 55 74

2003-04:

 Name PCO PCD PC Scott Niedermayer 43 71 114 Bryan McCabe 49 57 106 Chris Pronger 49 56 105 Kenny Jonsson 18 76 94 Nicklas Lidstrom 21 72 93 Adrian Aucoin 32 58 90 Sergei Zubov 25 65 90 Kim Johnsson 28 60 88 Mathieu Schneider 38 49 87 Wade Redden 34 53 87 Sami Salo 15 66 81 Zdeno Chara 35 44 79 Dick Tarnstrom 54 25 79 Philippe Boucher 17 61 78 Brent Sopel 31 45 76 Jordan Leopold 21 55 76 Mattias Ohlund 26 49 75 Tomas Kaberle 19 54 73 Kimmo Timonen 33 39 72 Dan Boyle 25 47 72

03-31-2012, 02:21 PM
#3
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
2005-06:

 Name PCO PCD PC Sergei Zubov 70 79 149 Nicklas Lidstrom 58 85 143 Chris Pronger 30 100 130 Tomas Kaberle 40 67 107 Dan Boyle 37 64 101 Lubomir Visnovsky 50 51 101 Mathieu Schneider 49 47 96 Tom Preissing 30 63 93 Brian Rafalski 20 73 93 Dion Phaneuf 40 49 89 Paul Martin 17 71 88 Joe Corvo 32 54 86 Marc-Andre Bergeron 24 60 84 Andrei Markov 33 50 83 Bryan McCabe 50 32 82 Philippe Boucher 35 46 81 Scott Niedermayer 37 38 75 Robyn Regehr 12 63 75 Steve Staios 18 55 73 Joni Pitkanen 40 33 73

2006-07:

 Name PCO PCD PC Nicklas Lidstrom 42 88 130 Sergei Zubov 46 67 113 Tomas Kaberle 42 68 110 Scott Niedermayer 49 52 101 Chris Pronger 39 58 97 Dan Boyle 46 47 93 Philippe Boucher 46 46 92 Brian Rafalski 33 59 92 Lubomir Visnovsky 46 45 91 Sami Salo 35 56 91 Ryan Whitney 41 44 85 Sheldon Souray 62 22 84 Ron Hainsey 17 65 82 Scott Hannan 11 71 82 Jay Bouwmeester 25 56 81 Brian Campbell 29 52 81 Mathieu Schneider 38 41 79 Bryan McCabe 39 40 79 Sergei Gonchar 40 38 78 Marc-Edouard Vlasic 7 69 76

03-31-2012, 02:30 PM
#4
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
2007-08:

 Name PCO PCD PC Nicklas Lidstrom 43 78 121 Brent Burns 51 56 107 Brian Rafalski 36 69 105 Zdeno Chara 55 42 97 Mike Green 53 44 97 Dion Phaneuf 60 33 93 Tomas Kaberle 35 58 93 Adrian Aucoin 23 65 88 Andrei Markov 49 34 83 Duncan Keith 23 60 83 Dennis Wideman 28 55 83 Sergei Gonchar 41 40 81 Marc-Edouard Vlasic -1 79 78 Ron Hainsey 12 66 78 Daniel Girardi 19 59 78 Mathieu Schneider 38 39 77 Paul Martin 15 61 76 Rostislav Klesla 2 72 74 Craig Rivet 19 56 75 Niklas Kronwall 24 50 74

2008-09:

 Name PCO PCD PC Mike Green 84 49 133 Nicklas Lidstrom 47 70 117 Brian Rafalski 39 63 102 Dan Boyle 51 51 102 Scott Niedermayer 37 55 92 Andrei Markov 53 36 89 Dennis Wideman 40 43 83 Marc-Edouard Vlasic 17 65 82 Shea Weber 44 38 82 Duncan Keith 30 49 79 Jan Hejda 12 65 77 Zdeno Chara 42 35 77 Chris Pronger 26 49 75 Joe Corvo 26 48 74 Paul Martin 21 52 73 Kimmo Timonen 26 46 72 Mark Streit 38 33 71 Sheldon Souray 41 30 71 Brian Campbell 28 42 70 Willie Mitchell 16 53 69

03-31-2012, 02:36 PM
#5
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
2009-10:

 Name PCO PCD PC Mike Green 71 33 104 Drew Doughty 49 55 104 Chris Pronger 32 70 102 Nicklas Lidstrom 33 67 100 Matt Carle 22 78 100 Duncan Keith 48 47 95 Dan Boyle 57 34 91 Chris Phillips 19 68 87 Tyler Myers 41 45 86 Andy Greene 19 67 86 Christian Ehrhoff 37 48 85 Brian Rafalski 28 55 83 Keith Yandle 33 49 82 Mark Streit 31 46 77 Shea Weber 36 42 78 Kimmo Timonen 20 53 73 Adrian Aucoin 29 43 72 Fedor Tyutin 17 55 72 Zdeno Chara 31 40 71 Brian Campbell 23 48 71

2010-11:

 Name PCO PCD PC Alex Pietrangelo 32 58 90 John Carlson 30 60 90 Brett Clark 19 70 89 Nicklas Lidstrom 43 44 87 Dan Boyle 46 36 82 Keith Yandle 41 40 81 Niklas Kronwall 28 52 80 Lubomir Visnovsky 54 25 80 Christian Ehrhoff 40 37 77 Brent Burns 41 34 75 Shea Weber 36 39 75 John-Michael Liles 33 41 74 Fedor Tyutin 16 58 74 Trevor Daley 17 56 73 Andy Greene 7 65 72 Dustin Byfuglien 55 17 72 Kris Letang 33 38 71 Mark Giordano 26 42 68 Erik Karlsson 38 30 68 Duncan Keith 22 46 68

 03-31-2012, 02:57 PM #6 TheDevilMadeMe Registered User     Join Date: Aug 2006 Location: Brooklyn Country: Posts: 45,927 vCash: 500 Oh, this is the famous metric that ranks Sergei Zubov first in 2006 because of shootout goals? The use of shootout statistics on the metric is obviously a big issue post-expansion.
03-31-2012, 02:59 PM
#7
overpass
Registered User

Join Date: Jun 2007
Posts: 3,978
vCash: 500
Quote:
 Originally Posted by Sturminator Nits that I would pick: - Mr. Ryder will admit to a few problems with the Shot Quality data. Specifically, the statkeepers in certain arenas may be high, but he has a built-in arena factor which seeks to remedy this issue.
I think there are still a few things that Shot Quality doesn't pick up on.

Basically it needs to identify those shots that NHL coaches and teams want to prevent above all else, and separate them from those shots that they are willing to allow to some degree. The biggest thing it misses, IMO, is lateral movement before the shot. If the offensive team passes the puck laterally or even just has room to make a lateral move with the puck before shooting it makes it a lot more difficult for the goaltender to stop. There's no way to identify this statistically without tracking it during the games.

The 1972 Summit Series is an interesting example of this. The blog Black Dog Hates Skunks did a rewatch of the 1972 Summit Series and counted scoring chances for each team. He found that Canada outchanced the Soviets at even strength by a good margin over the series. But his scoring chances were based primarily on the area the shot was taken from. And there's a quote out there from a Soviet official after the Series saying something like "We had 340 attacks on the move and they had only 270" - showing that the Soviets valued puck movement in the attack as well as the place the attack was taken from. From what I remember, Canada got a lot of chances down low where a forward would beat the defenceman wide and get a shot and a rebound on Tretiak - but Tretiak would have the angle covered and didn't have to move. The Soviets had more shots off passes, and when they got Dryden moving pucks started going through him.

This is why, in part, goaltending is overrated in a lot of statistical analysis. I agree with you that PC assigns too much credit to goaltending. It doesn't pass the smell test, especially when you look at what NHL teams pay goaltenders. Should Ottawa sign Brian Elliott to a big contract based on his save percentage and PC? Not unless Elliott can bring Ken Hitchcock, the St. Louis forwards, and their relentless backchecking and strong possession play along with him.

Quote:
 Originally Posted by Sturminator - Defensive Contribution does a poor job of seperating the actual defensive value of even strength partners. Two guys who skate together on the same pairing throughout the season are going to have nearly identical ES Defensive Contribution values in this metric, even though the eye test tells us that they may have widely different actual values in terms of preventing goals. Mr Ryder's metrics generally hate on Zdeno Chara, and this is one of the reasons why. It's also the reason why Brian Rafalski shows up as one of the best defensemen in the league when he is partnered with Nicklas Lidstrom.
Yes. Available data on outcomes can't separate partners. You'd basically have to design and track your own stats of good plays and bad plays to do so.

Quote:
 Originally Posted by Sturminator - Defensive Contribution also cannot take into account strength of matchups when distributing credits. Personally, I think this is an overblown factor in actual performance in the post-lockout era where defensive matchups are much more difficult to dictate without resort to simply icing the puck to get a line change. But insofar as this factor is meaningful, it is not accounted for in the numbers.
I dunno. To take one example Tom Preissing looked pretty good as Ottawa's 6th defenceman in 06-07, when he got heavily sheltered/offensive minutes. He was +40, and to my eye he was very effective along the offensive blueline, getting shots on net, and pinching to hold the line. When he signed with LA, he was exposed on the second pairing when he had to play defence and win puck battles in his own zone. I think it still makes a big difference when a defenceman has to "step up the ladder" and play against tougher competition. But this is hard to measure statistically because good players tend to play against good players - you need to use multiple iterations instead of single iteration qualcomp numbers.

Quote:
 Originally Posted by Sturminator - Finally, I think the value of goaltending is somewhat overrated in his analysis. It simply doesn't pass the smell test to me that, according to his analysis, the best players in the league year-after-year are goaltenders...and when I say the best I mean almost the entire top-20. Sorry, but Martin Gerber was not the 20th most valuable player in the league in 2007-08. This distortion (in my opinion) is another reason why Mr Ryder's metric hates on Zdeno Chara; it assigns such a huge portion of the defensive credit to Tim Thomas over the past few seasons that there is little left for Boston's defensemen.
Agreed.

03-31-2012, 03:07 PM
#8
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
Quote:
 Originally Posted by TheDevilMadeMe Oh, this is the famous metric that ranks Sergei Zubov first in 2006 because of shootout goals? The use of shootout statistics on the metric is obviously a big issue post-expansion.
Yes, I agree that the shootout points are not a great measure of player skill/production, but from a strict Win Shares perspective, he does have it about right. Penalty shots only score about 25% of the time, I believe. Going 4-for-4 or whatever Zubov did in that season is going to bring your team probably 2-3 points in the standings above an average 1-for-4 performance.

But yes, feel free to ignore the points assigned for shootout performance. They make sense in terms of the analytic model (helping your team win), but not really from an overall player performance model that measures who are actually the best hockey players.

There's a good argument that Zubov's performance was quite underrated that season even not counting the shootouts.

 04-01-2012, 12:04 AM #9 seventieslord Student Of The Game     Join Date: Mar 2006 Location: Regina, SK Country: Posts: 31,111 vCash: 500 I like this stuff, I like reading about it, I definitely use whatever pointed and undeniable results come out of it to tweak opinions I hold on players, but when it comes time to have a paragraphs-long conversation about it... I never feel that interested. It just sort of bores me to talk about, even though I can get lost in a table of numbers. Interesting to see that Jovanovski never got on any of those lists for PC. Also, it's clear that it is giving way too much credit to bottom-pairing players who weren't as valuable as the guys who played ahead of them, even if these guys definitely made the most of the icetime they received. THere has to be a built in way, within this model, to recognize "oh, so JM Liles has a better score than Dion Phaneuf? That's what happens when the coach shelters him and gives Dion big minutes against the toughest opposition." But right now there isn't. having done that study for all those "high GP" defensemen, I think I want to do something for the guys with fewer games now, something that is "apples to apples" for the likes of Keith, Weber, Suter, Jonsson, Phaneuf, - and then throw in a few "stragglers" that I included (somewhat unfairly to them) in the last comparison, like Svehla, Markov, Turnbull, Persson, Beck and Wilson. What next? Something that attempts to sort out the modern defensive defensemen? Hatcher, Foote, Ramsey, Lowe, Huddy, Steve Smith, Dallas Smith, Hajt, Burrows, Tinordi, Van Impe, The Watsons, Plager, Ohlund, Seiling, Dupont, Svoboda, Langevin, Macoun, Wesley, Rochefort, Regehr, The Samuelssons, Dailey, Maxwell, Gregg, Gusarov - and you could definitely call Russell, Beck, Johansson, Numminen, Svehla, and maybe a couple others from the previous study, guys who "straggle" the line between offense and defense and could go in either study. ...or maybe not. sounds really difficult. TOI would be important for these guys but so is their actual defensive ability, and I don't think adjusted ESGA/GP would do them justice.
04-01-2012, 12:40 AM
#10
nik jr
Registered User

Join Date: Sep 2005
Country:
Posts: 10,798
vCash: 500
Quote:
 Originally Posted by overpass I think there are still a few things that Shot Quality doesn't pick up on. Basically it needs to identify those shots that NHL coaches and teams want to prevent above all else, and separate them from those shots that they are willing to allow to some degree. The biggest thing it misses, IMO, is lateral movement before the shot. If the offensive team passes the puck laterally or even just has room to make a lateral move with the puck before shooting it makes it a lot more difficult for the goaltender to stop. There's no way to identify this statistically without tracking it during the games. This is why, in part, goaltending is overrated in a lot of statistical analysis. I agree with you that PC assigns too much credit to goaltending. It doesn't pass the smell test, especially when you look at what NHL teams pay goaltenders. Should Ottawa sign Brian Elliott to a big contract based on his save percentage and PC? Not unless Elliott can bring Ken Hitchcock, the St. Louis forwards, and their relentless backchecking and strong possession play along with him.
agree, and especially about lateral movement

i have seen some posts on this site and on others that ES shot quality does not vary much at all between teams, especially over more than 1 season. but that seems completely wrong to my eyes.

04-01-2012, 01:06 AM
#11
Hockey Outsider
Registered User

Join Date: Jan 2005
Country:
Posts: 4,135
vCash: 500
Quote:
 Originally Posted by overpass This is why, in part, goaltending is overrated in a lot of statistical analysis. I agree with you that PC assigns too much credit to goaltending. It doesn't pass the smell test, especially when you look at what NHL teams pay goaltenders.
Although I agree with you point that statistical analysis often overrates goaltending, let me play devil's advocate.

Let's suppose I'm a GM and I have to decide how much to spend on goaltending. Let's also suppose that goaltending is exceedingly important, which is what most advanced statistical analysis suggests. As a GM, I'd still be hesitant to give a huge pay cheque to goalies simply because there's so much variability in their performance - in any season, average goalies can play at a Vezina level and star goalies can be surprisingly mediocre. That can happen with forwards and defensemen too, of course, but I feel that there's far more variability in goalies' performances. (If you disagree with this point, then you'll disagree with the rest of my argument).

Since there's more unpredictability, paying goalies becomes a riskier proposition, therefore their salaries are lower than we'd expect compared to their importance to the team. In other words: a goalie might really be worth \$9 million per year, but I wouldn't pay them nearly that much if there's a good chance that they have an unexplained off-year and/or I can replace them with an AHL goaltending on a hot streak for a fraction of the price.

====

One issue with PC (or any purely statistical tool) is that it can't tell when a star player makes his linemate look better than he really is.

For example, in 2003, Hejduk is ranked the 2nd best forward in the NHL (behind Naslund). Forsberg is only 5th. It was obvious to anybody following the NHL that season (or, really, anybody who is knowledgeable about both players' careers) that Forsberg was the catalyst for the Avalanche that year and he made Hejduk look better than he really was. Of course, the calculation can't possibly "know" that Forsberg made Hejduk that year, but it's something that readers of the formula would need to take into account.

Cheechoo-Thornton in 2006 is an even more extreme example. Cheechoo is ranked 4th among forwards, while Thornton is only 12th! Again, it's obvious that Thornton made Cheechoo look better than he really was, even though the formula can't "know" that.

Another issue is that the system assumes that goals are worth more than assists (that's why Thornton '06 is ranked just 23rd among forwards in offensive points, despite winning the Art Ross!). I think that, generally, goals are worth more than assists, but for the truly gifted playmakers who consistently demonstrate the ability to elevate linemates (Thornton, Forsberg, Oates, Gretzky, Lemieux, etc), an assist is worth as much as the goal itself. Again, there's no way that a mathematical formula can "know" if the assist is Thornton taking a teammate to new heights, or Stu Grimson somehow lucking his way into a secondary assist. Thus, I think that PC consistently underrates elite playmakers.

04-01-2012, 01:09 AM
#12
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
Quote:
 Originally Posted by nik jr agree, and especially about lateral movement i have seen some posts on this site and on others that ES shot quality does not vary much at all between teams, especially over more than 1 season. but that seems completely wrong to my eyes.
Yes, the problem of lateral movement on shots is an excellent insight, and unfortunately something that is extremely difficult to account for the way stats are currently kept.

04-01-2012, 04:47 AM
#13
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
Nice to see you here, HO. I had hoped that this thread might attract guys like you and BM.

Quote:
 Originally Posted by Hockey Outsider Although I agree with you point that statistical analysis often overrates goaltending, let me play devil's advocate. Let's suppose I'm a GM and I have to decide how much to spend on goaltending. Let's also suppose that goaltending is exceedingly important, which is what most advanced statistical analysis suggests. As a GM, I'd still be hesitant to give a huge pay cheque to goalies simply because there's so much variability in their performance - in any season, average goalies can play at a Vezina level and star goalies can be surprisingly mediocre. That can happen with forwards and defensemen too, of course, but I feel that there's far more variability in goalies' performances. (If you disagree with this point, then you'll disagree with the rest of my argument). Since there's more unpredictability, paying goalies becomes a riskier proposition, therefore their salaries are lower than we'd expect compared to their importance to the team. In other words: a goalie might really be worth \$9 million per year, but I wouldn't pay them nearly that much if there's a good chance that they have an unexplained off-year and/or I can replace them with an AHL goaltending on a hot streak for a fraction of the price.
I think that the main reason goalie performance swings so much from season to season is directly connected to the overrating of goalies in PC-type analysis. Simply put, the defense in front of a goalie has a big say in his ultimate statistics, completely irrespective of his own play and even beyond that which is captured in Shot Quality metrics.

To wit: I have long believed that the upper-crust of #1 defensemen are generally the most valuable players in the league at any given time, ceteris parabis (that is: barring über-dominant goalies or forwards).

Quote:
 One issue with PC (or any purely statistical tool) is that it can't tell when a star player makes his linemate look better than he really is.
Yeah, this is a problem, as is the problem of assist value. Mr Ryder has tried, if not to statistically address, then to at least point out this problem on his site. He specifically broke down first assists during Thornton's Hart season and showed that Thornton not only led the league in assists, but was way out ahead in first assists. But obviously the model doesn't capture the value created here relative to players with more 2nd assists. It probably underrates high-end playmakers, yes, but then again, so does normal assist tabulation by the NHL.

Ultimately, these numbers must be informed by what we know of the relative contributions of players on the same unit, and strength of matchup information, insofar as it is relevant.

aside: I have been a critic of strength of matchup arguments for some time, especially as they apply to defensemen, not because I think matching up against bottom units is as valuable as doing the same against top units, but because I think that the frequency at which "soft minutes" occur for star players is a good deal lower than many of the arguments around here seem to assume, espcially in the post-lockout era. This is a particularly pernicious argument when it comes to modern offensive defensemen.

We have historical examples of guys like Housley and Gonchar who we know were weak defensively and were hidden at even strength a great deal of the time by their coaches. But of course, this shows up in their TOI numbers, so it's not that hard to perceive even without the anecdotal evidence. The problem is that The Housley Effect has grown to become more or less a cudgel which is used against all modern offensive defensemen regardless of the actual specifics of their individual situations. At this point, there is pretty much an assumption that if a modern defenseman is good offensively, then he must be playing sheltered minutes. This is utter tosh.

Housley and Gonchar are the most extreme end of the curve when it comes to defensemen. We should not go around judging all players who rememble them in any way stylistically by the standards we use to judge those two. Strength of matchups must obviously be accounted for in how we evaluate player performance, but it must be done so carefully and with some nuance. For some time, we have sadly been drifting back towards caveman hockey philosophies in the ATD.

04-01-2012, 08:57 AM
#14
overpass
Registered User

Join Date: Jun 2007
Posts: 3,978
vCash: 500
Quote:
 Originally Posted by Hockey Outsider Although I agree with you point that statistical analysis often overrates goaltending, let me play devil's advocate. Let's suppose I'm a GM and I have to decide how much to spend on goaltending. Let's also suppose that goaltending is exceedingly important, which is what most advanced statistical analysis suggests. As a GM, I'd still be hesitant to give a huge pay cheque to goalies simply because there's so much variability in their performance - in any season, average goalies can play at a Vezina level and star goalies can be surprisingly mediocre. That can happen with forwards and defensemen too, of course, but I feel that there's far more variability in goalies' performances. (If you disagree with this point, then you'll disagree with the rest of my argument). Since there's more unpredictability, paying goalies becomes a riskier proposition, therefore their salaries are lower than we'd expect compared to their importance to the team. In other words: a goalie might really be worth \$9 million per year, but I wouldn't pay them nearly that much if there's a good chance that they have an unexplained off-year and/or I can replace them with an AHL goaltending on a hot streak for a fraction of the price.
Part of the reason there is more unpredictability in goalie "performance" is that there is a lot of teammate and opponent performance included in goalie performance, if you measure goalie performance by save percentage.

For example, a goalie gets a shutout. He'll be the first star for the night. And obviously he performed well. But the opponents probably did a bad job of making their shots - they hit his shoulder instead if the top corner. And his teammates probably did a good job of limiting high-quality chances. Similarly, when a goalie allows a lot of goals, it's usually the case that his opponents were making good shots and his teammates were allowing high quality opportunities.

Teammate and opponent performance will mostly even out in the long run, like goalie performance, but they are a major source of variance within save percentage in the short run.

Quote:
 Originally Posted by Hockey Outsider One issue with PC (or any purely statistical tool) is that it can't tell when a star player makes his linemate look better than he really is. For example, in 2003, Hejduk is ranked the 2nd best forward in the NHL (behind Naslund). Forsberg is only 5th. It was obvious to anybody following the NHL that season (or, really, anybody who is knowledgeable about both players' careers) that Forsberg was the catalyst for the Avalanche that year and he made Hejduk look better than he really was. Of course, the calculation can't possibly "know" that Forsberg made Hejduk that year, but it's something that readers of the formula would need to take into account. Cheechoo-Thornton in 2006 is an even more extreme example. Cheechoo is ranked 4th among forwards, while Thornton is only 12th! Again, it's obvious that Thornton made Cheechoo look better than he really was, even though the formula can't "know" that. Another issue is that the system assumes that goals are worth more than assists (that's why Thornton '06 is ranked just 23rd among forwards in offensive points, despite winning the Art Ross!). I think that, generally, goals are worth more than assists, but for the truly gifted playmakers who consistently demonstrate the ability to elevate linemates (Thornton, Forsberg, Oates, Gretzky, Lemieux, etc), an assist is worth as much as the goal itself. Again, there's no way that a mathematical formula can "know" if the assist is Thornton taking a teammate to new heights, or Stu Grimson somehow lucking his way into a secondary assist. Thus, I think that PC consistently underrates elite playmakers.
One problem here is that these uberstats are trying to be valid for different lengths of time.

For example, Hockey Prospectus's GVT is published and updated for each game during the season, and is also published for careers.

If you look at one goal in isolation where two assists are awarded, and you don't include any subjective information about how it was created, I think it makes sense to give the goal scorer the most credit and the second assist the least credit. That's probably how it works on average.

But when you look at a full season, the players who consistently get lots of assists should probably get a bit more credit for each assist on average - because it appears they have a skill. And when you look at full careers we can be even more confident that high assist players deserve lots of credit. So at that point I don't think it's correct to value goals higher than assists, especially when evaluating star players.

Basically it's a regression thing - we aren't confident that one assist demonstrates playmaking talent, and it might be luck, so we regress the value of that assist. But the more assists we see from the player, the more we can be confident that there is real playmaking talent there, and the less the value should be regressed.

Last edited by overpass: 04-01-2012 at 10:38 AM.

 04-01-2012, 10:43 AM #15 overpass Registered User   Join Date: Jun 2007 Posts: 3,978 vCash: 500 A thought about the value of not taking penalties. I haven't studied this because there isn't a good historical database separating out penalties that put the team down a man, but I suspect that, at least for defencemen, taking minor penalties correlates with preventing ES goals to some degree. So I'm not sold on giving full credit for not taking penalties until this is studied. All else equal it's certainly valuable to avoid penalties. It would be interesting to estimate the value Lidstrom has provided over his career by not taking penalties, compared to, say, Pronger.
04-01-2012, 11:36 AM
#16
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
Quote:
 Originally Posted by overpass A thought about the value of not taking penalties. I haven't studied this because there isn't a good historical database separating out penalties that put the team down a man, but I suspect that, at least for defencemen, taking minor penalties correlates with preventing ES goals to some degree. So I'm not sold on giving full credit for not taking penalties until this is studied. All else equal it's certainly valuable to avoid penalties. It would be interesting to estimate the value Lidstrom has provided over his career by not taking penalties, compared to, say, Pronger.
While I agree with you that taking minor penalties almost certainly correlates at some level with ES goal prevention, that goal prevention should already be captured in a PC style defensive value metric, as well as in an analysis of the type seventies did.

The defenseman takes the penalty and prevents a good scoring chance. His line stays clean in terms of ES goals against, but the team is now in jeopardy (roughly 20% chance, historically) of getting scored on by the opposing powerplay. If the team gives up a goal here, it only shows up on the penalty-taker's ledger in the form of 2 PIMs. So yes, I think players should be punished for taking penalties, and quite possibly at "full value" (ie. about 1/5th of a marginal goal allowed).

04-01-2012, 01:10 PM
#17
Wrigley
2016 World Champs

Join Date: Dec 2007
Posts: 9,215
vCash: 3495
Quote:
 Originally Posted by overpass A thought about the value of not taking penalties. I haven't studied this because there isn't a good historical database separating out penalties that put the team down a man, but I suspect that, at least for defencemen, taking minor penalties correlates with preventing ES goals to some degree. So I'm not sold on giving full credit for not taking penalties until this is studied. All else equal it's certainly valuable to avoid penalties. It would be interesting to estimate the value Lidstrom has provided over his career by not taking penalties, compared to, say, Pronger.
Agreed. I've always thought Lidstrom's style is more beneficial than Pronger's or Stevens'. I want to say something about the value of the "big hit". I've seen guys on this board argue that if Stevens played today he would modify his hits to avoid suspensions. This doesn't make sense for 2 reasons. 1. You're assuming he can modify his personality and style of play and you don't know that is true. 2. I played hockey and if Stevens is not going to give me the "big hit", I'm not going to be intimidated and change my style of play and therefore he is going to be less effective.

 04-01-2012, 01:53 PM #18 Hockey Outsider Registered User     Join Date: Jan 2005 Country: Posts: 4,135 vCash: 500 Thanks - unfortunately I no longer have time to participate in the ATD but I still lurk here occasionally. Good discussion about the value of goaltending - I agree that PC seems to allocate too much value to goaltending and that probably leaves too little for elite defensemen. Look at this comment from page 49 of the 2011 file: "Chara remains the most over-hyped defenseman in the NHL – second in the Norris balloting but PC ranked him #34 with 60 PC." Even worse, Chara is only ranked 41st in PCD (defensive contribution). The problem is that Thomas has 278 PC points (2nd in the league), which suggests that he contributed virtually the same amount of value as Corey Perry, Daniel Sedin and Alex Ovechkin combined! That can't be right. Since too many PC points are assigned to Thomas, there aren't enough left for Chara. Keeping in mind the above disclaimer (i.e. I'm not completely sold on how PC evaluates defensemen), I find it interesting that the PC scores of top defensemen were very low in 2011 (which is consistent with the view that last season was a poor year for elite defensemen). Last year, no defenseman had 100 PC points; in every season from 2003 to 2010, 3-6 defensemen eclipsed the century mark. Last year's PC winners (Pietrangelo and Carlson, with 90 points each) would have ranked 12th, 6th, 10th, 11th, 8th, 6th and 8th over the past seven seasons. Regarding penalties - everyone realizes that penalties are bad because teams are much more likely to allow goals against while on the penalty kill. However, not enough is said about the other cost of going on the PK - the significant reduction in the team's offense. Non-coincidental penalties are so dangerous because they increase the penalized team's likelihood of allowing a goal against, and it also effectively shuts down their offense for two minutes. Is this (the "offensive opportunity cost") captured in the PC formula?
04-01-2012, 02:29 PM
#19
overpass
Registered User

Join Date: Jun 2007
Posts: 3,978
vCash: 500
Quote:
 Originally Posted by Sturminator While I agree with you that taking minor penalties almost certainly correlates at some level with ES goal prevention, that goal prevention should already be captured in a PC style defensive value metric, as well as in an analysis of the type seventies did. The defenseman takes the penalty and prevents a good scoring chance. His line stays clean in terms of ES goals against, but the team is now in jeopardy (roughly 20% chance, historically) of getting scored on by the opposing powerplay. If the team gives up a goal here, it only shows up on the penalty-taker's ledger in the form of 2 PIMs. So yes, I think players should be punished for taking penalties, and quite possibly at "full value" (ie. about 1/5th of a marginal goal allowed).
Yeah, if the metric is giving full credit for ES goal prevention it should give full credit for the negative effects of the penalty as well.

04-02-2012, 03:25 AM
#20
Sturminator
I voted for Kodos

Join Date: Feb 2002
Location: West Egg, New York
Country:
Posts: 8,612
vCash: 500
Quote:
 Originally Posted by Hockey Outsider Regarding penalties - everyone realizes that penalties are bad because teams are much more likely to allow goals against while on the penalty kill. However, not enough is said about the other cost of going on the PK - the significant reduction in the team's offense. Non-coincidental penalties are so dangerous because they increase the penalized team's likelihood of allowing a goal against, and it also effectively shuts down their offense for two minutes. Is this (the "offensive opportunity cost") captured in the PC formula?
It probably is not. In general, one of the best insights of modern hockey analytic methods is that they show us clearly the high cost of taking penalties, but you're right that even this PC method may underestimate their true cost to the team.

A robust metric would probably subtract from the penalty-taker the difference between the marginal per minute GF/GA of the team as ES and on the PK for each minute that he puts the team down a man. Of course, this also leaves the "partner problem" in the equation. That is, if your partner takes a minor penalty to prevent a good ES scoring chance, it shows up in your ES goals prevented, but he is the only one penalized for it. Accounting for this factor (as for the partner problem, in general) is likely impossible under the current models.

04-06-2012, 02:11 PM
#21
overpass
Registered User

Join Date: Jun 2007
Posts: 3,978
vCash: 500
Here's a look at minor penalties taken by Chris Pronger and Nicklas Lidstrom. Starting in 1997-98, when both became Norris contenders, through 2010-11.

I have left out major penalties because many of them are coincidental, and don't put the team down a man. Some of the minor penalties are probably coincidental as well, but I don't have a data source that separates those numbers.

 Year Pronger GP Lidstrom GP Pronger MinP Lidstrom MinP Pronger ESGA Lidstrom ESGA 1998 81 80 65 9 55 51 1999 67 81 49 7 59 61 2000 79 81 41 9 43 70 2001 51 82 30 9 26 65 2002 78 78 45 10 50 55 2003 5 82 5 19 7 55 2004 80 81 39 9 50 51 2006 80 80 37 25 47 48 2007 66 80 32 23 27 35 2008 72 76 43 20 34 36 2009 82 78 39 15 56 49 2010 82 82 32 12 45 56 2011 50 82 21 10 33 65 Total 873 1043 478 177 532 698 Per-season 67 80 37 14 41 54 Per-82 82 82 45 14 50 55

MinP=minor penalty taken, ESGA= on-ice even strength goal against.

Pronger has been on the ice for fewer ES goals against than Lidstrom, even on a per-game basis, despite playing similar ice time. This is one reason I've been high on Pronger in the past. But it appears that Pronger takes penalties as a substitute for allowing goals, compared to Lidstrom. So his team ends up allowing goals on the power play instead.

It's hard to quantify the effect without having the coincidental penalties split out, but if Pronger allows 5 fewer ESGA than Lidstrom but takes 30 more minor penalties, that looks pretty similar.

Looking at Lidstrom's career season by season, there is a very high negative correlation (-0.72) between minor penalties taken and ESGA.This correlation would undoubtedly be of a lower magnitude if those numbers were normalized to league average, but it's still interesting.

 04-06-2012, 02:15 PM #22 BenchBrawl joueur de hockey     Join Date: Jul 2010 Posts: 15,484 vCash: 800 Maybe , but what are Pronger's minor penalties? His toughness and intimidation factor , as well as his unpredictability might play a role in some forward's mind.Just a thought. Of course , it seems it doesn't change the EVGA stats , so I guess my point is invalid , but there's other things that might change in crucial moment of the game because of Pronger's intimidation factor.I would like to see the numbers for the playoffs.
04-06-2012, 02:28 PM
#23
overpass
Registered User

Join Date: Jun 2007
Posts: 3,978
vCash: 500
Quote:
 Originally Posted by BenchBrawl Maybe , but what are Pronger's minor penalties? His toughness and intimidation factor , as well as his unpredictability might play a role in some forward's mind.Just a thought. Of course , it seems it doesn't change the EVGA stats , so I guess my point is invalid , but there's other things that might change in crucial moment of the game because of Pronger's intimidation factor.I would like to see the numbers for the playoffs.
I'm sure Pronger's toughness and intimidation has its benefits. That's his intention, and his teammates and opponents seem to agree that it has value. But it should show up in the ESGA numbers already. It's possible that he throws players off their game so they are also affected when he is off the ice, I guess. I don't mean to write anything off, just trying to include as much as I can in the numerical analysis.

I'll look at the playoff numbers next.

04-06-2012, 02:38 PM
#24
BenchBrawl
joueur de hockey

Join Date: Jul 2010
Posts: 15,484
vCash: 800
Quote:
 Originally Posted by overpass I'm sure Pronger's toughness and intimidation has its benefits. That's his intention, and his teammates and opponents seem to agree that it has value. But it should show up in the ESGA numbers already. It's possible that he throws players off their game so they are also affected when he is off the ice, I guess. I don't mean to write anything off, just trying to include as much as I can in the numerical analysis. I'll look at the playoff numbers next.
EVGA is great but hockey is so tough to analyze with statistics since it's a fluid team game.I think we are getting closer and closer though , and it is good work!

Is it possible Lidstrom's ES numbers are also affected positively by the fact he always had great defensive forwards playing in front of him , from Yzerman to Fedorov to Datsyuk to Zetterberg.Not only those four players were great defensively , they also were part of a great puck possession offensive unit normally , leaving less time in the defensive zone at EV for Detroit.If only we could have the ''Time on Attack'' and ''Time on Defense'' numbers , so many things would become clearer , but I don't think we do and I'm not even sure they do it to this day which is pathetic if it's the case.

Of course , Lidstrom's own play helping the transition is also in the equation as to how many time on attack/defense they spend , but forwards are also a big part of it.

04-06-2012, 02:54 PM
#25
overpass
Registered User

Join Date: Jun 2007
Posts: 3,978
vCash: 500
Lidstrom and Pronger - playoffs

 Year Player GP ESGA MinP Player GP ESGA MinP 1998 Chris Pronger 10 8 13 Nicklas Lidstrom 22 14 4 1999 Chris Pronger 13 11 14 Nicklas Lidstrom 10 6 2 2000 Chris Pronger 7 6 11 Nicklas Lidstrom 9 9 2 2001 Chris Pronger 15 6 16 Nicklas Lidstrom 6 4 0 2002 Chris Pronger 9 3 7 Nicklas Lidstrom 23 15 1 2003 Chris Pronger 7 4 7 Nicklas Lidstrom 4 3 0 2004 Chris Pronger 5 1 8 Nicklas Lidstrom 12 6 2 2006 Chris Pronger 24 16 13 Nicklas Lidstrom 6 6 1 2007 Chris Pronger 19 8 13 Nicklas Lidstrom 18 9 3 2008 Chris Pronger 6 2 6 Nicklas Lidstrom 22 10 7 2009 Chris Pronger 13 8 6 Nicklas Lidstrom 21 10 3 2010 Chris Pronger 23 21 12 Nicklas Lidstrom 12 10 1 Total Chris Pronger 151 93 126 Nicklas Lidstrom 165 102 26 Per-playoff Chris Pronger 12.6 7.7 10.5 Nicklas Lidstrom 13.8 8.5 2.2 Per-82 Chris Pronger 82 50 68 Nicklas Lidstrom 82 51 13

In the playoffs, Pronger has taken even more penalties, and Lidstrom has taken fewer. There's an absolutely massive gap between them in the rate that they take minor penalties. (I would really like to see the coincidental penalties separated out, but don't feel like doing the work now.)

In the HOH top defencemen project, I posted the on-ice and off-ice ES goal ratio for several players in the playoffs, including Lidstrom and Pronger. Here are those numbers for the 1998-2010 playoffs.

Lidstrom - 1.28 R-ON, 1.32 R-OFF
Pronger - 1.50 R-ON, 0.87 R-OFF

It appears that Pronger has had a much larger impact on the ice at even strength...but the fact that Pronger will take 4-5 additional minor penalties in a 7 game series has to be weighted against that. Hard to quantify it exactly, as the on/off numbers have a lot of other influences and we don't have the coincidental penalties separated out.

Basically Pronger became more like Pronger in the playoffs.

Forum Jump