HFBoards

Go Back   HFBoards > General Hockey Discussion > By The Numbers
Mobile Hockey's Future Become a Sponsor Site Rules Support Forum vBookie Page 2
By The Numbers Hockey Analytics... the Final Frontier. Explore strange new worlds, to seek out new algorithms, to boldly go where no one has gone before.

Excellent New Goalie Stat (Save Percentage Above League Average)

Reply
 
Thread Tools
Old
10-10-2013, 12:04 PM
  #1
zeke
#freewilly
 
zeke's Avatar
 
Join Date: Mar 2005
Posts: 30,218
vCash: 500
Excellent New Goalie Stat (Save Percentage Above League Average)

Thought this was good enough and relevant enough for more than just the stats forum.

IMO, this should immediately replace raw sv% in all goalie conversations.

One of the fundamental problems with stats is that league norms change over the years, and even from year to year. So its usually a good idea to try and adjust stats based on that year's league averages.

In baseball, they use OPS+ to adjust OPS to league average (where a 100ops+ means league average).

Hockey-reference has decided to do the same with SV%.....so we now have SV%+....a stat that adjusts sv% to the league average sv% every year.

http://www.hockey-reference.com/leag...html#stats::15

This is an immediate upgrade on sv% in all contexts, so go ahead and use it as often as possible.


For Leafs fans, for example, this can tell us that Reimer's club-record .924sv% last year (115sv%+) was actually not quite as impressive as Belfour's .922sv% (117sv%+) in '03.

zeke is online now   Reply With Quote
Old
10-10-2013, 12:11 PM
  #2
HarrisonFord
Lost a bet
 
HarrisonFord's Avatar
 
Join Date: Jul 2011
Location: Toronto
Country: Canada
Posts: 9,926
vCash: 500
So 1.15SV+ = 15% above average?

HarrisonFord is offline   Reply With Quote
Old
10-10-2013, 12:13 PM
  #3
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
I've been doing exactly this for quite some time now. For instance:

http://hockeygoalies.org/bio/nhl/toronto.html
(scroll down to where save percentage has been calculated)

Z-Score tells how many standard deviations above (or below) league average a goaltender's performance was during the season.

GD tells how many goals a goaltender prevented during the season, above and beyond the league average (looking at the link above, it appears that H-R is doing this now, calling it GSAA).

GAR is the same as above, but comparing against a "replacement-level" goaltender (since GD would assign a value of zero to a league-average goaltender, but there's definite value in being average).

SNW% gives what the goaltender's winning percentage would be if he were on a team that (a) faces a league-average number of shots, and (b) scores a league-average number of goals. (SNW and SNL are then a goaltender's support-neutral wins and losses).

For more recent seasons, I also calculate a goaltender's game-to-game variance (as a measure of consistency) as well as the strength of schedule faced by the goaltender.

I describe these all in a bit more detail here: http://hockeygoalies.org/stats/glossary.html

Doctor No is offline   Reply With Quote
Old
10-10-2013, 12:13 PM
  #4
Ohashi_Jouzu
Registered User
 
Ohashi_Jouzu's Avatar
 
Join Date: Apr 2007
Location: Halifax
Country: Japan
Posts: 22,898
vCash: 500
Yeah, I noticed that the other day. On the history board it's common practice to do the leg work one's self and use this in conversation (goalies are routinely compared across eras over there, so you pretty much have to), but it's nice to have an online stats database doing the grunt work for us now - for everyone at once.

Ohashi_Jouzu is offline   Reply With Quote
Old
10-10-2013, 12:14 PM
  #5
billybudd
5 Mike Rupps
 
billybudd's Avatar
 
Join Date: Feb 2012
Posts: 12,050
vCash: 500
So basically, save percentage over the mean, if I"m understanding?

Seems reasonable.

billybudd is online now   Reply With Quote
Old
10-10-2013, 12:16 PM
  #6
Garbage Goal
Registered User
 
Join Date: Apr 2009
Posts: 17,721
vCash: 500
That's actually the first I've heard of it. Yeah, I like that stat.

Garbage Goal is online now   Reply With Quote
Old
10-10-2013, 12:16 PM
  #7
HarrisonFord
Lost a bet
 
HarrisonFord's Avatar
 
Join Date: Jul 2011
Location: Toronto
Country: Canada
Posts: 9,926
vCash: 500
Quote:
Originally Posted by Taco MacArthur View Post
I've been doing exactly this for quite some time now. For instance:

http://hockeygoalies.org/bio/nhl/toronto.html

(scroll down to where save percentage has been calculated)

Z-Score tells how many standard deviations above (or below) league average a goaltender's performance was during the season.

GD tells how many goals a goaltender prevented during the season, above and beyond the league average.

GAR is the same as above, but comparing against a "replacement-level" goaltender (since GD would assign a value of zero to a league-average goaltender, but there's definite value in being average).

SNW% gives what the goaltender's winning percentage would be if he were on a team that (a) faces a league-average number of shots, and (b) scores a league-average number of goals. (SNW and SNL are then a goaltender's support-neutral wins and losses).

For more recent seasons, I also calculate a goaltender's game-to-game variance (as a measure of consistency) as well as the strength of schedule faced by the goaltender.

I describe these all in a bit more detail here: http://hockeygoalies.org/stats/glossary.html
By just glancing at the data, I think there's a pretty obvious trend of either all or none of the regular goalies being above/below 0. It would be interesting if there was some sort of way to measure how much of a goaltenders SV% was based on their own performance, and how much was based upon the defensive system playing in front of them.

HarrisonFord is offline   Reply With Quote
Old
10-10-2013, 12:18 PM
  #8
HarrisonFord
Lost a bet
 
HarrisonFord's Avatar
 
Join Date: Jul 2011
Location: Toronto
Country: Canada
Posts: 9,926
vCash: 500
Quote:
Originally Posted by billybudd View Post
So basically, save percentage over the mean, if I"m understanding?

Seems reasonable.
Honestly it makes sense. Measuring exceptional performance is comparing the yearly performance of a player compared to his peers. Which is part of the Gretzky is the greatest player ever argument. If you look at his performance compared to the mean, it's pretty staggering.

HarrisonFord is offline   Reply With Quote
Old
10-10-2013, 12:18 PM
  #9
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
Quote:
Originally Posted by HarrisonFord View Post
By just glancing at the data, I think there's a pretty obvious trend of either all or none of the regular goalies being above/below 0. It would be interesting if there was some sort of way to measure how much of a goaltenders SV% was based on their own performance, and how much was based upon the defensive system playing in front of them.
That's true in many cases (although certainly not all), and it's something that we continue to investigate.

One improvement (that I don't present on my site) is normalizing a goaltender's save percentage for the number of shots faced in each manpower situation (even strength, power play, shorthanded). That helps to correct for goaltenders who face an inordinate amount of shorthanded situations.

Doctor No is offline   Reply With Quote
Old
10-10-2013, 12:20 PM
  #10
Grant
LL Genius
 
Grant's Avatar
 
Join Date: Jan 2012
Location: London
Country: Canada
Posts: 13,043
vCash: 50
I wish there was also an option to only show the starting goaltenders, or be able to eliminate all the goalies who played under a certain number of games. Because looking at the list, those that are above average is dominated by starting goaltenders (makes sense). Would be interested to see how much better/worse a goalie was compared to the other starting goaltenders around the league.

Grant is offline   Reply With Quote
Old
10-10-2013, 12:20 PM
  #11
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
One thing I note is that H-R's GSAA is slightly different than my comparable calculation (GD, or goal differential).

My guess is that for this calculation, H-R includes the goaltender in question's shots and saves in the league average calculation. I don't do that - my opinions is that if you have an exceptional goaltender, you want to compare him against everyone else (not everyone else including him).

But both ways can make sense, and reasonable folks can disagree.

Doctor No is offline   Reply With Quote
Old
10-10-2013, 12:24 PM
  #12
Mike Farkas
Hockey's Future Staff
Grace Personified
 
Mike Farkas's Avatar
 
Join Date: Jun 2006
Location: PA
Country: United States
Posts: 5,229
vCash: 500
Yeah, I was going to say, Taco has been providing this type of information and more on the HoH board for some time now.

With respect to H-R's work on this and stat guys everywhere, it's nothing new under the sun. It doesn't account for differences in modern goaltending and modern defensive tactics - not that anyone claimed it to be doing that. It does serve to help contextualize the numbers to the era a little bit. A .920 save pct. back in the day would win you a Vezina, now it's about average for a starter. I guess it helps in that calibration. Basically, echoing what O-J said.

If people are really into this kind of stuff, like I said (and now that I've scrolled up, like he said), Taco does a lot of math stuff that is way over my head: http://hfboards.hockeysfuture.com/sh...85&postcount=3

Mike Farkas is offline   Reply With Quote
Old
10-10-2013, 12:24 PM
  #13
HarrisonFord
Lost a bet
 
HarrisonFord's Avatar
 
Join Date: Jul 2011
Location: Toronto
Country: Canada
Posts: 9,926
vCash: 500
Quote:
Originally Posted by Taco MacArthur View Post
That's true in a lot of cases, and it's something that we continue to investigate.

One improvement (that I don't present on my site) is normalizing a goaltender's save percentage for the number of shots faced in each manpower situation (even strength, power play, shorthanded). That helps to correct for goaltenders who face an inordinate amount of shorthanded situations.
Yup. Might be worth trying to look at the historical SV% of different coaches over their careers as well, and trying to find a way to eliminate the system bias. Shorthanded bias is a good one to eliminate as well. I'd assume that the most accurate indicator of goaltender performance would come at even strength

HarrisonFord is offline   Reply With Quote
Old
10-10-2013, 12:25 PM
  #14
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
Quote:
Originally Posted by HarrisonFord View Post
Yup. Might be worth trying to look at the historical SV% of different coaches over their careers as well, and trying to find a way to eliminate the system bias. Shorthanded bias is a good one to eliminate as well. I'd assume that the most accurate indicator of goaltender performance would come at even strength
Coaches are something that I use in my predictive model - it doesn't add a lot of lift, but it's definitely an effect.

Doctor No is offline   Reply With Quote
Old
10-10-2013, 12:29 PM
  #15
HarrisonFord
Lost a bet
 
HarrisonFord's Avatar
 
Join Date: Jul 2011
Location: Toronto
Country: Canada
Posts: 9,926
vCash: 500
Quote:
Originally Posted by Taco MacArthur View Post
Coaches are something that I use in my predictive model - it doesn't add a lot of lift, but it's definitely an effect.
True. Just like with everything, you'll have different coaches having a varying effect on it. Coaches that stick out to me in the game today are Tippett and MacLean.

Good on you for undertaking this, it seems like an interesting study. And probably an exhaustive one too on first glance to an outsider it probably seems so easy to accurately goaltender performance, but there's so many contributing factors to SV% outside of JUST the goalies abilities

HarrisonFord is offline   Reply With Quote
Old
10-10-2013, 12:30 PM
  #16
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
Quote:
Originally Posted by Mike Farkas View Post
If people are really into this kind of stuff, like I said (and now that I've scrolled up, like he said), Taco does a lot of math stuff that is way over my head: http://hfboards.hockeysfuture.com/sh...85&postcount=3
Thanks! As for the last part, that just means that I need to explain it better.

Two of the barriers for any "advanced" statistic are (1) making it understandable and (2) making it accessible. And the term "advanced" doesn't help, either, because it scares a lot of people off.

Raw save percentage used to be considered an advanced statistic, and people were much more comfortable with wins/losses, goals-against average, and shutouts to measure a goaltender's performance. Over time, it's become part of the common parlance. Part of that is that people now have a good sense of the save percentage scale, what a "great" save percentage is, what a "lousy" save percentage is, and the levels in between.

I'm at the point where I'm comfortable working with the save percentage translations (GD, GAR, SNW%, Z-Score), but I'm still learning the nuances of the new metrics I've put together (for consistency and strength of opponent). I'm writing a short paper on the latter, so that should hopefully help people feel more comfortable with it.

Doctor No is offline   Reply With Quote
Old
10-10-2013, 12:35 PM
  #17
Fire Sather
new Niclas Wallin?
 
Fire Sather's Avatar
 
Join Date: Oct 2002
Location: Connecticut
Country: United States
Posts: 20,421
vCash: 500
Send a message via AIM to Fire Sather
I'll watch the games

Fire Sather is online now   Reply With Quote
Old
10-10-2013, 12:38 PM
  #18
Cashville
RIP Lindback
 
Cashville's Avatar
 
Join Date: Apr 2011
Location: Washington, DC
Country: United States
Posts: 3,324
vCash: 500
Frankly, I think you need to go a step further and make it ES SV%+, but yeah it's good.

Cashville is offline   Reply With Quote
Old
10-10-2013, 12:40 PM
  #19
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
Bringing a few things to the table to help show why save percentage translations are important.

Here's a list of all Vezina Trophy winners:
http://hockeygoalies.org/awards/vezina.html

Over time, you can see the raw (unadjusted) save percentages fluctuate, as periods of offense dominated, and periods of defense dominated. Did all of the goaltenders in the 1980s suck? Maybe, but unlikely.

If you look at the adjusted metrics (Z-Score, Goal Differential, and Goals Above Replacement), you can still see some outliers - this is purportedly the list of each year's best goalie, after all - but the pattern evens out a bit.

It's also easier to pick out the truly remarkable performances. Note Dominik Hasek's 93% (raw) save percentage in 1993-94 - if you look at his z-score, you can see that his performance was 4.6 standard deviations above the league average. What "z score" measures is the likelihood that an average goaltender would put together a season like the one in question (since we've all seen average goaltenders put together stretches of great play). A z-score of 4.6 means that an average goaltender would reproduce Hasek's 1993-94 season about once every 475,000 years.

If you click through (on the above page), you can see the individual goaltenders' careers evolve (under REGULAR SEASON STATISTICS or POSTSEASON STATISTICS). It helps to compare (for instance) Patrick Roy's career with Dominik Hasek's career with Martin Brodeur's career (and you can see that each was remarkable in his own way).

Doctor No is offline   Reply With Quote
Old
10-10-2013, 12:40 PM
  #20
13th Floor
Registered User
 
13th Floor's Avatar
 
Join Date: Oct 2008
Country: United States
Posts: 4,873
vCash: 500
Quote:
Originally Posted by Fire Sather View Post
I'll watch the games
You must be busy.

13th Floor is offline   Reply With Quote
Old
10-10-2013, 12:44 PM
  #21
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
Quote:
Originally Posted by Fire Sather View Post
I'll watch the games
This is a common (and flippant) rejoinder, and I'm sure that you're quietly smiling to yourself about how you just "stuck it to the statheads".

However, "watching the games" is just as useless as "putting your head in a book" for properly evaluating players (both are necessary).

You should read Daniel Kahneman's book "Thinking Fast and Slow", which largely deals with narrative bias. Our brains are exceptionally biased, and we generally use whatever facts support our preconceived notions about players (and ignore anything that disagrees with those notions). It doesn't help that sports shows replay "highlight" goals and saves, which places a disproportionate share of those events into our memory stores.

We also don't have the ability to watch (and store in our brains) sample sizes for all goaltenders at once, let alone across multiple seasons.

Bottom line: for player evaluation, anyone who tells you that statistics are useless is just as wrong as anyone who tells you that watching the games is useless. You need *both* to properly evaluate.

Doctor No is offline   Reply With Quote
Old
10-10-2013, 12:47 PM
  #22
5 Minute Major
Registered User
 
5 Minute Major's Avatar
 
Join Date: Dec 2010
Location: Binghamton, NY
Country: United States
Posts: 2,656
vCash: 500
I wish there was a stat kept for quality scoring chance save percentage. Maybe there is and I don't know that?

5 Minute Major is offline   Reply With Quote
Old
10-10-2013, 12:48 PM
  #23
RPGrizzly
Registered User
 
RPGrizzly's Avatar
 
Join Date: Jan 2006
Country: United States
Posts: 399
vCash: 500
Quote:
Originally Posted by Taco MacArthur View Post
I've been doing exactly this for quite some time now. For instance:

http://hockeygoalies.org/bio/nhl/toronto.html
(scroll down to where save percentage has been calculated)

Z-Score tells how many standard deviations above (or below) league average a goaltender's performance was during the season.

GD tells how many goals a goaltender prevented during the season, above and beyond the league average (looking at the link above, it appears that H-R is doing this now, calling it GSAA).

GAR is the same as above, but comparing against a "replacement-level" goaltender (since GD would assign a value of zero to a league-average goaltender, but there's definite value in being average).

SNW% gives what the goaltender's winning percentage would be if he were on a team that (a) faces a league-average number of shots, and (b) scores a league-average number of goals. (SNW and SNL are then a goaltender's support-neutral wins and losses).

For more recent seasons, I also calculate a goaltender's game-to-game variance (as a measure of consistency) as well as the strength of schedule faced by the goaltender.

I describe these all in a bit more detail here: http://hockeygoalies.org/stats/glossary.html
Wow, thanks for this. I've always thought there was a lot more to the typical goalie stats you see, I had no idea the data was already out there! Great work, I'll definitely be spending some time on your website.

I don't have any problems with the SV%+ stat on H-R. It's pretty interesting.

RPGrizzly is offline   Reply With Quote
Old
10-10-2013, 12:48 PM
  #24
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
Quote:
Originally Posted by 5 Minute Major View Post
I wish there was a stat kept for quality scoring chance save percentage. Maybe there is and I don't know that?
The problem with that (from a technical perspective) is that "quality scoring chance" isn't well-defined. Do you know how some teams always lead the league in hits (or blocked shots) by a large margin? It's because a "hit" in Arena A is different from a "hit" in Arena B.

If the definition could be standardized, it'd be very useful (and it would be great to see).

Doctor No is offline   Reply With Quote
Old
10-10-2013, 12:49 PM
  #25
Doctor No
Mod Supervisor
Retired?
 
Doctor No's Avatar
 
Join Date: Sep 2005
Posts: 24,354
vCash: 500
Quote:
Originally Posted by RPGrizzly View Post
Wow, thanks for this. I've always thought there was a lot more to the typical goalie stats you see, I had no idea the data was already out there! Great work, I'll definitely be spending some time on your website.

I don't have any problems with the SV%+ stat on H-R. It's pretty interesting.
Thanks - appreciated! Let me know if you have questions; like I said, part of my goal is to get more people thinking about this sort of thing (since I believe that it leads to a more effective understanding of goaltenders).

Doctor No is offline   Reply With Quote
Reply

Forum Jump


Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -5. The time now is 01:10 PM.

monitoring_string = "e4251c93e2ba248d29da988d93bf5144"
Contact Us - HFBoards - Archive - Privacy Statement - Terms of Use - Advertise - Top - AdChoices

vBulletin Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
HFBoards.com is a property of CraveOnline Media, LLC, an Evolve Media, LLC company. 2014 All Rights Reserved.