By The NumbersHockey Analytics... the Final Frontier. Explore strange new worlds, to seek out new algorithms, to boldly go where no one has gone before.
This is a little statistic I calculate each season - it's basically a linear transformation of save percentage (so if you don't like save percentage, you won't like this either, and I'm not going to argue with you about it here ).
The statistics presented:
GD (Goal Differential) - how many goals better (or worse) a netminder is than a league-average NHL goaltender facing the same number of shots.
GARG (Goals Above Replacement Goaltender) - how many goals better (or worse) a netminder is than a replacement-level NHL goaltender facing the same number of shots. I define "replacement level" to be a save percentage 0.015 below league average (this is a bit of a SWAG). Replacement level goaltenders are plentiful and easily available.
SNW/SNL (Support-Neutral Wins and Losses) - if the goaltender had received "average" goal support, how many wins and losses would he have.
I sort this table by GARG, because there's a definite value in playing at a "league average" level for a decent period of time.
Good catch! Thanks to the powers of modern technology, I had to manually combine Garon's (and Gerber's, Tellqvist's and LaBarbera's) statistics to create an aggregate total of his 2008-09 performance. I then forgot to delete the rows corresponding to their individual team performances.
Nice work, this is an interesting study. I like how it's translated into a metric of value which allows direct comparison between the goalies in terms of total value. I also like the general design/methodology of the metric.
If I understand correctly, your metric rests on a couple of fundamental assumptions:
- that save % better measures the quality of goaltending than the more direct GAA (of course various factors influence each)
- that the replacement level of goalies is a constant .015 below league average of save %
If you already have the data in a format that allows you to make additional calculations relatively quickly, you might want to try some alternative approaches and see how the results compare.
Have you considered using the more direct GAA? I'm not familiar with much work on goalies, so I wouldn't be surprised if that had already been done, or you have other reason to believe your approach superior to that one.
Whether you use save % or GAA, you might also consider different ways of determining replacement level goaltending. For instance, you could approach it as follows:
- Sort the top 2N (where N is # teams in league) goalies in NHL in terms of games played. Assume the top 1N (in terms of games, or alternatively in terms of either save % or GAA) of those 2N are the "starters" and that the remainder are "backups." Use either the mean or median goalie among goalies 1N+1 to 2N as the replacement level. An alternative to selecting the top 1N goalies is to simply use the "actual" starters and then use the same basic method from there.
- Sort all the goalies in terms of games played or GAA/save% (if use either of latter two, could use min. games or top ~2.5N goalies in games). If sorted by save % or GAA, use the 2N+1 goalie as replacement level. If sorted by games, use a weighted average of GAA/save% for all goalies > 2N in games.
- Sort the top 2N goalies in one of the above ways, and calculate a standard deviation of GAA/save% among those goalies. Then calculate replacement level as X standard deviations below the mean of those 2N goalies.
I'm not saying any of these will necessarily yield better results, but I think they at least some are worth considering. Besides being arbitrary, a fixed number may be more or less accurate as the depth of goaltending and/or league parity vary. I know it's difficult to determine replacement level, but the approaches I outlined above rest on logic of different sorts that might be accepted over a SWAG (however accurate that may be).
A suggestion, maybe? Don't know if it's a good idea because of the small sample size. But pro-rating the '94-95 season might be a decent idea for the individual season stats.
Does anyone have any views on these methods of determining replacement level goalies? This assumes that GAA and/or Save % are used as the primary basis of the metric for goaltender value.
A) Sort the top 2N (where N is # teams in league) goalies in NHL in terms of games played. Assume the top 1N (in terms of games, or alternatively in terms of either save % or GAA) of those 2N are the "starters" and that the remainder are "backups." Use either the mean or median goalie among goalies 1N+1 to 2N as the replacement level. An alternative to selecting the top 1N goalies is to simply use the "actual" starters and then use the same basic method from there.
B) Sort all the goalies in terms of games played or GAA/save% (if use either of latter two, could use min. games or top ~2.5N goalies in games). If sorted by save % or GAA, use the 2N+1 goalie as replacement level. If sorted by games, use a weighted average of GAA/save% for all goalies > 2N in games.
C) Sort all the goalies by GAA/Sv% (with min. games) and use the 1N+1 goalie as replacement level.
The basis of "A" is that a goalie with the mean/median level of the back-ups should be available by trade for a relatively reasonable amount. The basis of "B" is that goalies not considered starter/backup quality should be available for very little. The basis of "C" is that a goalie who's not of starter quality should be considered replacement level.
When I did this, I did something that was a combination science/art - I looked at each NHL team, and considered who the top two goaltenders were for each team at the start of the season.
Then, I compared their aggregated statistics with those of all other goalies (goalies called up from the minors either due to poor play or injury).
I arrived at about a save percentage difference of about 0.015 for each of the three seasons I tested, and that passed my sniff test, so I went with it. I'd like to do something more rigorous sometime once I have the time.
When I did this, I did something that was a combination science/art - I looked at each NHL team, and considered who the top two goaltenders were for each team at the start of the season.
Then, I compared their aggregated statistics with those of all other goalies (goalies called up from the minors either due to poor play or injury).
I arrived at about a save percentage difference of about 0.015 for each of the three seasons I tested, and that passed my sniff test, so I went with it. I'd like to do something more rigorous sometime once I have the time.
That seems reasonable, and similar to the methods which I've considered (you chose "B" it sound like). In the thread for your study, you said it was a "SWAG" (wild guess), but it seems you actually did some research and made a very educated guess in choosing (league avg. SV% - .015) for replacement level. From what I know, as we go back in time, the parity was lower, total league talent was lower, and SV% differed from more recent levels, so I wonder if the differential varied significantly in past decades?
I hope you didn't view my feedback on your study as negative, as it certainly was not intended that way. You did some nice work there.
Is SV% considered more reliable than GAA due to less influence from team/external factors? I haven't looked at goalies too closely, so just wondering what the consensus is.
Is SV% considered more reliable than GAA due to less influence from team/external factors? I haven't looked at goalies too closely, so just wondering what the consensus is.
SV% is better than GAA. Even strength SV% is even better than regular SV% too.
SV% is better than GAA. Even strength SV% is even better than regular SV% too.
Where do you find ES SV% and how far back is it available?
Here's some replacement levels I calculated for goalies for the last 28 seasons. "Med 2N" is the top 2N goalies in games played (N= number of teams) are sorted by GAA or SV%. The median of the 1N+1 to 2N goalies is taken as replacement level (middle of the road backup). For instance, in a 30 team league, take the top 60 in GP, sort them by the relevant metric (GAA/SV%), then average goalies #45 & #46 in that metric. "Med N+1" sorts the top 2N goalies in games and uses N+1. "Avg." is the league average. "GVT" is 4% more goals allowed than league average.
YEAR
Med N+1
Med 2N
Avg.
GVT
2012
91.3%
90.4%
91.4%
91.1%
2011
91.4%
90.7%
91.3%
91.0%
2010
90.9%
90.4%
91.1%
90.7%
2009
91.0%
90.0%
90.8%
90.4%
2008
90.8%
90.0%
90.9%
90.5%
2007
90.5%
89.3%
90.5%
90.1%
2006
90.0%
89.3%
90.1%
89.7%
2004
91.0%
90.5%
91.1%
90.7%
2003
90.8%
89.7%
90.9%
90.5%
2002
90.6%
90.1%
90.8%
90.4%
2001
90.1%
89.3%
90.3%
89.9%
2000
90.5%
89.7%
90.4%
90.0%
1999
90.6%
89.8%
90.8%
90.4%
1998
90.5%
90.0%
90.6%
90.2%
1997
90.3%
89.7%
90.5%
90.1%
1996
90.0%
88.7%
89.8%
89.4%
1995
90.2%
89.0%
90.1%
89.7%
1994
89.3%
88.3%
89.5%
89.1%
1993
88.6%
87.8%
88.5%
88.0%
1992
88.6%
88.1%
88.8%
88.4%
1991
88.9%
87.8%
88.6%
88.1%
1990
88.0%
87.0%
88.1%
87.6%
1989
88.0%
87.1%
87.9%
87.4%
1988
87.8%
86.8%
88.0%
87.5%
1987
88.1%
87.4%
88.0%
87.5%
1986
87.5%
86.1%
87.4%
86.9%
1985
87.7%
86.3%
87.5%
87.0%
1984
87.1%
86.3%
87.3%
86.8%
Last edited by Czech Your Math: 08-07-2012 at 08:35 PM.
For what it's worth, I now have goal differentials (and goals above replacement) up, for all NHL seasons starting in the early 1950s (regular and postseason), for all WHA regular seasons, and for other recent seasons.
).
