HFBoards (http://hfboards.hockeysfuture.com/index.php)
-   By The Numbers (http://hfboards.hockeysfuture.com/forumdisplay.php?f=241)
-   -   Corsi QoC-type adjusted sv% stat for goaltenders? (http://hfboards.hockeysfuture.com/showthread.php?t=1306553)

 seventieslord 12-11-2012 04:47 PM

Corsi QoC-type adjusted sv% stat for goaltenders?

I dislike getting into long posts about numbers because I always feel like I’m not doing the best job of getting the point across compared to guys like Iain Fyffe, overpass, Czech Your Math and Contrarian Goaltender can talk the numbers better than I can. But since I think I have an idea that hasn’t been done yet, I’m going to try to do my best to slog through this.

OK, let’s start with CORSI. It’s a simple metric that calculates a player’s +/- in shots directed at the net. However, it is very dependent on Quality Of Competition (QoC) – A player who faces the opposition’s best players (by CORSI) will have a harder time posting a better CORSI himself, so it’s important to consider QoC when looking at his CORSI and comparing to someone else.

With that said, I think the majority of the stat community now agrees that sv% is the best way to judge a goalie’s individual performance – or, stated more explicitly, “sv% after adjusting for factors such as shot over/undercounting, shot quality, special teams, and others”.

One metric that is often popular is the “comparison to backups” method because it eliminates factors such as special teams, shot quality, coaching tactics and shot over/undercounting. Goalies on the same team in the same season should have these factors somewhat wash out, meaning we can compare them more directly. And it’s generally said that most backups are of a rather uniform vanilla quality.

However, that last part, as much as we would like it to be true, isn’t. Sometimes there are great backups that should be starters on another team, some are more of a 1B goalie, some are average backups, some are sub-replacement level scrubs. Over decades, this should all wash out, but maybe it doesn’t.

I’ve seen posts that attempt to take the next step by saying “goalie A outperformed his backups by 10 points over this time, while goalie B only outperformed them his 2 points. However, goalie B had better backups because of how they performed relative to other goalies when on other teams”

So, my thought was, is it possible to take this all the way? Can there be a stat where a goalie’s “sv% +/-“ versus different goalies can be turned into an adjusted sv% that more or less takes QoC into consideration?

To give a really simple example, Andre Racicot’s career line would look like this:

1991: 13 minutes, 15 sv% points below Patrick Roy, 29 points ahead of JC Bergeron, 24 points ahead of Chabot*
1992: 436 minutes, 19 points below Roy, 8 ahead of Melanson*
1993: 1433 minutes, 13 points below Roy*
1994: 500 minutes, 68 points below Roy, 27 points below Kuntar, 10 points below Tugnutt*

also, I know the “proper” way to compare save percentages is percentage wise, and not just “points above or below” – I use that for illustration and as shorthand only.

On their own, those numbers don’t mean a lot. But after this is done for all goalies since the start of sv%, the system running it would them account for the fact that Patrick Roy always outperformed his backups significantly and spit out a number representing how Racicot performed relative to normalized competition.

I think this is the same in principle to QoC for CORSI. We don’t subjectively define who the best players are, we define it by CORSI. So a CORSI metric that considers QoC would “fix” a player (call him player C) with a very high CORSI earned by playing 3rd rate competition, by seeing that these 3rd rate players always had poor CORSIs against everyone, and that this player C just happened to face 3rd rate competition a disproportionately high percentage of the time, and “fix” his score to “predict” what his CORSI would look like against average competition. It could do the same for a top checker who always matches up against powerful scoring lines.

Couldn’t a goaltending stat like that be done somewhat easily? We can’t say how good Chris Osgood’s sv% was just by looking at it, and comparing to the league average still leaves the door wide open for Detroit factors to influence it, so we compare to other goalies on his teams. But he played with Vernon, and with Howard, and they’re good, so that’s not really fair, right? So look at how Vernon did compared to other goalies on other teams. But were they good or bad? How did they do on other teams? And so on and so forth.

Unless there’s some obvious piece I’m missing, I think that this could be put together relatively easily by someone with an understanding of scripts in excel and a save% spreadsheet. I can provide the latter to anyone willing to give it a go.

I have invited the smarter stats guys to this thread in hopes that they will provide their take on this question. I’m really sorry if this doesn’t make a lot of sense, fellas.

 Bleedred 12-11-2012 04:58 PM

I'd like to know what Brodeur's was last year. There's someone who swears by CORSI stats, but constantly needs to point out how below average Brodeur has been because of his combined 905 save percentage the last two years.

 TheDevilMadeMe 12-11-2012 04:58 PM

I think this would work well with goalies in tandem situations, where the starter and backup both play a significant number of different teams in different situations. But it gets tough when the starter plays 70 games and the backup 12, especially since the backup's games are not going to be randomly distributed. Some coaches will play backups on the 2nd half of a back to back (which will hurt the backup's numbers). Other coaches will only play the backups against weaker teams (which will help the backup's numbers relative to a starter).

 Chalupa Batman 12-11-2012 05:02 PM

Sample size issues will certainly be a concern.

I see two dimensions of QoC here - do goaltenders play against stronger or weaker competition than is reflected in save percentage, and then (once that is accounted for), do goaltenders pay better or worse than their teammates?

It would be hard to extract the signal from the noise (although perhaps not impossible).

 seventieslord 12-11-2012 05:07 PM

Quote:
 Originally Posted by TheDevilMadeMe (Post 56459807) I think this would work well with goalies in tandem situations, where the starter and backup both play a significant number of different teams in different situations. But it gets tough when the starter plays 70 games and the backup 12, especially since the backup's games are not going to be randomly distributed. Some coaches will play backups on the 2nd half of a back to back (which will hurt the backup's numbers). Other coaches will only play the backups against weaker teams (which will help the backup's numbers relative to a starter).
Quote:
 Originally Posted by Taco MacArthur (Post 56459887) Sample size issues will certainly be a concern. I see two dimensions of QoC here - do goaltenders play against stronger or weaker competition than is reflected in save percentage, and then (once that is accounted for), do goaltenders pay better or worse than their teammates? It would be hard to extract the signal from the noise (although perhaps not impossible).
I should have mentioned, it would be weighted. So the seasons that are insignificant sample sizes would not have a large overall impact on the goalie's career result.

As far as backups getting easier or harder starts from team to team... I can't think of a way to account for that.

 TheDevilMadeMe 12-11-2012 05:15 PM

Quote:
 Originally Posted by seventieslord (Post 56460081) As far as backups getting easier or harder starts from team to team... I can't think of a way to account for that.
To fix the issue of backups who only face weak teams, you could create a "strength of opposition" value based on the goals-for or shooting percentage of the opposing team (not sure which would be better, probably shooting percentage) and use it to weigh each game - in other word's a lot of work.

To deal with the issue of back-to-backs, you would have to look at the average drop in performance in the second game of back to backs.

 Czech Your Math 12-11-2012 07:33 PM

I think it would be very difficult to determine just how good the backups were, for many of the reasons given: small sample sizes, games played not always random, etc. Players also change in ability due to advancement, age, injuries, etc., so being X % below Goalie Y's SV% in one season may not be the same as in another season.

It might be better to use regression somehow, with a dependent variable of SV% (% or %points above/below avg.) and independent variables such as Shots Against per Game (or amount above/below avg.), Shot differential (Corsi?), PK% (or amount above/below avg.), etc.

 overpass 12-11-2012 09:55 PM

I think this idea is an excellent framework for evaluating goaltenders. I've played around with similar ideas, but haven't ever put the work in to do it properly.

Ideally the process would go through the following steps, using any goalie rate stat of your choice (winning percentage, GAA, save percentage, or even shutout percentage).

1. Calculate the expected (goalie stat) for each goaltender based on their opponents faced, home/road split of games played, and travel schedule surrounding games played. Use empirical data to find the magnitude of each adjustment.

2. Adjust all goaltender seasonal (goalie stat) by the expected (goalie stat) from step 1. For example, if a goalie has a 0.550 winning % and an expected 0.490 winning %, his adjusted winning % after this step is 0.560.

3. Calculate the difference between goaltenders and their goalie teammates in (goalie stat) for each goaltender-season.

4. Find the career difference a goalie and his goalie teammates by taking an average of the seasonal differences calculated in step 3, with each seasonal difference weighted by (the lesser of goalie GP or his goalie teammates' GP).

5. All goaltenders now have a career +/- difference in (goalie stat) relative to their teammates. Calculate the career +/- difference for each goaltender's goalie teammates, weighted by the same weight each teammate had in step 4 .

6. Adjust the career +/- difference in (goalie stat) for all goaltenders for their strength of goalie teammates, which was derived in step 5. Repeat this step several times if desired.

By far the biggest problem is that goaltenders don't necessarily have a constant ability level as this method assumes. And the method gives the most weight to half seasons, so this can mean that a longtime starting goalie is rated primarily by his weakest seasons. For example, Tony Esposito's career estimate would be heavily informed by his age 38, 39, and 40 seasons, because they were among the few seasons that he didn't play 65+ games. Any individual player can have his rating easily adjusted by picking which "prime seasons" to include and which to exclude, but that process isn't really feasible for a global rating in which each player's rating is important.

 Henkka 12-12-2012 01:18 AM

Quote:
 Originally Posted by TheDevilMadeMe (Post 56459807) I think this would work well with goalies in tandem situations, where the starter and backup both play a significant number of different teams in different situations. But it gets tough when the starter plays 70 games and the backup 12, especially since the backup's games are not going to be randomly distributed. Some coaches will play backups on the 2nd half of a back to back (which will hurt the backup's numbers). Other coaches will only play the backups against weaker teams (which will help the backup's numbers relative to a starter).
This really matters a lot. Usually the crappiest backups are those who doesn't play a lot behind a +70 game starter. Their gameshape is just missing.

I've seen it thousand times that when teams use an AHL goaltender who is in theory weaker than that backup on the bench, the AHL-guy playes better because of better gameshape (playing all the time in the minors).

I've involved in some fantasy league, that has unique way to rank goaltenders, and boy I can say, it rates them good. The best goaltenders from last 3 seasons are Lundqvist, Rinne, Quick and Thomas and the weakest group was Pavelec, Brodeur, Roloson, Crawford and the worst of all Steve Mason.

The system uses league average save percentage, and then relative difference to others, so those relative values as a career average would work over decade to decade from high scoring era to low-scoring era.

 Chalupa Batman 12-12-2012 08:53 AM

Quote:
 Originally Posted by Henkka (Post 56472755) The system uses league average save percentage, and then relative difference to others, so those relative values as a career average would work over decade to decade from high scoring era to low-scoring era.
That's the reasonably standard approach (at least in this section), although we usually weight by time played as well.

 Jason MacIsaac 12-12-2012 09:47 PM

I would think a higher QOC would increase the shot total, not necessarily the sv%.

 TheDevilMadeMe 12-13-2012 10:09 AM

Quote:
 Originally Posted by Jason MacIsaac (Post 56498815) I would think a higher QOC would increase the shot total, not necessarily the sv%.
Depends on if you accept that different teams have inherently different shooting percentages.

 Jason MacIsaac 12-13-2012 10:23 AM

Quote:
 Originally Posted by TheDevilMadeMe (Post 56507873) Depends on if you accept that different teams have inherently different shooting percentages.
I certainly don't accept that, unless you're talking about arena bias. I just don't think the gap between teams is enough to create that sh% skill. Maybe precap but not now.

 overpass 12-13-2012 11:54 AM

Quote:
 Originally Posted by Jason MacIsaac (Post 56508129) I certainly don't accept that, unless you're talking about arena bias. I just don't think the gap between teams is enough to create that sh% skill. Maybe precap but not now.
Check out the shooting percentages for the dynasty Oilers.

 Jason MacIsaac 12-13-2012 12:42 PM

Quote:
 Originally Posted by overpass (Post 56510141) Check out the shooting percentages for the dynasty Oilers.
Did you read my post? I understand the Oilers were outshot in many games during their dynasty, the game is different/closer now days.

 overpass 12-13-2012 05:46 PM

Quote:
 Originally Posted by Jason MacIsaac (Post 56511271) Did you read my post? I understand the Oilers were outshot in many games during their dynasty, the game is different/closer now days.
The OP used Andre Racicot as an example. Obviously this concept should have an historical application. It's very relevant that shooting percentages of teams differ.

Also the fact that team shooting percentages currently lie within a fairly narrow band is specific to today's NHL and is not a fundamental hockey truth. Why limit the metric based on conditions that could change at any time?

 Jason MacIsaac 12-14-2012 01:03 AM

Quote:
 Originally Posted by overpass (Post 56520603) The OP used Andre Racicot as an example. Obviously this concept should have an historical application. It's very relevant that shooting percentages of teams differ. Also the fact that team shooting percentages currently lie within a fairly narrow band is specific to today's NHL and is not a fundamental hockey truth. Why limit the metric based on conditions that could change at any time?
I don't believe with a stricter cap that it will change. If anything that band will become narrower. There maybe an outlier some year of a perfect cap team but moving forward I don't believe team sh% is a skill we should focus on. This is especially true comparing goaltenders from previous eras to post cap era.

 Bleedred 12-14-2012 01:53 AM

Isn't there some kind of stat similar to CORSI that measures how good a goaltender played? Like for instance a goaltender who faces higher quality shots with a lower save percentage, might come out better than the high save percentage, but low opportunity goaltenders?

 Jason MacIsaac 12-14-2012 02:01 AM

Quote:
 Originally Posted by Bleedred (Post 56531003) Isn't there some kind of stat similar to CORSI that measures how good a goaltender played? Like for instance a goaltender who faces higher quality shots with a lower save percentage, might come out better than the high save percentage, but low opportunity goaltenders?
There was a DIGR method but Mike Schuckers didn't continue work on it.

 pdd 12-15-2012 01:38 AM

I remember seeing an article a while back that was a strength of opposition adjustment;

Don't know if anything else was ever published, but the adjusted RS numbers for Fuhr and Moog were included and pretty startlingly different given how close they are raw.

 All times are GMT -5. The time now is 07:03 PM.

vBulletin Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
HFBoards.com, A property of CraveOnline, a division of AtomicOnline LLC ©2009 CraveOnline Media, LLC. All Rights Reserved.