HFBoards

Go Back   HFBoards > General Hockey Discussion > By The Numbers
By The Numbers Hockey Analytics... the Final Frontier. Explore strange new worlds, to seek out new algorithms, to boldly go where no one has gone before.

Total Hockey Rating (THoR): A comprehensive statistical rating of NHL Players

Reply
 
Thread Tools
Old
03-02-2013, 10:27 AM
  #26
XX
... Waiting
 
XX's Avatar
 
Join Date: Dec 2002
Location: 48th State
Country: United States
Posts: 27,171
vCash: 500
Can't account for system differences, situational differences (playing with the lead vs protecting one) or likely even positional differences between forwards. These measurements are never very meaningful. I think trying to boil hockey down to 1 rating is missing the point. In baseball, every pitch may as well be an entire game, statistically speaking, so numbers work well. Hockey is much too fluid for accurate or all-encompassing numbers.

XX is offline   Reply With Quote
Old
03-02-2013, 11:56 AM
  #27
number72
Registered User
 
Join Date: Oct 2011
Posts: 5,883
vCash: 500
Any ranking that doesn't have Kessel and Phaneuf in the top 10 in the NHL is not credible

number72 is offline   Reply With Quote
Old
03-02-2013, 05:03 PM
  #28
fedfed
Moderator
@FedFedRMNB
 
fedfed's Avatar
 
Join Date: Oct 2010
Location: Moscow City
Country: Russian Federation
Posts: 3,521
vCash: 500
Really, it looks like a stat for the sake of stat just because it comes nowhere close to reflecting what's happening. In general, stats should give predictable results: Malkin IS better than Kennedy, Kopitar IS better than Steen. If stat doesn't give such results, it can't be trusted to be a judge in a closer comparisons and last checked, that's what we need from stats.

fedfed is offline   Reply With Quote
Old
03-04-2013, 11:42 AM
  #29
pdd
Registered User
 
Join Date: Feb 2010
Posts: 5,578
vCash: 500
Quote:
Originally Posted by number72 View Post
Any ranking that doesn't have Kessel and Phaneuf in the top 10 in the NHL is not credible
You're referring to one that's limited to current Maple Leafs, right?

pdd is offline   Reply With Quote
Old
03-04-2013, 12:16 PM
  #30
ChiTownPhilly
Registered User
 
ChiTownPhilly's Avatar
 
Join Date: Feb 2010
Location: S Jersey near Philly
Country: United States
Posts: 285
vCash: 500
Send a message via AIM to ChiTownPhilly
Quote:
Originally Posted by ChadSC View Post
One of the hockey related research papers is "Total Hockey Rating (THoR): A comprehensive statistical rating of National Hockey League forwards and defensemen based upon all (sic) on-ice events" - and is a very interesting look at valuing players.

"Here we present a new comprehensive rating that accounts for other players on the ice will a give player as well as the impact of where a shift starts and of every non-shooting events such as (emphasis mine) turnovers and hits that occur when a player is on the ice.
I recognize the legal maxim that states "hard cases make bad law," but let me give you an example of one of the most memorable plays I've ever seen in person- and postulate that the metric described above has little way of taking the contribution into account.

Philadelphia v. up-and-coming Ottawa team. Legion of Doom line. Renberg carries in puck from R Wing. Lindros gets deep into the zone, and angles to L side, then cuts behind the net as if preparing to start a cycle. Two defenders give complete attention to Lindros, which leaves Renberg partially open, and John LeClair completely open. Cross-ice pass/score.

Now Lindros had more to do with that goal than any player... and he didn't even touch the puck...

I for Incomplete on the measurement attempt as given above. Needs to return to the shed for some overhaul work---

ChiTownPhilly is offline   Reply With Quote
Old
03-04-2013, 06:44 PM
  #31
NugentHopkinsfan
Registered User
 
NugentHopkinsfan's Avatar
 
Join Date: Dec 2009
Location: Vancouver
Posts: 9,087
vCash: 500
Quote:
Originally Posted by XX View Post
Can't account for system differences, situational differences (playing with the lead vs protecting one) or likely even positional differences between forwards. These measurements are never very meaningful. I think trying to boil hockey down to 1 rating is missing the point. In baseball, every pitch may as well be an entire game, statistically speaking, so numbers work well. Hockey is much too fluid for accurate or all-encompassing numbers.
It won't stop people with too much free time from trying to create ratings that are seriously flawed and tell you nothing more than watching the games will tell you.

NugentHopkinsfan is offline   Reply With Quote
Old
03-04-2013, 06:48 PM
  #32
Chalupa Batman
Mod Supervisor
 
Chalupa Batman's Avatar
 
Join Date: Sep 2005
Posts: 23,030
vCash: 500
Quote:
Originally Posted by NugentHopkinsfan View Post
It won't stop people with too much free time from trying to create ratings that are seriously flawed and tell you nothing more than watching the games will tell you.
(1) What's it to you what people do with their free time? (Follow-up: what do you do with your free time?)

(2) Do you think that they're trying to create ratings that are seriously flawed? Knowledge evolves.

I think you're forgetting which sub-forum you are currently reading.

Chalupa Batman is offline   Reply With Quote
Old
03-05-2013, 12:04 PM
  #33
ChiTownPhilly
Registered User
 
ChiTownPhilly's Avatar
 
Join Date: Feb 2010
Location: S Jersey near Philly
Country: United States
Posts: 285
vCash: 500
Send a message via AIM to ChiTownPhilly
Quote:
Originally Posted by Richer View Post
Say this with a straight face: "Grabovski is only slightly better than Crosby statistically."

I think THoR needs some work.
How does this happen?? I think I have an hypothesis...

Any reasonable system of competitive measurement needs to take into account strength of opposition, and do so in a manner that keeps from spiralling into the vortex of infinite regression. Now, particularly in the football world, Jeff Sagarin has come up with ideas that border on elegant- but really, my favorite (which doesn't necessarily mean best) solution(s) to this came from George Ignatin. Now Ignatin's work focused on the pros- and here, there are only 32 entities that call for measuring. Sagain's most famous work (BCS College Football Teams) is more challenging, by a factor of at least four. In the NHL, however, there are over 500 players that log meaningful ice-time, so answering a "strength of opposition" challenge here would be immense.

The intuitive hole in assigning parity to Grabovski (breath*) and Crosby is that perhaps strength of opposition was not sufficiently accounted for- or even ignored. One could reasonably anticipate that the former may draw the injury-fill-in/cold & flu season 7th defenseman, whereas the future Hall-of-Famer is getting a steady diet of the top pairing, continuously.

*[Salving my conscience by not mentioning them in the same breath.]

ChiTownPhilly is offline   Reply With Quote
Old
03-10-2013, 03:24 PM
  #34
ekespou
Registered User
 
Join Date: Mar 2013
Posts: 3
vCash: 500
I think there are at least two glaring problems with this paper:

1) Not sure if this is just inexcusably poor writing or whether it's actually part of the theoretical foundation for the analysis, but the paper expressly states more than once, and does seem to rely on the assumption, that it is "evaluating players for every event that happens on the ice." I.e., all stats kept by RTSS. Obviously those stats are severely deficient and incomplete, which means that if the paper does in fact assume that the stats kept by RTSS are coextensive with all meaningful on-ice actions then its results are going to be skewed, probably significantly.

2) With the exception of a few express attempted adjustments to the RTSS stats (e.g., MSG shot locations), the study makes no attempt to differentiate for quality of the on-ice actions that are recorded as stats. In a sense this criticism is included within #1, in that the idea is that just as RTSS fails to do, so THor does not make any allowance for the difference in quality of shots, hits, etc. etc.

I'm new to the boards and expect there may be some disagreement about this second point, as in many ways Corsi and Fenwick suffer from the same problem.

ekespou is offline   Reply With Quote
Old
03-10-2013, 08:43 PM
  #35
Wizeman*
Registered User
 
Join Date: Jan 2013
Posts: 2,624
vCash: 500
Quote:
Originally Posted by ekespou View Post
I think there are at least two glaring problems with this paper:

1) Not sure if this is just inexcusably poor writing or whether it's actually part of the theoretical foundation for the analysis, but the paper expressly states more than once, and does seem to rely on the assumption, that it is "evaluating players for every event that happens on the ice." I.e., all stats kept by RTSS. Obviously those stats are severely deficient and incomplete, which means that if the paper does in fact assume that the stats kept by RTSS are coextensive with all meaningful on-ice actions then its results are going to be skewed, probably significantly.

2) With the exception of a few express attempted adjustments to the RTSS stats (e.g., MSG shot locations), the study makes no attempt to differentiate for quality of the on-ice actions that are recorded as stats. In a sense this criticism is included within #1, in that the idea is that just as RTSS fails to do, so THor does not make any allowance for the difference in quality of shots, hits, etc. etc.

I'm new to the boards and expect there may be some disagreement about this second point, as in many ways Corsi and Fenwick suffer from the same problem.
I agree. I do think its incomplete

Wizeman* is offline   Reply With Quote
Old
03-11-2013, 02:43 AM
  #36
LiveeviL
No unique points
 
LiveeviL's Avatar
 
Join Date: Jan 2009
Location: Jämtland, Sweden
Country: Sweden
Posts: 5,370
vCash: 50
Send a message via ICQ to LiveeviL
Quote:
Originally Posted by CanadianHockey View Post
It's tracked by humans. One guy might see two players bump along the boards and call it a hit, while another won't. Repeat that x10 hits per game x82 games per season and you've got a large discrepancy.
I think humans are involved in a lot of hockey statistic gathering. I guess it is a reliability problem, but how does it differ from other metrics?

LiveeviL is offline   Reply With Quote
Old
04-13-2013, 11:49 PM
  #37
Hardyvan123
tweet@HardyintheWack
 
Join Date: Jul 2010
Location: Vancouver
Country: Canada
Posts: 12,093
vCash: 500
Quote:
Originally Posted by CanadianHockey View Post
It's tracked by humans. One guy might see two players bump along the boards and call it a hit, while another won't. Repeat that x10 hits per game x82 games per season and you've got a large discrepancy.
This is the major problem with hockey and stats, prepared to say baseball and football to a lesser degree where plays and their impact can be isolated.

Bob McGowan from the fan in Toronto, whether one likes him or not hit the nail on the head in his hockey arguments book.

To paraphrase him roughly "take away the actual scoring plays in any NHL game and ask people to look at all the other plays in the game and then determine who won the game. This prediction is very random in hockey as scoring chances often go against the flow of the play ect... It's more indicative in baseball and football (all non scoring plays in total) in determining which team probably won the game.

Hockey Prospectus also talks about the high amount of variance or random chance in winning in the NHL today, it's more often than people think or believe or wish it is.

Hockey is always going to lag behind Baseball and even Football in any statistical analysis, and making any conclusions form the data, because of the subjective nature of non scoring plays and their value as well.

Hardyvan123 is offline   Reply With Quote
Old
04-14-2013, 05:34 AM
  #38
doakacola*
 
Join Date: Feb 2009
Posts: 9,231
vCash: 500
[QUOTE=Hardyvan123;63919349]This is the major problem with hockey and stats, prepared to say baseball and football to a lesser degree where plays and their impact can be isolated.

Bob McGowan from the fan in Toronto, whether one likes him or not hit the nail on the head in his hockey arguments book.

To paraphrase him roughly "take away the actual scoring plays in any NHL game and ask people to look at all the other plays in the game and then determine who won the game. This prediction is very random in hockey as scoring chances often go against the flow of the play ect... It's more indicative in baseball and football (all non scoring plays in total) in determining which team probably won the game.

Hockey Prospectus also talks about the high amount of variance or random chance in winning in the NHL today, it's more often than people think or believe or wish it is.

Hockey is always going to lag behind Baseball and even Football in any statistical analysis, and making any conclusions form the data, because of the subjective nature of non scoring plays and their value as well.[/QUOT

Find taking our the goals and evaluating a game a non - starter.

doakacola* is offline   Reply With Quote
Old
04-17-2013, 06:17 AM
  #39
Bad News Bears
MACHADO ABOUT NOTHIN
 
Bad News Bears's Avatar
 
Join Date: May 2009
Location: Australia
Country: Australia
Posts: 3,552
vCash: 623
Quote:
Originally Posted by Richer View Post
Say this with a straight face: "Grabovski is only slightly better than Crosby statistically."

I think THoR needs some work.
"We also note that had Sidney Crosby, who is ranked 15th, played two full seasons he
would have been in the top 10 and possibly in the top 5."

Bad News Bears is offline   Reply With Quote
Old
04-19-2013, 02:34 PM
  #40
schuckers
Registered User
 
Join Date: Feb 2013
Posts: 34
vCash: 500
Update to THoR

For those interested, we've update the THoR model to add a rink effect and score effects. This to reduce bias in the RTSS and in teams changing style of play with the lead.

Latest version of THoR can be found here:
http://www.statsportsconsulting/thor

Some can be found at this link.
http://statsportsconsulting.com/2013/04/19/total-hockey-rating-thor-results-through-041413/


Top 10 in total contribution are
1. ANZE KOPITAR
2. ERIC STAAL
3. JONATHAN TOEWS
4. PATRICE BERGERON
5. LOGAN COUTURE
6. HENRIK SEDIN
7. CHRIS KUNITZ
8. ERIK KARLSSON
9. JOHN TAVARES
10. RYAN GETZLAF


-Schuckers

schuckers is offline   Reply With Quote
Old
04-20-2013, 03:24 PM
  #41
Dilettante
Registered User
 
Dilettante's Avatar
 
Join Date: Oct 2006
Posts: 278
vCash: 500
Quote:
Originally Posted by ChiTownPhilly View Post
I recognize the legal maxim that states "hard cases make bad law," but let me give you an example of one of the most memorable plays I've ever seen in person- and postulate that the metric described above has little way of taking the contribution into account.

Philadelphia v. up-and-coming Ottawa team. Legion of Doom line. Renberg carries in puck from R Wing. Lindros gets deep into the zone, and angles to L side, then cuts behind the net as if preparing to start a cycle. Two defenders give complete attention to Lindros, which leaves Renberg partially open, and John LeClair completely open. Cross-ice pass/score.

Now Lindros had more to do with that goal than any player... and he didn't even touch the puck...

I for Incomplete on the measurement attempt as given above. Needs to return to the shed for some overhaul work---
I think this is a really important point that speaks to the nested nature of hockey data. Using your example, even though LeClair scored the goal and Renberg got the assist, everyone on the ice (including the opposition) had an effect. I imagine Lindros being on the ice significantly increased the probability that his linemates would get a point regardless of whether he recorded an objective stat (i.e. goal, assist, hit, etc.), perhaps looking at his contribution that way might more accurately assess his value?

Dilettante is offline   Reply With Quote
Old
04-21-2013, 08:15 PM
  #42
schuckers
Registered User
 
Join Date: Feb 2013
Posts: 34
vCash: 500
Quote:
I think this is a really important point that speaks to the nested nature of hockey data. Using your example, even though LeClair scored the goal and Renberg got the assist, everyone on the ice (including the opposition) had an effect. I imagine Lindros being on the ice significantly increased the probability that his linemates would get a point regardless of whether he recorded an objective stat (i.e. goal, assist, hit, etc.), perhaps looking at his contribution that way might more accurately assess his value?
If you look at the THoR model, it credits players for their effect in exactly this way.

schuckers is offline   Reply With Quote
Old
04-23-2013, 06:58 PM
  #43
ekespou
Registered User
 
Join Date: Mar 2013
Posts: 3
vCash: 500
Quote:
Originally Posted by schuckers View Post
If you look at the THoR model, it credits players for their effect in exactly this way.
I haven't looked at the model but that absolutely is not what the Sloan conference paper says the model does.

ekespou is offline   Reply With Quote
Old
04-24-2013, 02:55 PM
  #44
schuckers
Registered User
 
Join Date: Feb 2013
Posts: 34
vCash: 500
To quote from the THoR Sloan paper
Quote:
Note that for every event on the ice we will distribute value for that event to all of the players on the ice. This is done to account for the effect of an individual, for example, a Sidney Crosby or a Shea Weber, who may impact events but is not directly involved in a given event such as a shot.

schuckers is offline   Reply With Quote
Old
04-24-2013, 05:37 PM
  #45
Dilettante
Registered User
 
Dilettante's Avatar
 
Join Date: Oct 2006
Posts: 278
vCash: 500
Quote:
Originally Posted by schuckers View Post
If you look at the THoR model, it credits players for their effect in exactly this way.
Well I'll be damned, so it does! Maybe you can help clarify a few things for me:
1) Don't understand why the NP20 value is the probability that an event lead to goal after 20s for the home team minus the probability for the away team. Wouldn't a high NP20 just mean that a player was on ice for more events that led to goals in home games? BTW, I wonder what the results would look like if instead of the NP20 value being home probability - away probability it was goal for probability - goal against probability.
2) Maybe I missed it but was it coded anywhere whether the goal that was scored 20s after the event was for or against for each player on the ice?
3) When you say "The exceptions to this valuation of events are shots, goals and penalties" does it it mean there was a significant probability that a goal was scored 20s after a goal? Wasn't 'goal' the dependent?
4) Do you have any descriptive statistics? For instance, which players were on the ice for the most events that led to goals for or against?

Hope you don't mind the interrogation, I think it's a really interesting paper.

Dilettante is offline   Reply With Quote
Old
04-24-2013, 11:23 PM
  #46
Verviticus
Registered User
 
Join Date: Jul 2010
Posts: 4,812
vCash: 695
Quote:
Originally Posted by schuckers View Post
For those interested, we've update the THoR model to add a rink effect and score effects. This to reduce bias in the RTSS and in teams changing style of play with the lead.
how on earth did you only now add something that takes score effects into account

edit: if this sounds antagonistic, good, because i can't for the life of me fathom how in good faith one could come up with a 'comprehensive ranking of players' and not account for very obvious things like that

Verviticus is offline   Reply With Quote
Old
04-25-2013, 12:21 PM
  #47
Chalupa Batman
Mod Supervisor
 
Chalupa Batman's Avatar
 
Join Date: Sep 2005
Posts: 23,030
vCash: 500
Quote:
Originally Posted by Verviticus View Post
edit: if this sounds antagonistic, good, because i can't for the life of me fathom how in good faith one could come up with a 'comprehensive ranking of players' and not account for very obvious things like that
Don't let "perfect" be the enemy of "good". Trying to build a model with all of the bells and whistles right out of the box is usually a good way to not build anything at all.

And "very obvious things" are all relative. He probably included some things that you wouldn't have thought of.

Chalupa Batman is offline   Reply With Quote
Old
04-25-2013, 12:56 PM
  #48
billybudd
5 Mike Rupps
 
billybudd's Avatar
 
Join Date: Feb 2012
Posts: 9,642
vCash: 500
Quote:
One criticism of THoR from Scott Cullen at TSN can be found here and it is the fact that THoR ignores scoring rate, the rate of shots by a given player that are goals. He validly points out that shots from Steven Stamkos are more likely to go in than shots by Tyler Kennedy. True. However, we know that shooting rates fluctuate a good deal over short periods like over a season. Further, we are not just counting shots by Stamkos and Kennedy in their value but the shots both taken by their teammates and opponents when they are on the ice. We’ll continue to assess this further and potentially tweak THoR. But when we tested this idea in the past it lead to much greater fluctuation in estimated player value
So the long and short of it is that you're using Corsi as a significant aspect of these ratings. Corsi's only valuable when two teams (or ten total players) have the same objective. An objective like (oversimplified): get the puck into the offensive zone, try to get it to someone with a clean look at the net then direct it toward net.

Corsi gives misleading values for Kennedy because that's not his objective. His objective is: get the puck into the offensive zone, shoot (regardless of where he is, what the angle is, what the goalie's doing and what's between him and the net).

Some of Corsi's critics have asked "well, what happens if a team just throws the puck on net from anywhere...wouldn't that game the corsi stat?" Tyler Kennedy's consistently high values suggest that it absolutely does.

For a visual illustration of this, check out the clusters and shot volumes among these two players with similar Corsi, using distance as a rough proxy for shot quality

Good shots

Toews
81 shots

http://somekindofninja.com/nhl/index...&search=Search


Kennedy
38 shots

http://somekindofninja.com/nhl/index...&search=Search


bad shots

Toews
32 shots

http://somekindofninja.com/nhl/index...&search=Search


Kennedy
108 shots

http://somekindofninja.com/nhl/index...&search=Search


Corsi counts these as if they're the same. They're not (and I assume the differences would only get more pronounced if you were to factor in missed and blocked shots), which is why it rates Kennedy as an equivalent possession player to Jonathan Toews, even though anyone with eyes can see that idea's a joke.

Edit: updated. accidentally had it sorted to away games only


Last edited by billybudd: 04-25-2013 at 01:10 PM.
billybudd is online now   Reply With Quote
Old
04-25-2013, 01:42 PM
  #49
schuckers
Registered User
 
Join Date: Feb 2013
Posts: 34
vCash: 500
Quote:
Originally Posted by Verviticus View Post
how on earth did you only now add something that takes score effects into account

edit: if this sounds antagonistic, good, because i can't for the life of me fathom how in good faith one could come up with a 'comprehensive ranking of players' and not account for very obvious things like that
We have been planning to add score effects for some time but they didn't make it into the paper for MIT Sloan. It is and will continue to be a work in progress.

Here's a link to the slides for the conference prepared in advance that mentions score effects.
http://www.sloansportsconference.com/?p=10193

schuckers is offline   Reply With Quote
Old
04-25-2013, 01:56 PM
  #50
schuckers
Registered User
 
Join Date: Feb 2013
Posts: 34
vCash: 500
Quote:
Originally Posted by Dilettante View Post
Well I'll be damned, so it does! Maybe you can help clarify a few things for me:
1) Don't understand why the NP20 value is the probability that an event lead to goal after 20s for the home team minus the probability for the away team. Wouldn't a high NP20 just mean that a player was on ice for more events that led to goals in home games? BTW, I wonder what the results would look like if instead of the NP20 value being home probability - away probability it was goal for probability - goal against probability.
2) Maybe I missed it but was it coded anywhere whether the goal that was scored 20s after the event was for or against for each player on the ice?
3) When you say "The exceptions to this valuation of events are shots, goals and penalties" does it it mean there was a significant probability that a goal was scored 20s after a goal? Wasn't 'goal' the dependent?
4) Do you have any descriptive statistics? For instance, which players were on the ice for the most events that led to goals for or against?

Hope you don't mind the interrogation, I think it's a really interesting paper.
Not a problem.
1. NP20=P(home goal in 20s )-P(away goal in 20s) ~ home players - away players + other stuff. If NP20 is negative that gives extra value to away players. The key is coding home players positive and away players as negative to mirror the structure of NP20.

2. Absolutely.

3. For penalties we took not prob of goal in first 20 seconds of PP but average PP rate for the length of the penalty. For shots and goals we took P(Shot=Goal) + next 20 seconds.

4. We haven't done much of those as we've focused on building the model. I would think they could be found at stats.hockeyanalysis.com

schuckers is offline   Reply With Quote
Reply

Forum Jump


Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -5. The time now is 12:34 PM.

monitoring_string = "e4251c93e2ba248d29da988d93bf5144"
Contact Us - HFBoards - Archive - Privacy Statement - Terms of Use - Advertise - Top - AdChoices

vBulletin Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
HFBoards.com is a property of CraveOnline Media, LLC, an Evolve Media, LLC company. ©2014 All Rights Reserved.