HFBoards Estimating Player Contribution in Hockey with Regularized Logistic Regression
 User Name Remember Me? Password
 Register FAQ Members List Search Today's Posts Mark Forums Read
 Mobile Hockey's Future Become a Sponsor Site Rules Support Forum vBookie Page 2
 By The Numbers Hockey Analytics... the Final Frontier. Explore strange new worlds, to seek out new algorithms, to boldly go where no one has gone before.

# Estimating Player Contribution in Hockey with Regularized Logistic Regression

06-17-2013, 01:05 PM
#26
Registered User

Join Date: Nov 2007
Posts: 10,971
vCash: 500
Evaluations

Quote:
 Originally Posted by Fugu I agree. This is exactly the argument I use against expanding ice size. People assume that defenders will abandon their posts and just skate around the perimeter with the forwards, although at least in transition there is more room for better skaters to take advantage of the situation. This would just lead to tougher defensive systems from coaches (like the neutral zone trap). You're right that the shots need to be qualified. Specifically with Datsyuk, I'd like to hear what people think about his stats-- using the exact same criteria for all players mentioned, why is he standing out?
Old hockey metric - Puck Possession Times combined with modern video techniques and evaluations.

Going back to the origins of hockey, there is a simple truism, a player on the ice has possession of the puck or he does not. Viewers of HNIC during the playoffs have been fed tidbits by the talking heads telling them that player "X" had the puck for A in the defensive zone, B in the neutral zone, C in the offensive zone.

Each Puck Possession is interpreted as a touch. Each touch by each player is evaluated from start to finish. Subtracting the Puck Possession Time from the TOI yields non Puck Possession Times. The players efforts without the puck (efforts to regain possession) are also evaluated. Then totaled and voila, a player like Pavel Datsyuk rates near the top regardless of the metrics that are floating around.

Comment about evaluations. SH% metric or the SV% metric blend shots. Evaluations do not. A 25 foot shot taken by the same player at different times would likely produce different evaluations especially when all factors from choice to execution,are considered. Same applies for every other aspect measured, passes, saves for goalies, handling the puck, etc.

06-17-2013, 01:10 PM
#27
Doctor No
Retired

Join Date: Sep 2005
Posts: 23,351
vCash: 500
Quote:
 Originally Posted by Canadiens1958 Comment about evaluations. SH% metric or the SV% metric blend shots. Evaluations do not. A 25 foot shot taken by the same player at different times would likely produce different evaluations especially when all factors from choice to execution,are considered. Same applies for every other aspect measured, passes, saves for goalies, handling the puck, etc.
Agreed. Now, how do you hold that all in your head when comparing players? (And how do you account for biases that would happen in evaluation, such as an evaluator being at ice-level for one shot, but in the 200 level for another, or the evaluator's mood changing from day to day influencing his/her views, and things like that)?

(Bottom line - first-hand evaluations are important, and no one who should be taken seriously claims otherwise. However, numerical techniques are also important, and anyone who exclusively uses one or the other is missing a significant part of the picture. Let's not derail this thread to too large of a degree - please use the stickied thread to talk specifically about the merits of scouts vs. stats.)

06-17-2013, 02:07 PM
#28
Doctor No
Retired

Join Date: Sep 2005
Posts: 23,351
vCash: 500
Quote:
 Originally Posted by Fugu Specifically with Datsyuk, I'd like to hear what people think about his stats-- using the exact same criteria for all players mentioned, why is he standing out?
Without seeing the coefficients and formula applied on a player-by-player basis, it's hard to speculate specifically about Datsyuk. It's possible that Datsyuk gets an inordinate share of the credit in these situations, which could be related to different sets of luck among line pairings. I don't know enough about Pavel to know the number of unique line combinations for him.

It's also possible that (even strength, which is what is being measured) he truly is more relevant than the other stars.

06-17-2013, 02:23 PM
#29
Fugu
HFBoards

Join Date: Nov 2005
Location: Pac NW
Country:
Posts: 29,621
vCash: 500
Quote:
 Originally Posted by Taco MacArthur Without seeing the coefficients and formula applied on a player-by-player basis, it's hard to speculate specifically about Datsyuk. It's possible that Datsyuk gets an inordinate share of the credit in these situations, which could be related to different sets of luck among line pairings. I don't know enough about Pavel to know the number of unique line combinations for him. It's also possible that (even strength, which is what is being measured) he truly is more relevant than the other stars.
Obviously I'm too biased to truly comment on the results, but after watching him for his entire career, I feel he possesses the puck far more than anyone I've seen, and if he doesn't have it, he can disrupt most efforts by the opposition. I've felt he's the best overall player in the league for several years now.

 06-17-2013, 04:54 PM #30 Cunneen Registered User   Join Date: May 2013 Posts: 94 vCash: 500 Wow, I missed a lot since my last comment. The point I am making is not that we should use shot based analysis only. Rather it that we need to account for variance in shooting percentages. So my way of doing it (as well as Eric T's prefered way) is to start with shot based analysis and then regress SH%. Fournier, you point out that regressing towards the NHL mean shooting percentage is not ideal, since we know that some players are better shooters than others. This is of course true, and there are ways in which our regression can be refined. Eric talks more about this in this article http://www.broadstreethockey.com/201...gression-goals And there are many other variables that can be accounted for besides the ones that Eric looks at in that article. Competition, usage, teamates, and goaltending are the four obvious variables that a very smart person could tinker with and isolate from the talent in SH%. Let's remember that even if we regressed every player's SH% towards the league average SH%, our predicative power would still be great using these new regressed Shooting Percentage numbers combined with shot differentials compared to using strictly goal rates. But we would be foolish to not refine our regression and be more precise. Taco, your point is spot on. A player's SH% regression should be refined based on sample size, usage, teamates, competition, etc. We can't be regressing everything to the league average SH%
06-17-2013, 05:35 PM
#31
Cunneen
Registered User

Join Date: May 2013
Posts: 94
vCash: 500
Quote:
 Originally Posted by Fugu Obviously I'm too biased to truly comment on the results, but after watching him for his entire career, I feel he possesses the puck far more than anyone I've seen, and if he doesn't have it, he can disrupt most efforts by the opposition. I've felt he's the best overall player in the league for several years now.
Among all players in NHL who have played over 2000 minutes in the past 5 NHL seasons, Datsyuk has the highest Corsi For % in the NHL at 58.2%.

Datsyuk's relative Corsi since 2007

2007-2008: 16.3

2008-2009: 11

2009-2010: 9.9

2010-2011: 9.1

2011-2012: 13.2

2012-2013 : 11.6

He has done this even while playing tough minutes (Rel Corsi QoC of over .5 every year and over 1 three of the 6 years).

This is evidence that Datsyuk is one of the best puck possession players in the NHL in recent years. He drives possession forward at an amazing rate, even while playing top players on the opposing team. He has probably been the best (and if not, one of the best) player in the NHL as a whole for the past 6 seasons

06-17-2013, 09:29 PM
#32
hatterson
Global Moderator

Join Date: Apr 2010
Location: North Tonawanda, NY
Country:
Posts: 11,409
vCash: 874
Quote:
 Originally Posted by Cunneen Let's remember that even if we regressed every player's SH% towards the league average SH%, our predicative power would still be great using these new regressed Shooting Percentage numbers combined with shot differentials compared to using strictly goal rates. But we would be foolish to not refine our regression and be more precise.
Ultimately I think this is the biggest piece missing from this analysis, it's predictive power has not yet been established. Although I don't believe that was the author's goal as much as to provide the framework for the analysis and have others see if it truly means anything (a common approach in academia)

The two questions I believe are still outstanding that I apply to basically all metrics:
1.) How correlated are the results from a given chunk of time (say the 4 years in the study) to another chunk of time (say the 1.5 years since)?
2.) If the results do show a correlation, how predictive are they of a teams overall success if a team either has a large number of positive producers or negative producers?

Basically 1 is asking, is the metric even statistically relevant. If past performance under this metric cannot predict future performance then I really don't care about it.

2 is asking, if the metric can derive future performance from past performance, how well does it predict team performance based upon the summation of individual performance. If having a team full of high performers under this metric doesn't correlate with winning then again, I don't care.

Again, I don't believe that this is a failure of the authors by any stretch, it was simply out of scope of their paper and I look forward to future analysis to determine if this method is truly valuable.

__________________
Come join us on the By The Numbers forum. Take a look at our introduction post if you're new. If you have any questions, feel free to PM me.

Last edited by hatterson: 06-17-2013 at 09:34 PM.

06-17-2013, 09:41 PM
#33
Cunneen
Registered User

Join Date: May 2013
Posts: 94
vCash: 500
Quote:
 Originally Posted by hatterson Ultimately I think this is the biggest piece missing from this analysis, it's predictive power has not yet been established. Although I don't believe that was the author's goal as much as to provide the framework for the analysis and have others see if it means anything (a common approach in academia) The two questions I believe are still outstanding that I apply to basically all metrics: 1.) How correlated are the results from a given chunk of time (say the 4 years in the study) to another chunk of time (say the 1.5 years since)? 2.) If the results do show a correlation, how predictive are they of a teams overall success if a team either has a large number of positive producers or negative producers? Basically 1 is asking, is the metric even statistically relevant. If past performance under this metric cannot predict future performance then I really don't care about it. 2 is asking, if the metric can derive future performance from past performance, how well does it predict team performance based upon the summation of individual performance. If having a team full of high performers under this metric doesn't correlate with winning then again, I don't care. Again, I don't believe that this is a failure of the authors by any stretch, it was simply out of scope of their paper and I look forward to future analysis to determine if this method is truly valuable.
by this analysis, do you mean mine? or the papers? Because much work has been done to show the predicative power or lack of predictive power of SH%, goal rates, and goal rates adjusted with regressed shooting percentages. (hint, goal rates with regressed shooting percentages are far more predictive than normal goal rates).

06-17-2013, 09:47 PM
#34
Cunneen
Registered User

Join Date: May 2013
Posts: 94
vCash: 500
Quote:
 Originally Posted by hatterson Ultimately I think this is the biggest piece missing from this analysis, it's predictive power has not yet been established. Although I don't believe that was the author's goal as much as to provide the framework for the analysis and have others see if it truly means anything (a common approach in academia) The two questions I believe are still outstanding that I apply to basically all metrics: 1.) How correlated are the results from a given chunk of time (say the 4 years in the study) to another chunk of time (say the 1.5 years since)? 2.) If the results do show a correlation, how predictive are they of a teams overall success if a team either has a large number of positive producers or negative producers? Basically 1 is asking, is the metric even statistically relevant. If past performance under this metric cannot predict future performance then I really don't care about it. 2 is asking, if the metric can derive future performance from past performance, how well does it predict team performance based upon the summation of individual performance. If having a team full of high performers under this metric doesn't correlate with winning then again, I don't care. Again, I don't believe that this is a failure of the authors by any stretch, it was simply out of scope of their paper and I look forward to future analysis to determine if this method is truly valuable.
But if you mean by the paper's analysis, then i agree with you. The paper is backward looking, rather than forward looking, which is a very dangerous view sometimes

06-17-2013, 10:13 PM
#35
hatterson
Global Moderator

Join Date: Apr 2010
Location: North Tonawanda, NY
Country:
Posts: 11,409
vCash: 874
Quote:
 Originally Posted by Cunneen by this analysis, do you mean mine? or the papers? Because much work has been done to show the predicative power or lack of predictive power of SH%, goal rates, and goal rates adjusted with regressed shooting percentages. (hint, goal rates with regressed shooting percentages are far more predictive than normal goal rates).
Quote:
 Originally Posted by Cunneen But if you mean by the paper's analysis, then i agree with you. The paper is backward looking, rather than forward looking, which is a very dangerous view sometimes
I mean the analysis in the paper.

Basically is this version of adjusted plus/minus (which is fundamentally what this is) better than other attempts to adjust plus/minus data.

06-17-2013, 11:45 PM
#36
Patman
Registered User

Join Date: Feb 2004
Posts: 320
vCash: 500
Quote:
 Originally Posted by Taco MacArthur Some thoughts on the paper... It's true that regression to the mean is a real thing with shooting percentage, but as Fourier pointed out (and I'm going to state it differently, so please correct me if you disagree) each player is different (and is going to have their own long-term "real" shooting percentage that we'll never know). This would further be modified by the situations that a player is placed in and the teammates he plays with. There's also a significant amount of small sample variability, which would regress to a player/situation's "mean" over the long term, but to say that everything regresses to the same mean is throwing the baby out with the bathwater. The paper essentially drives towards a metric more refined than CORSI (which itself is a more refined version of plus-minus, although Jim Corsi never intended it as such when he developed it). I like the logistic Bayesian approach that they use - we've done similar techniques in predictive modeling efforts, and I'm typically happy with the results. They also ignore PP/SH/OT goals in the analysis (although they acknowledge that the modeling approach could handle it) - this is a minor flaw in my mind, not a major one (and in that sense, they measure exactly what they say that they're going to measure, and don't hide that fact - it's a proxy for even-strength performance). The use of a Laplacian prior distribution to guard against overfitting their solution is an interesting one (and something that I hadn't considered before). I want to think more about this - it isn't immediately intuitive to me - but it seems to work well. A lot of these modeling approaches rely upon the shot (and other data) from the league - while I prefer these approaches in general, they do suffer from a lack of consistency in recording, and until those kinks are controlled for, I like that the authors here don't rely upon it. I think that each approach could benefit from the other, and that there will be a meeting in the middle at some point. I'm not sure that I like how they incorporate goaltenders into the mix; traditionally, goaltenders haven't been measured by plus-minus well. On the other hand, you could make the same argument that forwards and defensemen have different roles on the ice, and to that end, they simply let the model fit to what was there. I'd like to see the full breadth of data to analyze, to see if there is a bias in the results towards/against certain positions or types of players (such as defensive forwards). In general, that's my least favorite part of the analysis - you can't see the calculations or the player-level results. Regarding the "value for money" concept, I disagree with using "average" as the baseline (if I truly understand what was done here) - average players definitely have positive value; this is borne out both by empirical evidence in terms of what teams offer average players, and in terms of certain teams' failings for the lack of such averageness at critical positions/times. I've been exploring doing something similar to the MCMC approach, but using an agent-based (complex systems) approach to player evaluation. The authors note that a more realistic scenario would be looking at a game as a series of Poisson processes (with lambda based upon who was on the ice), and I believe that this would fall out readily from a CS-based approach.
I have been wanting to run a model along the lines of...

log(rate)=OFFTeamPlayerA+...+OFFTeamPlayerF-DEFTeamPlayerA-...-DEFTeamPlayerF+PrevailingGoalRate

along with appropriate off-set adjustment and adjustments for number of skaters on ice.

There will be SERIOUS issues with this model with MCMC and the rest and its likely I'd persue a random effects type model. That's just going to be the way of life as there's no real way to handle sampling of all the player terms as a vector. One conditioned on the rest all the way through.

That being said, this doesn't rule out that this could compute fast-ish as each conditional step would be a log-convex distribution and thus an adaptive envelope can be placed given fast sampling (Gilks, 1992). MCMC methods tend to be high correlation monte carlo methods. One way around it, go very large. There's also an issue in regards to the fact that all the contrasts are dependent save any action from a prior used in the Bayesian setting. That is, players are only relative to each other... same as teams.

Now is this going to be "right"... of course not, but I think it'll take things further than before.

Of course, this shouldn't really influence the notionals of the play of the game. It'll be more interesting to see what it draws out. On the other hand, large variances will be unavoidable.

---

edit: For the record i haven't read the paper. quick googling says this is in the machine learning field so I suspect there may be some dimension reducing techniques of some variety.

edit #2: Poisson process is usually the easiest way to go. Frankly this is unforsakenly complex to start. I'm unfamiliar with agent-based models, I've been told by our local expert there isn't really a core. No fundamental law of _____. I don't have any interest into going into it as such. Of course, I think at heart this is stuff I might have REALLY been interested in as I was hoping my studies would lead me into things that behave dynamically.

Nevertheless, my understanding is that agent models and statistics are in some ways difficult to square as the paradigms are wholly different. One is data in search of a model, the other a model in search of data... and never the two shall meet.

Last edited by Patman: 06-17-2013 at 11:55 PM.

 06-18-2013, 06:37 AM #37 schuckers Registered User   Join Date: Feb 2013 Posts: 39 vCash: 500 Patman, Take a look at Brian Macdonald's work on expected goals with a model similar to the one you're proposing....
06-18-2013, 09:28 AM
#38
VinnyC
vancity, c-bus, 'peg

Join Date: May 2011
Location: Na'ē panjā
Country:
Posts: 10,769
vCash: 500
Quote:
 Originally Posted by Cunneen by this analysis, do you mean mine? or the papers? Because much work has been done to show the predicative power or lack of predictive power of SH%, goal rates, and goal rates adjusted with regressed shooting percentages. (hint, goal rates with regressed shooting percentages are far more predictive than normal goal rates).
Could you link to some articles that explore the matter? I'm trying to learn a bit more about analytics.

06-18-2013, 09:55 AM
#39
Cunneen
Registered User

Join Date: May 2013
Posts: 94
vCash: 500
Quote:
 Originally Posted by VinnyC Could you link to some articles that explore the matter? I'm trying to learn a bit more about analytics.

06-18-2013, 05:46 PM
#40
VinnyC
vancity, c-bus, 'peg

Join Date: May 2011
Location: Na'ē panjā
Country:
Posts: 10,769
vCash: 500
Quote:
 Originally Posted by Cunneen
Cheers

Forum Jump