HFBoards

Go Back   HFBoards > General Hockey Discussion > By The Numbers
Mobile Hockey's Future Become a Sponsor Site Rules Support Forum vBookie Page 2
Notices

By The Numbers Hockey Analytics... the Final Frontier. Explore strange new worlds, to seek out new algorithms, to boldly go where no one has gone before.

The technical value of a pick

Reply
 
Thread Tools
Old
06-21-2016, 09:55 AM
  #1
sfan
Registered User
 
Join Date: Jun 2013
Location: Ottawa
Country: Canada
Posts: 533
vCash: 500
The technical value of a pick

Hi folks, I'm very interested in you questions, feedback and suggestions on an analysis I have done on the success of OHL draft picks. The main post and PDF can be found here on the OHL Trades thread.

Also, what other work has been done in the past to determine the relative value of traded picks?

Thanks!

sfan is offline   Reply With Quote
Old
06-27-2016, 07:42 PM
  #2
sfan
Registered User
 
Join Date: Jun 2013
Location: Ottawa
Country: Canada
Posts: 533
vCash: 500
Forgive me for pinging this thread one last time in the hopes of constructive feedback and suggestions. This time I'll cross-post the content here, originally from the OHL Trades 2016-17 Season Thread, both below and attached (pdf).

I realize most here are primarily interested in the NHL and this analysis is based on OHL, but the objective of the analysis is not league or level dependent. Generally, I'm curious about exploring ways to evaluate GM decision making over time, analyzing the historical performance of trades and draft picks. To accomplish this I am building a database of historical draft picks and the subsequent success of those picks.

Quote:
Originally Posted by sfan
Technical value of draft picks, Forwards and Defense
I have updated my initial experimental analysis to include all skaters (not goalies yet) from the 2007 through to 2011 drafts and their entire OHL careers. The methodology is as described below. Some notes:
- A total of 874 forwards and 487 defensemen were analysed.
- Very remarkably, there was effectively no statistical PPG difference in the performance of the five season cohorts of 138 forwards playing more than 100 OHL career games that were drafted in rounds 2 through 5. Each round-cohort ranged between an average of .51 to .57 PPG with a standard deviation of between .26 and .28 PPG.
- The only relevant value difference between pick in rounds 2 through 5 is the declining probability that a forward picked in a later round would make roster and exceed 100 career games. 79% of round 2 forward picks exceeded 100 OHL career games and 42% of round 5 picks exceed 100 GP.
- I realize that PPG is an very incomplete performance metric for defensemen but I do not have anything else I can easily use. Be that as it may, it is a relevant metric that also somewhat reflects player time on ice and power play utilization, in addition to just point production.
- The average PPG and standard deviation for defensemen were also consistent for cohorts drafted in rounds 3 to 5, however 2nd round defensemen were markedly stronger and very close to the value of first round defense picks.

Again, I'd appreciate and suggestions to improve this analysis. I'd also like to know what minimum threshold of goaltender games played would indicate the value of their picks. Provisionally, I am looking at metrics like a minimum of 2 seasons, 40 games played and no less than 20 games per year.
Quote:
Originally Posted by sfan View Post
Iíve been curious about the value of picks in trades. Obviously, the first rule of value is that in any specific transaction it is determined by the price that a seller and buyer agree upon. Both may have specific reasons (asset needs or surplus) to go above or below a technical value. Furthermore, players have differing intangible skills or weaknesses that can affect their trade value.

This said, it seems to me that future picks should have a pretty straight-forward technical value based on the probability that a given pick in a given round results in a roster worthy productive player. I will limit this exercise to forwards, perhaps the easiest position to assess with simple stats. Here is the approach I took:
  1. Scraped data from the 2008, '09 and '10 OHL drafts (this allowed me to examine stats from their entire OHL career)
  2. Identified, by round, the forwards drafted then the subset of forwards that had OHL careers more than 100 games
  3. Aggregated their total points, games, and points per game
A summary of the data can be downloaded below (PDF; login required).

With this limited data set, here is what the numbers roughly tell in terms of the value of a pick:
  • About 100% of first rounders have full OHL careers with .85 points per game
  • About 2/3rds of second and third rounders become solid OHLers with an expected average career production of about .6 pts per game
  • About 1/3rd of fourth and fifth rounders become solid OHLers with an expected average career production of about .5 pts per game
  • About 1/5th of players drafted after the 6th round become solid OHLers with an expected average career production of about .5 pts per game
Additional notes and comments:
  • More years need to be analyzed as the sample size for 4th and 5th rounders is rather small.
  • People smarter than me could characterize the performance variation among players drafted within 'round cohorts'. Looking at the player details visually the difference is substantial.
  • The average PPG performance difference between many 1st and 4th rounders was not that great, yet it is subjectively evident that most first rounders get substantially higher quality and quantity of time on ice. How much of this PPG delta is attributable skill and how much is context?
  • Would a comparable analysis for defenders (games played and PPG) be similarly relevant?
  • Evaluating goalies would be more of a hassle as the data can't be as easily scraped with my limited manual skills
.

Any comments, questions, suggestions?

Quote:
Originally Posted by sfan View Post
Hi folks, I'm very interested in you questions, feedback and suggestions on an analysis I have done on the success of OHL draft picks. The main post and PDF can be found here on the OHL Trades thread.

Also, what other work has been done in the past to determine the relative value of traded picks?

Thanks!
Attached Files
File Type: pdf OHL 2007-11 Draft Analysis 2016.06.19b.pdf‎ (179.0 KB, 11 views)

sfan is offline   Reply With Quote
Old
06-28-2016, 12:51 AM
  #3
Connor McDaigle
Eetu, Brute?
 
Connor McDaigle's Avatar
 
Join Date: Mar 2015
Posts: 1,036
vCash: 289
I has originally read this back when you first posted it, but I had no questions to ask, since the analysis is sound. But I have a few now after reading again.

What conclusions do you draw from this? What sort of pick trades are fair value? How many 4ths equal a 2nd? How different will the results be when dealing with the NHL draft?

Connor McDaigle is online now   Reply With Quote
Old
06-28-2016, 07:02 AM
  #4
sfan
Registered User
 
Join Date: Jun 2013
Location: Ottawa
Country: Canada
Posts: 533
vCash: 500
Thanks for reaching out!
Quote:
Originally Posted by Connor McDaigle View Post
I has originally read this back when you first posted it, but I had no questions to ask, since the analysis is sound. But I have a few now after reading again.

What conclusions do you draw from this? What sort of pick trades are fair value? How many 4ths equal a 2nd? How different will the results be when dealing with the NHL draft?
I'll respond specifically in the case of picks used for forwards. A similar approach can be taken for defense. Goalies are a more complex case as mentioned in my notes. I have some ideas for a pick performance metric that integrates my currently distinct forward and defense analysis.

Some provisional conclusions:
1) Other than elite 1st rounders, which upwardly skew that cohort's mean PPG, picks in rounds 2 to 5 produce successful (defined by more than 100 GP) forwards with the essentially same average PPG.

2) I think we can ignore in-round variability in pick performance because the PPG standard deviations are very similar.

3) The probability-adjusted likelihood of a forward pick PPG are as follows:
Round 1 - 0.82
Round 2 - 0.44
Round 3 - 0.40
Round 4 - 0.32
Round 5 - 0.21
Round 6+ - 0.08
The relative value of the picks are now simple to derive. Two 2nd or third round picks rounds are roughly equal to a 1st round. 2nd and 3rd round picks are effectively equal in value. Two 5th round picks are equal in value to a 2nd or 3rd round pick. After the 5th round, picks become more of a crap shoot.
Doing this for junior hockey is certainly easier than for NHL. Maximum junior hockey careers are 4-5 years and most players that make roster do so by their second season of eligibility. Thus the time frame and data set needed to evaluate the value of a pick is much smaller than the NHL. Also, with richer data and automated analytic analysis, a more nuanced evaluation of what constitutes a successful pick is possible.

sfan is offline   Reply With Quote
Old
06-28-2016, 03:57 PM
  #5
FanCos
Rookie User
 
Join Date: Jun 2016
Posts: 3
vCash: 500
Hi, I hope I do not sound too harsh, but wanted to point out something that might make you reconsider your approach.

What you producded here is something called endogenous selection effect. From the draftees in the later rounds, you are effectively picking those who turned out to be good players. As such, this confounds your analysis of the expected value of a draft pick considerably.

Ex ante, the teams do not know which prospects will turn out to be 100+ games OHL players. So you are involded here in ex post analysis, where you actually compute some sort of conditional probability: suppose that given a player passes the threshold of 100 games, what will be his ppg.

Of course, that particular sample of draft picks which turns out to record more than 100 games essentially consists of good players irrespective of the draft round - now they would not otherwise play more than 100 games, right? The problem is, the teams do not have but partial information which ones, and that's why they are spread around multiple draft rounds. This restricted sample is also reason you do not find statistically significant difference between later draft rounds.

**So to compute the expected value of a draft pick, you have to feature in the drecreasing probability that a player will be actually good enough to play more than 100 games.

**EDIT: Sorry, I did not read the last post in the thread properly before replying. However, what I said about the lack of statistically significant results between rounds is still correct.


Last edited by FanCos: 06-28-2016 at 04:32 PM. Reason: Did not read the last post
FanCos is offline   Reply With Quote
Old
06-28-2016, 05:39 PM
  #6
sfan
Registered User
 
Join Date: Jun 2013
Location: Ottawa
Country: Canada
Posts: 533
vCash: 500
Quote:
Originally Posted by FanCos View Post
Hi, I hope I do not sound too harsh, but wanted to point out something that might make you reconsider your approach.

What you producded here is something called endogenous selection effect. From the draftees in the later rounds, you are effectively picking those who turned out to be good players. As such, this confounds your analysis of the expected value of a draft pick considerably.

Ex ante, the teams do not know which prospects will turn out to be 100+ games OHL players. So you are involded here in ex post analysis, where you actually compute some sort of conditional probability: suppose that given a player passes the threshold of 100 games, what will be his ppg.

Of course, that particular sample of draft picks which turns out to record more than 100 games essentially consists of good players irrespective of the draft round - now they would not otherwise play more than 100 games, right? The problem is, the teams do not have but partial information which ones, and that's why they are spread around multiple draft rounds. This restricted sample is also reason you do not find statistically significant difference between later draft rounds.

**So to compute the expected value of a draft pick, you have to feature in the drecreasing probability that a player will be actually good enough to play more than 100 games.

**EDIT: Sorry, I did not read the last post in the thread properly before replying. However, what I said about the lack of statistically significant results between rounds is still correct.
Thanks for your comment FanCos, I genuinely welcome this kind of feedback :-)

First I will reply to your original objection. If I understand you correctly I think I may have simply failed to explain my intent and method well enough and will try to do so again below.

However if am misunderstanding your objection, please reply with an improved method. I googled endogenous selection effect and bias and, reading parts of your explanation, I admit that I am not sure I fully understand you.

When you said: "So to compute the expected value of a draft pick, you have to feature in the decreasing probability that a player will be actually good enough to play more than 100 games." This is what I have done. I looked at all the picks (F&D) for each round in the 5 years and calculated the % of those players (in each round) that exceeded 100 games. These are the sample set I need to characterize (mean & SD PPG). However I can't assume that all future player drafted will exceed 100 GP because, as we'd expect, players drafted in lower rounds are less likely to make and remain on roster.

So, for example, referring to Round 2 Forward data in the PDF:
57 were drafted ("# Drafted")
45 played over 100 career games ("# > 100GP")
79% of 2nd rounders played over 100 games ("% >100GP"; 45/57)
0.56 is the average PPG of these 45 players ("AvePPG")
0.44 is the Probability Adjusted Pick Value PPG ("PPG Value"; 0.56 x 79% = 0.44)

What I am suggesting this means is that, based on these data, the likely average productive value of future 2nd round forward picks is equal to .44 PPG. Some 2nd round picks will be wasted because of various reasons (won't sign, be good enough, or will quit, etc).

Similarly, the Probability Adjusted Pick Value PPG for a 5th round pick is .21. This method and this number does in fact reflect "the decreasing probability that a (5th round) player will be actually good enough to play more than 100 games."

Now to your revised objection. I do realize I probably shouldn't have used the phrase "statistically significant" because I was using it informally based on the standard deviations and PPG values being quite close and overlapping.

I am speculating that, rather than a methodological limitation to my proposed approach, the reason that forwards that exceeded 100 games have closely matched average & SD PPG is simply that if they didn't perform within that band, they would be simply moved off the roster to make room for the players that can.

Let me know if this response satisfies your objections. Thanks again.


Last edited by sfan: 06-29-2016 at 07:21 PM.
sfan is offline   Reply With Quote
Reply

Forum Jump


Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -5. The time now is 09:55 PM.

monitoring_string = "e4251c93e2ba248d29da988d93bf5144"

vBulletin Copyright ©2000 - 2016, Jelsoft Enterprises Ltd.
HFBoards.com is a property of CraveOnline Media, LLC, an Evolve Media, LLC company. ©2016 All Rights Reserved.