HFBoards

Go Back   HFBoards > NHL Eastern Conference > Atlantic Division > Ottawa Senators
Mobile Hockey's Future Become a Sponsor Site Rules Support Forum vBookie Page 2
Notices

Testing Cost Per Point

Reply
 
Thread Tools
Old
02-07-2014, 06:20 AM
  #1
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Testing Cost Per Point

In the past month or two, Eugene Melnyk has repeated that "cost per point" (CPP) is the most valuable measure of team success. His consistent use of this phrase made me curious about two things. First, where does it come from? And second, is there any substance behind this performance measure?

When I researched the "cost per point" in hockey I discovered that Forbes magazine uses this measure to evaluate team success and to establish which teams are getting the most bang for their buck. Here is a link to a Forbes article that uses cost per point to evaluate which NHL teams are "cost effective":
http://www.forbes.com/sites/prishe/2...ms-of-2011-12/

The way Forbes sets this up is to first figure out cost per point per team (CPP=salary/points), and then calculate the median CPP among the 30 teams (median is the middle score in the pack). It then places each team in relation to the baseline (which is the median CPP) to establish the effiiency ratio (ER=CPP/median). So if the median CPP is $100,000, and your team's CPP is $90,000, your ER is 90%. Note that lower efficiency ratios equal greater efficiency, because you are spending less than average to get a particular result (which is points in the standings). Based on this article, the Sens were the second most cost effective team in the NHL in 2011-12 when using cost per point as the measuring stick (ER=82.2%).

I'm going to get a bit geeky here for a bit, so if you are into stats and data collection please follow along. Also follow along if you are interested in knowing how I got my numbers. Otherwise, you can safely skip the rest of the rest of this post and move on to the next one, where I will present what I found.

The author of the Forbes article drew salary information from USA Today data. That data can be found here:
http://content.usatoday.com/sportsda...salaries/team/

I have a few serious issues with USA Today salary data. If you click on the "About NHL Salaries" tab at the top right of their table, you can find their disclaimers and methods of counting salary. The worst part is they do not properly take player movement into account, and (unless I am mistaken, their writeup is not comprehensive) they essentially add salaries of players with the teams at the end of the year. What ends up happening if you do not pro rate salaries correctly is that the final tallies are not at all accurate. For example, the salary cap for 2011-12 was set at $64.3 MM. However, USA Today has 8 teams above that line, with the biggest spender over $71 MM. The cap floor for 2011-12 was set at $48.3 MM, but 4 teams are listed as being below that figure with the smaller spender under $30 MM. Of course real salary is not the same as cap hit, which explains a part of that. But when you look at the NY Islanders sitting at $29.58MM it is clear there is a problem with the data. For this reason I opted to use CapGeek historical data, which can be found at:
http://www.capgeek.com/archive/?year_id=2011

When you look at CapGeek histocial data, the "spending" category refers to real team salaries that are all accurately pro-rated in instances where there has been player movement.The limitation of using CapGeek data is that they only go back 4 years. But I will take 4 years of good data over 10 years of bad data.

I am going to replace median CPP with mean CPP. The mean average score is calculated by adding the CPP of all teams under consideration, and then dividing by the sum by the number of teams you are looking at. So when I look at CPP for a season, I will add the CPP of all 30 teams and then divide by 30. I am comfortable making this switch because I will have a much lower variance than the author of the Forbes article. The payoff is that I will be able to do more with the data down the road, when I have data for more seasons to work with.

So what practical difference does it make to use mean CPP, and to use CapGeek rather than USA Today data? The range of team salaries presented in USA Today data is much larger than CapGeek data, while the amount of total points eaned in an NHL season is constant. This means the ER will be smaller in the data I used than when is presented in the Forbes article. For example, the Forbes article lists the ER of the 2011-12 Sens as being 82.2%, but when I used the CapGeek data I calculated this number to be 86.5%.

Furthermore, while Forbes places the Sens 2nd in ER for that season, when I ran the numbers the Sens placed 4th in that category.

__________________________________________________ __________________________________________________ __________________________________________________ __________________________________________________ _______

It is now safe for all of you non-geeks to start reading again.


Research question: Is there any substance behind using cost per point as a measure of team performance? Melnyk went so far as to say that is the only meaningful measure but, hyperbole aside, I decided to let the numbers do the talking.

First up I'll present the basic descdriptives for the 4 season period under consideration here. The following table provides some useful descriptives about Cost Per Point sorted by season, including the mean average, standard deviation (SD), and range of values (2012-13 figures are pro-rated). The mean cost per point goes up each year with the rise in the cap, which is expected.

Season Mean Standard Deviation lowest highest
2009-10 584,213 105,405 401,207 931,127
2010-11 596,002 74,375 488,894 788,697
2011-12 649,211 100,796 501,808 943,698
2012-13 682,634 121,341 504,931 937,798

Next is a basic correlation. I ran the correlation only taking seasons 09-10 through 12-13 into account, and it worked out to be 0.358. When I ran the same test with ER and team success, the correlation worked out to be 0.370. Note in both cases the actual correlation would have been negative, because a smaller number indicates higher spending in the first test, and a small number indicates a better ER in the second test. While the correlation between CCP and team success is slightly better than the correlation between spending and team success, this tiny difference could easily be accounted for by error rather than being a real difference.

The next step I used to test whether CPP has any substance as a performance measure is I rank ordered the best through worst ER teams from 09-10 through 12-13 (120 total cases). The results surprised me a bit.

Rank Team ER Team Success Spending
1 2009-10 Coyotes 68.67% Lost in 1st Rnd 29th
2 2012-13 Ducks 73.97% Lost in 1st Rnd 21st
3 2012-13 Blues 74.58% Lost in 1st Rnd 29th
4 2012-13 Black Hawks 75.00% Stanley Cup Champions 5th
5 2009-10 Predators 75.62% Lost in 1st Rnd 28th
6 2009-10 Captals 76.15% Lost in 1st Rnd 18th
7 2011-12 Predators 77.30% Lost in 2nd Rnd 24th
8 2011-12 Blues 77.56% Lost in 2nd Rnd 23rd
9 2012-13 Penguins 79.49% Lost in 3rd Rnd 7th
10 2010-11 Lightening 82.03% Lost in 3rd Round 23rd

The thing that impressed me is that the teams that were the most efficient financially, as a group, had well above average levels of success and none missed the playoffs. Two of the three teams on this list that made it to the final four or better were actually in the top 10 in spending in their particular years, while 7 of 8 teams in the bottom 2/3 of the league in payroll were eliminated in the first or second round. In other words, it is possible that teams that spend efficiently make it to the playoffs more often, but teams that spend more money while spending wisely are the ones that go the furthest

With this in mind I looked at the bottom 10 ER teams to see if a pattern emerged.

Rank Team ER Team Success Spending
111 2012-13 Flames 124.60% Missed Playoffs 16th
112 2011-12 Oilers 125.91% Missed Playoffs 16th
113 2012-13 Flyers 126.96% Missed Playoffs 1st
114 2010-11 Senators 127.80% Missed Playoffs 13th
115 2009-10 Leafs 131.65% Missed Playoffs 6th
116 2010-11 Devils 132.33% Missed Playoffs 1st
117 2012-13 Panthers 136.88% Missed Playoffs 19th
118 2012-13 Lightening 137.38% Missed Playoffs 10th
119 2011-12 Blue Jackets 145.36% Missed Playoffs 14th
120 2009-10 Oilers 159.38% Missed Playoffs 4th

None of the bottom ten teams in ER over the past 4 seasons made it to the playoffs. In fact, only 1 of the bottom 25 ER teams over the past 4 season made it to the playoffs. That is pretty remarkable.

Cost per point and efficiency rating are all about proportions. When you have an expensive team, and that team tanks during the regular season, your CPP will be huge and your ER will be above the mean. The reverse is also true, where if a team spends very little, and has a high level of success during the regular season, the result will be a low CPP and an ER that is below the mean (reminder: low ER is good). These are pretty straightforward. However, there are other possibilities. What if a team spends quite a bit, but they have a fantastic season and finish near the top of the league? That team will also have a low CPP and a low ER. In other words, you can potentially spend piles of cash and still look good using CPP and ER measures.


This final table divides teams from the past four seasons in two categories: those that are at or below average in ER (which is good), and those that are above average in ER (which is bad). Please click on the thumbnail below to expand it to a full sized image of the table.

DPPCrossTab.JPG

When looking at this table, a couple of things really jump out. First off, Teams with above league average CPP are twice as likely as to miss the playoffs, while those with below average CPP are more than four times as likely to make it to the final four in the playoffs. However, when broken down further, teams who are in the top 10 in league spending and who also have below average CPP are still more like to do something in the playoffs (i.e. win at least one round) than teams in the middle third or bottom third in league spending combined. For those of you who want to revisit my post "Testing Whether We Need to Spend to Win" to compare these figures to 6 year data focusing only on spending, it can be found at http://hfboards.hockeysfuture.com/sh....php?t=1595501 ).

So is Melnyk correct about cost per point being the only metric that matters? Sort of. CPP is a very good measure of how well teams spend their money, and teams who do well in CPP clearly get a lot of bang for their buck. This makes it a great performance measure for lower budget teams. However, Melnyk is incorrect about the importance of spending. Teams who spend more still have a greater likelihood of doing something in the playoffs than teams that do not, even when you factor in CPP.

Limitations:
1) I was reduced to 4 years of data for this analysis, which meant I only had 120 cases to work with at this time. As a result, I stuck to crosstabs rather than doing fancier calculations to test mediating and moderating effects. In the future I hope to get more reliable data for other seasons.
2) Team salary data appears to be inconsistent from one source to another. This is frustrating. This analysis is only as good as the data CapGeek provides. I am comfortable with that for now.

StefanW is offline   Reply With Quote
Old
02-07-2014, 06:35 AM
  #2
Hammertyme
Registered User
 
Hammertyme's Avatar
 
Join Date: Jun 2006
Location: Gatineau/Ottawa
Country: Canada
Posts: 955
vCash: 500
Quote:
Originally Posted by StefanW View Post
In the past month or two, Eugene Melnyk has repeated that "cost per point" (CPP) is the most valuable measure of team success. His consistent use of this phrase made me curious about two things. First, where does it come from? And second, is there any substance behind this performance measure?

When I researched the "cost per point" in hockey I discovered that Forbes magazine uses this measure to evaluate team success and to establish which teams are getting the most bang for their buck. Here is a link to a Forbes article that uses cost per point to evaluate which NHL teams are "cost effective":
http://www.forbes.com/sites/prishe/2...ms-of-2011-12/

The way Forbes sets this up is to first figure out cost per point per team (CPP=salary/points), and then calculate the median CPP among the 30 teams (median is the middle score in the pack). It then places each team in relation to the baseline (which is the median CPP) to establish the effiiency ratio (ER=CPP/median). So if the median CPP is $100,000, and your team's CPP is $90,000, your ER is 90%. Note that lower efficiency ratios equal greater efficiency, because you are spending less than average to get a particular result (which is points in the standings). Based on this article, the Sens were the second most cost effective team in the NHL in 2011-12 when using cost per point as the measuring stick (ER=82.2%).

I'm going to get a bit geeky here for a bit, so if you are into stats and data collection please follow along. Also follow along if you are interested in knowing how I got my numbers. Otherwise, you can safely skip the rest of the rest of this post and move on to the next one, where I will present what I found.

The author of the Forbes article drew salary information from USA Today data. That data can be found here:
http://content.usatoday.com/sportsda...salaries/team/

I have a few serious issues with USA Today salary data. If you click on the "About NHL Salaries" tab at the top right of their table, you can find their disclaimers and methods of counting salary. The worst part is they do not properly take player movement into account, and (unless I am mistaken, their writeup is not comprehensive) they essentially add salaries of players with the teams at the end of the year. What ends up happening if you do not pro rate salaries correctly is that the final tallies are not at all accurate. For example, the salary cap for 2011-12 was set at $64.3 MM. However, USA Today has 8 teams above that line, with the biggest spender over $71 MM. The cap floor for 2011-12 was set at $48.3 MM, but 4 teams are listed as being below that figure with the smaller spender under $30 MM. Of course real salary is not the same as cap hit, which explains a part of that. But when you look at the NY Islanders sitting at $29.58MM it is clear there is a problem with the data. For this reason I opted to use CapGeek historical data, which can be found at:
http://www.capgeek.com/archive/?year_id=2011

When you look at CapGeek histocial data, the "spending" category refers to real team salaries that are all accurately pro-rated in instances where there has been player movement.The limitation of using CapGeek data is that they only go back 4 years. But I will take 4 years of good data over 10 years of bad data.

I am going to replace median CPP with mean CPP. The mean average score is calculated by adding the CPP of all teams under consideration, and then dividing by the sum by the number of teams you are looking at. So when I look at CPP for a season, I will add the CPP of all 30 teams and then divide by 30. I am comfortable making this switch because I will have a much lower variance than the author of the Forbes article. The payoff is that I will be able to do more with the data down the road, when I have data for more seasons to work with.

So what practical difference does it make to use mean CPP, and to use CapGeek rather than USA Today data? The range of team salaries presented in USA Today data is much larger than CapGeek data, while the amount of total points eaned in an NHL season is constant. This means the ER will be smaller in the data I used than when is presented in the Forbes article. For example, the Forbes article lists the ER of the 2011-12 Sens as being 82.2%, but when I used the CapGeek data I calculated this number to be 86.5%.

Furthermore, while Forbes places the Sens 2nd in ER for that season, when I ran the numbers the Sens placed 4th in that category.

__________________________________________________ __________________________________________________ __________________________________________________ __________________________________________________ _______

It is now safe for all of you non-geeks to start reading again.


Research question: Is there any substance behind using cost per point as a measure of team performance? Melnyk went so far as to say that is the only meaningful measure but, hyperbole aside, I decided to let the numbers do the talking.

First up I'll present the basic descdriptives for the 4 season period under consideration here. The following table provides some useful descriptives about Cost Per Point sorted by season, including the mean average, standard deviation (SD), and range of values (2012-13 figures are pro-rated). The mean cost per point goes up each year with the rise in the cap, which is expected.

Season Mean Standard Deviation lowest highest
2009-10 584,213 105,405 401,207 931,127
2010-11 596,002 74,375 488,894 788,697
2011-12 649,211 100,796 501,808 943,698
2012-13 682,634 121,341 504,931 937,798

Next is a basic correlation. I ran the correlation only taking seasons 09-10 through 12-13 into account, and it worked out to be 0.358. When I ran the same test with ER and team success, the correlation worked out to be 0.370. Note in both cases the actual correlation would have been negative, because a smaller number indicates higher spending in the first test, and a small number indicates a better ER in the second test. While the correlation between CCP and team success is slightly better than the correlation between spending and team success, this tiny difference could easily be accounted for by error rather than being a real difference.

The next step I used to test whether CPP has any substance as a performance measure is I rank ordered the best through worst ER teams from 09-10 through 12-13 (120 total cases). The results surprised me a bit.

Rank Team ER Team Success Spending
1 2009-10 Coyotes 68.67% Lost in 1st Rnd 29th
2 2012-13 Ducks 73.97% Lost in 1st Rnd 21st
3 2012-13 Blues 74.58% Lost in 1st Rnd 29th
4 2012-13 Black Hawks 75.00% Stanley Cup Champions 5th
5 2009-10 Predators 75.62% Lost in 1st Rnd 28th
6 2009-10 Captals 76.15% Lost in 1st Rnd 18th
7 2011-12 Predators 77.30% Lost in 2nd Rnd 24th
8 2011-12 Blues 77.56% Lost in 2nd Rnd 23rd
9 2012-13 Penguins 79.49% Lost in 3rd Rnd 7th
10 2010-11 Lightening 82.03% Lost in 3rd Round 23rd

The thing that impressed me is that the teams that were the most efficient financially, as a group, had well above average levels of success and none missed the playoffs. Two of the three teams on this list that made it to the final four or better were actually in the top 10 in spending in their particular years, while 7 of 8 teams in the bottom 2/3 of the league in payroll were eliminated in the first or second round. In other words, it is possible that teams that spend efficiently make it to the playoffs more often, but teams that spend more money while spending wisely are the ones that go the furthest

With this in mind I looked at the bottom 10 ER teams to see if a pattern emerged.

Rank Team ER Team Success Spending
111 2012-13 Flames 124.60% Missed Playoffs 16th
112 2011-12 Oilers 125.91% Missed Playoffs 16th
113 2012-13 Flyers 126.96% Missed Playoffs 1st
114 2010-11 Senators 127.80% Missed Playoffs 13th
115 2009-10 Leafs 131.65% Missed Playoffs 6th
116 2010-11 Devils 132.33% Missed Playoffs 1st
117 2012-13 Panthers 136.88% Missed Playoffs 19th
118 2012-13 Lightening 137.38% Missed Playoffs 10th
119 2011-12 Blue Jackets 145.36% Missed Playoffs 14th
120 2009-10 Oilers 159.38% Missed Playoffs 4th

None of the bottom ten teams in ER over the past 4 seasons made it to the playoffs. In fact, only 1 of the bottom 25 ER teams over the past 4 season made it to the playoffs. That is pretty remarkable.

Cost per point and efficiency rating are all about proportions. When you have an expensive team, and that team tanks during the regular season, your CPP will be huge and your ER will be above the mean. The reverse is also true, where if a team spends very little, and has a high level of success during the regular season, the result will be a low CPP and an ER that is below the mean (reminder: low ER is good). These are pretty straightforward. However, there are other possibilities. What if a team spends quite a bit, but they have a fantastic season and finish near the top of the league? That team will also have a low CPP and a low ER. In other words, you can potentially spend piles of cash and still look good using CPP and ER measures.


This final table divides teams from the past four seasons in two categories: those that are at or below average in ER (which is good), and those that are above average in ER (which is bad). Please click on the thumbnail below to expand it to a full sized image of the table.

Attachment 70609

When looking at this table, a couple of things really jump out. First off, Teams with above league average CPP are twice as likely as to miss the playoffs, while those with below average CPP are more than four times as likely to make it to the final four in the playoffs. However, when broken down further, teams who are in the top 10 in league spending and who also have below average CPP are still more like to do something in the playoffs (i.e. win at least one round) than teams in the middle third or bottom third in league spending combined. For those of you who want to revisit my post "Testing Whether We Need to Spend to Win" to compare these figures to 6 year data focusing only on spending, it can be found at http://hfboards.hockeysfuture.com/sh....php?t=1595501 ).

So is Melnyk correct about cost per point being the only metric that matters? Sort of. CPP is a very good measure of how well teams spend their money, and teams who do well in CPP clearly get a lot of bang for their buck. This makes it a great performance measure for lower budget teams. However, Melnyk is incorrect about the importance of spending. Teams who spend more still have a greater likelihood of doing something in the playoffs than teams that do not, even when you factor in CPP.

Limitations:
1) I was reduced to 4 years of data for this analysis, which meant I only had 120 cases to work with at this time. As a result, I stuck to crosstabs rather than doing fancier calculations to test mediating and moderating effects. In the future I hope to get more reliable data for other seasons.
2) Team salary data appears to be inconsistent from one source to another. This is frustrating. This analysis is only as good as the data CapGeek provides. I am comfortable with that for now.

Good interesting data. I think they also need to figure in the ratio of goals for and against to become a better predictor. Cost of NET points using point for - 1/5(5 on 5) points against.


Last edited by Hammertyme: 02-07-2014 at 06:48 AM.
Hammertyme is offline   Reply With Quote
Old
02-07-2014, 07:29 AM
  #3
Caeldan
Moderator
Whippet Whisperer
 
Caeldan's Avatar
 
Join Date: Jun 2008
Country: Canada
Posts: 13,345
vCash: 100
If you haven't - cross post this into the by the numbers forum if you want 'peer review'... Some interesting stuff in there in general.

Caeldan is offline   Reply With Quote
Old
02-07-2014, 07:31 AM
  #4
operasen
Registered User
 
Join Date: Apr 2004
Posts: 4,933
vCash: 500
Actually makes a bit of sense. Very good find. Thanks for the read and the insight.

operasen is offline   Reply With Quote
Old
02-07-2014, 08:26 AM
  #5
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Quote:
Originally Posted by Canuckman44 View Post
Good interesting data. I think they also need to figure in the ratio of goals for and against to become a better predictor. Cost of NET points using point for - 1/5(5 on 5) points against.
Thank you for your comment, it is appreciated. Goals for and against tend to be poor predictors in hockey. The reason is you can win 9-1, and the lose the next six games by one goal. If you try to aggregate goals for and against it looks as though you are coming out ahead when you are not. I can try it (and probably will try it) when I have enough data to put together a model, but I do not hold out a lot of hope that it will add much.

Quote:
Originally Posted by Caeldan View Post
If you haven't - cross post this into the by the numbers forum if you want 'peer review'... Some interesting stuff in there in general.
You would be surprised at how many people here can give excellent peer reviews . I normally only run numbers specific to the Sens so I haven't really given any thought to posting over there, but this one seems general enough to have a larger audience. When I finally track down better data to increase the number of seasons in my sample I will probably cross post it there. Thanks for the suggestion!

Quote:
Originally Posted by operasen View Post
Actually makes a bit of sense. Very good find. Thanks for the read and the insight.
No problem, I am glad you enjoyed it. This one was a surprising amount of work because I had to sort through a lot of different salary information before making a decision on which data to use and how to organize it. A surprising amount of work went into this post, which only had 4 tables or so. If it was not entertaining to me I definitely would have given up in frustration long before finishing.

StefanW is offline   Reply With Quote
Old
02-07-2014, 08:34 AM
  #6
Kellogs
G'night Sweet Prince
 
Kellogs's Avatar
 
Join Date: Dec 2008
Location: Ottawa
Country: Canada
Posts: 3,263
vCash: 500
Nice job with all the work. My comments/questions are below:

Quote:
I am going to replace median CPP with mean CPP. The mean average score is calculated by adding the CPP of all teams under consideration, and then dividing by the sum by the number of teams you are looking at. So when I look at CPP for a season, I will add the CPP of all 30 teams and then divide by 30. I am comfortable making this switch because I will have a much lower variance than the author of the Forbes article. The payoff is that I will be able to do more with the data down the road, when I have data for more seasons to work with.

So what practical difference does it make to use mean CPP, and to use CapGeek rather than USA Today data? The range of team salaries presented in USA Today data is much larger than CapGeek data, while the amount of total points eaned in an NHL season is constant. This means the ER will be smaller in the data I used than when is presented in the Forbes article. For example, the Forbes article lists the ER of the 2011-12 Sens as being 82.2%, but when I used the CapGeek data I calculated this number to be 86.5%.

Furthermore, while Forbes places the Sens 2nd in ER for that season, when I ran the numbers the Sens placed 4th in that category.
Any particular reason for using the mean rather than the median other than the unreliable nature of Forbes'/USA Today's data? By that I mean, did you also run the calculations using CapGeek's numbers using the median rather than the mean for the ER calculation and see what differences there are? I would be interested in seeing the differences between the median and the mean as the cap goes up and less teams are capable of keeping up with the rising cap, leading to lower spending.

Quote:
The mean cost per point goes up each year with the rise in the cap, which is expected.
In your table, the 2010-2011 season really stands out as the highest CPP is significantly lower than in the other three years (where they stay relatively constant). Any theories as to why that is?

Quote:
Next is a basic correlation. I ran the correlation only taking seasons 09-10 through 12-13 into account, and it worked out to be 0.358. When I ran the same test with ER and team success, the correlation worked out to be 0.370.
Are you talking about a linear regression and your R values came out to be .358 and .370? That doesn't seem like a strong correlation. Or am I misunderstanding this comment?

Quote:
The thing that impressed me is that the teams that were the most efficient financially, as a group, had well above average levels of success and none missed the playoffs.

...

None of the bottom ten teams in ER over the past 4 seasons made it to the playoffs. In fact, only 1 of the bottom 25 ER teams over the past 4 season made it to the playoffs. That is pretty remarkable.
Isn't this conclusion implied by the very definition of the efficiency ratio? I'll have more comments on this later.

Quote:
When looking at this table, a couple of things really jump out. First off, Teams with above league average CPP are twice as likely as to miss the playoffs, while those with below average CPP are more than four times as likely to make it to the final four in the playoffs. However, when broken down further, teams who are in the top 10 in league spending and who also have below average CPP are still more like to do something in the playoffs (i.e. win at least one round) than teams in the middle third or bottom third in league spending combined.
Solid work, and your research and conclusions are in line with what I've been saying, in that spending to the cap floor is not necessarily the main cause for the team being out of a playoff spot or not being competitive (at least at the time of the season when things were pretty bad).

I would bet that the same conclusions could be reached if you ran the numbers pre-2005 Lockout. The Sens' ER at the time would have probably been at the top of the league, and much like the data now, and the fact that we had a payroll significantly lower than the other top teams was probably one of the main reasons why those Sens' teams found themselves out of the playoffs in the 1st or 2nd rounds. A lot of the "reasons" that were attributed to the Sens' lack of playoff success (choking, coaching, europeans, lack of grit, goaltending etc.) were completely overblown and ignored the simple fact that you still need to spend a lot of money to win a league championship.

My guess would be that in the pre-salary cap era, fewer teams with a combined low ER and gross salary spending would make the playoffs compared to now since the cap has reduced the disparity between the top and bottom spenders. So in theory, just being efficient in spending should give you a competitive team that can make the playoffs where as prior to 2005, you probably had to be super-efficient to achieve the same.

As an aside, have you thought about approaching the owner of the capgeek website? They used to have the numbers for 2005-2009, and maybe only keep the last four seasons due to limited space to hold all their data. I would hope that he has some sort of archive with all the data, even if it isn't formatted or organized like the current data of the past 4 seasons, it's still something you could work with.

Kellogs is offline   Reply With Quote
Old
02-07-2014, 08:55 AM
  #7
DrEasy
Out rumptackling
 
DrEasy's Avatar
 
Join Date: Oct 2010
Posts: 4,572
vCash: 500
I really skimmed, and this might not make any sense but: CPP and team success look like dependent variables to me (points are what get you to the playoffs). Running a correlation there is a bit weird.

DrEasy is online now   Reply With Quote
Old
02-07-2014, 08:57 AM
  #8
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Quote:
Originally Posted by Kellogs View Post
Any particular reason for using the mean rather than the median other than the unreliable nature of Forbes'/USA Today's data? By that I mean, did you also run the calculations using CapGeek's numbers using the median rather than the mean for the ER calculation and see what differences there are? I would be interested in seeing the differences between the median and the mean as the cap goes up and less teams are capable of keeping up with the rising cap, leading to lower spending.
I will calculate the medians later tonight after my daughter has gone to bed and I can focus on it. The advantage to using medians for the Forbes data was that it is less influenced by outliers that are well above or well below the rest of the sample. They had quite a few of those due to the issues with the data they were using. I picked mean average instead because it opens up possibilities down the road for tests that compare means. I was ok with means because I did not have the same issues with outliers in CapGeek data. It was more planning for the future and saving work down the line than anything specific right now.

Quote:
Originally Posted by Kellogs View Post
In your table, the 2010-2011 season really stands out as the highest CPP is significantly lower than in the other three years (where they stay relatively constant). Any theories as to why that is?
Good catch. I just ran a couple of quick scattergraphs to create a quick visual to see if anything strange is going on, and nothing jumped out. I will take a closer look later on what I can play with the data more. Very interesting question.

Quote:
Originally Posted by Kellogs View Post
Are you talking about a linear regression and your R values came out to be .358 and .370? That doesn't seem like a strong correlation. Or am I misunderstanding this comment?
That was sloppy on my part for not being specific. I just ran Pearson correlations. I wouldn't do a linear regression with such a small number of cases.

Quote:
Originally Posted by Kellogs View Post
Isn't this conclusion implied by the very definition of the efficiency ratio? I'll have more comments on this later.
I'll hold off then until you make your other comments. I look forward to them.

Quote:
Originally Posted by Kellogs View Post
Solid work, and your research and conclusions are in line with what I've been saying, in that spending to the cap floor is not necessarily the main cause for the team being out of a playoff spot or not being competitive (at least at the time of the season when things were pretty bad).
Thank you very much, I am very happy that you found them interesting.

Quote:
Originally Posted by Kellogs View Post
As an aside, have you thought about approaching the owner of the capgeek website? They used to have the numbers for 2005-2009, and maybe only keep the last four seasons due to limited space to hold all their data. I would hope that he has some sort of archive with all the data, even if it isn't formatted or organized like the current data of the past 4 seasons, it's still something you could work with.
I have not thought of this, but I will definitely do it. Thanks for the suggestion. (I snipped out some of your other suggestions to save a bit of room because this is getting long, but I think they are all interesting and worth testing when I have the data to do so.)

StefanW is offline   Reply With Quote
Old
02-07-2014, 08:58 AM
  #9
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Quote:
Originally Posted by DrEasy View Post
I really skimmed, and this might not make any sense but: CPP and team success look like dependent variables to me (points are what get you to the playoffs). Running a correlation there is a bit weird.
Team Success is a dependent variable, CPP is an independent variable.

StefanW is offline   Reply With Quote
Old
02-07-2014, 09:03 AM
  #10
Xspyrit
Registered User
 
Xspyrit's Avatar
 
Join Date: Jun 2008
Location: Montreal, Canada
Country: Monaco
Posts: 17,097
vCash: 1000
But.... 1 playoff win in 77 years!

Xspyrit is offline   Reply With Quote
Old
02-07-2014, 09:30 AM
  #11
Icelevel
+++--++++--=+
 
Icelevel's Avatar
 
Join Date: Sep 2009
Country: Canada
Posts: 16,112
vCash: 500
i'm going to pay more attention to you now. looks good. will read later

Icelevel is offline   Reply With Quote
Old
02-07-2014, 09:32 AM
  #12
Qward
Because! That's why!
 
Qward's Avatar
 
Join Date: Jul 2010
Location: Behind you, look out
Posts: 14,455
vCash: 500
I wish Melnyk never saw Moneyball.

Qward is offline   Reply With Quote
Old
02-07-2014, 09:39 AM
  #13
The Fuhr*
 
Join Date: Feb 2004
Location: Hamilton,Ontario
Country: Canada
Posts: 35,762
vCash: 500
I'd tell Melnyk to spend more money then so the team is higher in the standings.

More money = more points
Team has cap space

The Fuhr* is offline   Reply With Quote
Old
02-07-2014, 11:37 AM
  #14
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Quote:
Originally Posted by Xspyrit View Post
But.... 1 playoff win in 77 years!
Not sure what you mean. But hot dogs and jam, and pony to you too!

Quote:
Originally Posted by Icelevel View Post
i'm going to pay more attention to you now. looks good. will read later
I look forward to reading your comments.

Quote:
Originally Posted by Qward View Post
I wish Melnyk never saw Moneyball.
You and me both. I guess it shouldn't surprise me that the whole cost per point thing is from Forbes, but it did a little. I was hoping that he contracted a consulting firm to run some numbers for him, and that it was based on that. Not that there is anything wrong with Forbes, but it is a very non-specific analysis.

Quote:
Originally Posted by The Fuhr View Post
I'd tell Melnyk to spend more money then so the team is higher in the standings.

More money = more points
Team has cap space
I ran the numbers on that, and teams in the top third in spending have a far better chance of going far in the playoffs. However, teams that do not spend wise populate the bottom of the cost per point rankings and tend to miss the playoffs all together.

I would like to see him spend wisely, but be open to increasing the budget to a point where we can really contend.

StefanW is offline   Reply With Quote
Old
02-07-2014, 11:41 AM
  #15
Icelevel
+++--++++--=+
 
Icelevel's Avatar
 
Join Date: Sep 2009
Country: Canada
Posts: 16,112
vCash: 500
did anyone else see the comments melnyk made a while ago (before these latest moneyball comments) about how he was well aware of the statistical history that teams that spend more are more successful in the playoffs?

i've been looking for it since i read it but can't find it now. anyone have a link to it?

Icelevel is offline   Reply With Quote
Old
02-07-2014, 11:42 AM
  #16
Sens Mile
Registered User
 
Sens Mile's Avatar
 
Join Date: Sep 2008
Posts: 4,103
vCash: 500
Quote:
Originally Posted by Qward View Post
I wish Melnyk never saw Moneyball.
Thats funnt

Sens Mile is offline   Reply With Quote
Old
02-07-2014, 11:51 AM
  #17
Qward
Because! That's why!
 
Qward's Avatar
 
Join Date: Jul 2010
Location: Behind you, look out
Posts: 14,455
vCash: 500
Quote:
Originally Posted by Sens Mile View Post
Thats funnt
You laugh now, just wait till he watches "Draft Day"

Qward is offline   Reply With Quote
Old
02-07-2014, 11:54 AM
  #18
Mercurial
#lalala
 
Join Date: Oct 2009
Country: Canada
Posts: 2,232
vCash: 500
So if you get less points, no matter what you spend you don't make the playoffs. Got it.

How about cost per cup, that's all that matters.

Mercurial is offline   Reply With Quote
Old
02-07-2014, 01:13 PM
  #19
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Quote:
Originally Posted by Icelevel View Post
did anyone else see the comments melnyk made a while ago (before these latest moneyball comments) about how he was well aware of the statistical history that teams that spend more are more successful in the playoffs?

i've been looking for it since i read it but can't find it now. anyone have a link to it?
I don't have a link to Melnyk's comments, but there is a discussion about how teams that spend do better in the Forbes article that I linked to in the OP.

StefanW is offline   Reply With Quote
Old
02-07-2014, 01:35 PM
  #20
Ether Prodigy
Fearless Leader
 
Ether Prodigy's Avatar
 
Join Date: Mar 2007
Location: Ottawa
Country: Canada
Posts: 19,290
vCash: 500
I'm reading on my phone but Ottawa is in bottom 10 and has made the palyoffs the last two years and isn't the total for the last 4 years 3/4? Or am I misreading ?

Ether Prodigy is offline   Reply With Quote
Old
02-07-2014, 01:39 PM
  #21
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Quote:
Originally Posted by The Reg Season SC View Post
I'm reading on my phone but Ottawa is in bottom 10 and has made the palyoffs the last two years and isn't the total for the last 4 years 3/4? Or am I misreading ?
I don't break it down to the point where Ottawa's results are singled out. The only time Ottawa appears in the charts is the 2010-11 Senators, who had one of the worst ERs in the past 4 seasons and did not make the playoffs.

I think those tables would be very tough to read on a phone. I did not take that into consideration when I built the tables.

StefanW is offline   Reply With Quote
Old
02-07-2014, 01:52 PM
  #22
DrEasy
Out rumptackling
 
DrEasy's Avatar
 
Join Date: Oct 2010
Posts: 4,572
vCash: 500
Quote:
Originally Posted by Mercurial View Post
So if you get less points, no matter what you spend you don't make the playoffs. Got it.
Yep, that was exactly my objection to the methodology.

DrEasy is online now   Reply With Quote
Old
02-07-2014, 02:00 PM
  #23
BankStreetParade
Registered User
 
Join Date: Jan 2013
Location: Ottawa
Country: Canada
Posts: 2,035
vCash: 500
I decided to chart the total money spent by team from the 2009-2010 season until the 2012-2013 season. It's a cumulative spreadsheet that shows the total dollars spent by teams on their rosters as well as the total points accumulated over that same time period. I think the findings not only disprove that spending = winning or spending = points it also proves that spending and winning are not related (for the most part).

Other findings:
  • Bruins, at 9, only team in top 10 total spending to win a championship.
  • None of the teams that spent the most money in any one season won a championship.
  • None of the championship teams finished top 5 in spending their winning year.
  • The average for points over that time span is 329.
  • The average for salary over that time span is $226,563,677.80.
  • Calgary has extracted the lowest value relative to money spent.
  • Phoenix has extracted the highest value relative to money spent.
  • The only team on both top 10 lists below to win a championship is Boston.

Top ten teams in points (in order):
  1. Vancouver 390
  2. Pittsburgh 387
  3. Chicago 387
  4. Washington 377
  5. San Jose 371
  6. Detroit 364
  7. Boston 358
  8. Phoenix 354
  9. Los Angeles 353
  10. Philadelphia-St. Louis (tied 10th) 346

Top ten teams in spending (in order):
  1. Philadelphia $259,555,063
  2. Vancouver $257,313,149
  3. Pittsburgh $249,370,783
  4. Montreal $246,609,161
  5. NY Rangers $245,362,833
  6. Washington $244,537,653
  7. Calgary $244,433,686
  8. San Jose $244,168,686
  9. Boston $242,327,090
  10. Detroit $240,446,628

In relation to Ottawa:

19th in overall spending.
20th in overall points.
3 teams have spent more but have less points over that time span.
3 teams have spent less but have more points.

Many point to Detroit as a big spender. However, Detroit spends consistently. From 2009 to 2013, they only increased their spending by $4 million.
Attached Files
File Type: xlsx cap v points.xlsx‎ (13.5 KB, 5 views)


Last edited by BankStreetParade: 02-07-2014 at 02:06 PM.
BankStreetParade is offline   Reply With Quote
Old
02-07-2014, 02:31 PM
  #24
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Quote:
Originally Posted by Mercurial View Post
So if you get less points, no matter what you spend you don't make the playoffs. Got it.

How about cost per cup, that's all that matters.
Quote:
Originally Posted by DrEasy View Post
Yep, that was exactly my objection to the methodology.
Ok, I have to admit that I think I am missing something. That can happen when you look at the same numbers for a while. From my perspective the cost per point is purely relational, in the sense that you can spend a lot and get a lot of points or a few point, or spend little and get a lot of people or a few points. The CPP theoretically does not necessarily have to be linked to making the playoffs.

So I need you guys to elaborate so that I get wrap my head around what I am missing here. Yes, low points mean missing the playoffs. But I was not talking about point totals per se. The main measures were cost per point, and success level which was divided into making the playoffs, out in first round, out in second round, out in third round, runner up, and cup winner. How does "if your point total is low no matter what you spend you don't get into the playoffs" fit into this?

StefanW is offline   Reply With Quote
Old
02-07-2014, 02:32 PM
  #25
StefanW
Registered User
 
StefanW's Avatar
 
Join Date: Mar 2013
Location: Ottawa
Country: Canada
Posts: 6,282
vCash: 50
Quote:
Originally Posted by BankStreetParade View Post
I decided to chart the total money spent by team from the 2009-2010 season until the 2012-2013 season. It's a cumulative spreadsheet that shows the total dollars spent by teams on their rosters as well as the total points accumulated over that same time period. I think the findings not only disprove that spending = winning or spending = points it also proves that spending and winning are not related (for the most part).
Other findings:
.
I don't have time at the moment to double check the figures, but I am curious about the bolded part. What standard did you use for proving or disproving something?

StefanW is offline   Reply With Quote
Reply

Forum Jump


Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -5. The time now is 03:35 AM.

monitoring_string = "e4251c93e2ba248d29da988d93bf5144"

vBulletin Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
HFBoards.com is a property of CraveOnline Media, LLC, an Evolve Media, LLC company. 2017 All Rights Reserved.