Topper you have hit the nail so hard it's gone through the board.
While Charon, Drance, Desjardins and Johnson, among many others, use stats and techniques to quantify what they believe there is a whole other alchemy involved in Advanced Stats.
First off some of them are incredibly arrogant and use their work to support outcomes they predicted long ago- I have yet to see one say they were off. I guess that's a privilege of being one of the first 'out' on this front. They are also very possessive and argumentative in regards to challenges of their work, Tyler Dellow and Desjardins being the leaders by far.
So what makes it so special? Well start with Corsi- it's a way of looking
at the game and drawing some simple conclusions about what a player does on the ice but does it tell a more complete story than observation or does the information support the observations? Almost a chicken and the egg scenario.
Look at Desjardins' site and look at Quality of Competition- what determines it?
There are numerous ways we could go about this. We could average out the points-per-game of opposing players(as Jonathan Willis has suggested, and which works reasonably well, particularly when you have no ice time information), but I think the best place to start is with what I called "Relative +/-" or "Rating" in part 1. Relative +/- adjusts a player's on-ice +/- relative to his team's +/- while he was off the ice. In general, it corrects for the boost players get from playing on a good offensive team and vice-versa.
If we average that rating across all of a player's opponents, weighting for how much time they played against one another, then we have an estimate of how good a player's opponents were relative to their teams. In a general sense, first line players have the best ratings, so players who play against the first line should see the highest opponent rating. That average opponent rating is the "Quality of Competition" faced by a given player.
Ask them what they think of using plus/minus as a stat and then see that it's completely valid for them in this scenario. Of the top 30 scorers in the NHL 9 are in the minus, almost 30% So how does Desjardins 'adjust' the +/- as he says above?
We can make a small improvement on +/- by subtracting the +/- when a player is off the ice from it. That is, if a player was +1 goal per 60 minutes on the ice and his team was even when he was off, he ends up appearing the same as a guy who was even on the ice while his teammates were -1 per 60 minutes. It's not perfect, but it does make an adjustment for how good a player's teammates were. This statistic has several names - relative +/-, On-Ice/Off-Ice +/-, or simply "Rating", as I've called it on the stats page.
Here's the list: http://www.behindthenet.ca/2008/new_5_on_5.php?sort=7§ion=goals&mingp=40&mintoi=10&team=&pos=
Where I have a problem with all this is hockey is a game with unique variables unlike other sports. If Datsyuk is a +/- there is no way to determine how he got there by Desjardins' stats or anyone else's. In his list it shows Rene Bourque as 3rd for 'rating'. How many turnovers does Datsyuk create? How many points from turnovers does he earn? How often did his team mate convert on his work? How many scoring chances does he create? What makes Datsyuk one of the best players in the game is how he handles the variables within the game and what he does and none of those stats tell you how- they only support that he does but also Rene Bourque does too.
Look at stats like shots attempted. Wouldn't it be nice to know how often a team recovers missed shots in their defensive zone and in the offensive zone? If you are a puck possession team like the Canucks it's pretty important especially if you adhere to strict zone start regimens.
I think the advanced stats are incredibly interesting and tell a good and relevant story for teams about players and past performance but I believe there are betters ways to do it. The Canucks have their own team who do advanced stats and it sure as hell would be interesting to know what they track as I bet it's far different than what we see.