HOH Top 40 Goaltenders - Round 1 Screening Process
Project administrators sent all Top 60 lists for Round 1 of the project to a group of screeners for approval. The purpose of the screening process was to screen out lists that demonstrated insufficient knowledge or care, or that didn't match the intent of the project according to the consensus of participants.
We had three screeners. Two were non-participants - longtime HOH poster overpass who was a co-administer for the HOH defensemen project, and longtime HOH poster EagleBelfour who was a screener for the HOH defenseman project. One was a project participants - reckoning, who was also a screener for the HOH defenseman project.
The screeners did not know the identity of the list submitter. Their verdict was given solely on the contents of the list. Project administrators TheDevilMadeMe and seventieslord knew the identity of the submitters of the lists, and for that reason did not participate in the screening process.
In the end, screeners accepted one list by a vote of 2-1, rejected one list by a vote of 2-1, and accepted all the other lists unanimously.
Screeners were given the first 12 submitted lists to look at, so they could compare them to each other. They were then given groups of more lists as they were sent in.
Screeners used the HOH Top 100 Players list created in 2008 as a guide. Any lists that did not contain all goalies from this list were to be flagged and sent back to be reworked. While players from this list did not have to be included in the final list, we wanted to make sure it was for a good reason. This did not become an issue, as all submitted lists contained all goalies from the 2008 list.
Screeners were to reject lists that, in their opinion, showed a lack of care. They could use the 2008 top Players list as a reference, but weren't limited to it. One list was rejected for this reason.
Screeners were to reject lists that, in their opinion, were heavily slanted toward or against certain eras or that failed to consider the full scope of hockey history. No lists were rejected for this reason.
Screeners rejected this list for showing an apparently lack of care. Both screeners who rejected the list pointed to the inclusions of Dunc Munro and Steve Thomas, neither one a goaltender. Steve Thomas could have been intended to be Tim Thomas, but nobody knew what to do with Dunc Munro.
Screeners also pointed to the lack of Hugh Lehman, Hap Holmes, Tom Barrasso, and Tim Thomas - this was the only list of the first 12 to exclude any of them (though later we included a list that excluded only Barrasso). One comment: "There might be a Western league blind spot with the omission of Lehman and Holmes. And with the lack of care previously demonstrated, any of Lehman, Holmes and Barrasso may simply have been overlooked."
One screener was confused by the apparent inconsistency at the top - " I don't think I ever saw Patrick Roy that low on a list, and Charlie Gardiner that high! It's hard to put Martin Brodeur that low at #13. If that person put that much emphasis on regular season, Turk Broda should go lower for sure," but agreed that the big issues were the inclusion of non-goalies and the exclusion of goalies that were on everyone else's lists.
One screener said that he didn't think the list maker took the process seriously. The other screener who voted to reject recommended that if we give the list maker a second chance, we should just mention the lack of early goalies from the Western leagues and say that there were a couple of non-goalies on the list, rather than get into specifics.
We initially followed the advice of the second screener, but ended up changing our minds and giving the list maker the specifics he asked for (as to the non-goalies on his list and the omitted names).
Last edited by TheDevilMadeMe: 01-29-2013 at 05:45 PM.