Turning Archive 2006

Some thoughts on judging

Ellis Walentine
>I missed the chat regarding judging, but I'd like to toss two belated cents into the discussion. I have been involved in judging woodworking contests for over ten years, and I have played by all kinds of judging rules. I have come to the conclusion that technical judging systems are way more trouble than they're worth.

Ultimately, judging is a subjective activity, no matter how you weigh it down with criteria and categories and rules and arithmetic. When you impose all these "objective" mathematical criteria, what gets lost is "which piece is best?" In other words, the operation can be a complete success, but the patient dies.

I would propose a system of considering the merits of the pieces in *this* contest (whatever contest you are judging) relative to each other, not measured against some absolute scale. No two judges can interpret an absolute scale the same anyhow. Have two or three simple criteria -- we always use design, craftsmanship and degree of difficulty as ours -- that all the judges can compare the entries by.

Let each judge come to his own ordering of the winners and losers, and then have a confab about the results to make sure that all the judges are in final agreement, even if one (or more) judge(s) has to compromise. I call this "judging by acclamation," and I think it is the only way to assure that, by some unintentional trick of arithmetic, the prizes aren't awarded to entries that nobody thinks should have won.

Besides being more realistic, this method is much faster because it doesn't require judges to assign numerical values to multiple aspects of each piece, and there is no number-crunching to do when all these sheets are tallied. Instead, the judges walk around the room, doing a preliminary scan of all the pieces up for consideration and making note of which ones they like best. Then, as a committee, they discuss their personal selections, debating them if necessary until a consensus has been reached.

This dialogue is vital, because it is the best way for judges to edify each other and assure that everyone is equally informed. Often, I've found that a judge will miss something the first time around, or they will be unfamiliar with a technique or other aspect of a piece that might affect their vote. The combined experience of the committee as a whole assures that they're all on the same page.

It's to be expected that different judges will order the entries differently. It's up to the judging committee to decide which pieces should ultimately score higher, based on a frank discussion of the judging criteria. Usually, these criteria are easy to apply individually, making consensus more objective and easier to reach.

I've found that a lot of people are pretty impassioned about how they think contests should be judged, so let the chips fall where they may.


© 1998 - 2017 by Ellis Walentine. All rights reserved.
No parts of this web site may be reproduced in any form or by
any means without the written permission of the publisher.