A formal decision-making (multi-criteria decision analysis) view on judging:
Judging is subjective according to several criteria (speed/control, criticality of turns, innovation, appropriateness, style (length of ride)) based on sensory input (sight, sound, emotion)
Judging is subject to several stimulus and response biases typical when sensory inputs used for estimation (if you like something it seems better than it is):
1. Centering bias: Judges tend to come back to the range of scale they are comfortable with. Also, if stimulus changes (surf, age group, average ability), judge compensates only gradually, not instantly.
2. Contraction bias: Large stimulus are underjudged and small stimuli overjudged (scale ends up shrinking)
3. Logarithmic bias: a 10 is not 2x better than a 5, so it's easy to get some score, but harder to get more score.
So the scale used by most judges looks like this:
album_showpage.php?pic_id=2927
Rather than judging on the straight line from 0-10, average judges work on the curved line. Nothing to get a 2. Easy to get a 5, but hard to even get an 8. Number of turns are important, but quality undervalued. Almost all waves are judged in the yellow region.
A good judge works much closer to the linear scale.
album_showpage.php?pic_id=2928
TOADS and wiggles get nothing. One big move on the outside gets a big score (after all it moves the wave into the "good" category immediately). The wave only gets into the "good" range after at least 1 big move. Tens are achievable in almost every heat.
Hoags did a great job of emphasising this at the PI contest. But let's face it, judging is hard. It's not easy to step onto the panel and get it right for at least a couple of heats.
I'd suggest that the scale be
even more detailed. Something like this:
Paddle in, get up, go for big turn, not make it fall off..1.5 max
Get up, never drive off bottom or top...2.5 max (regardless of how many half-turns)
Get up, big whack on inside section of medium wave...+-4
Get up, first turn is big whack on medium wave...+-6
Get up, first turn is big whack on big wave...+-8
etc.