Help request: Please assess rating method

[Persell]

I want to rate submissions and was wondering if it were possible if luminaries here could constructively assess my thinking. I want approach rating submissions using a systematic, consistent method that I hope will remain consistent regardless of what it is that I'm rating. Thereby giving utility to my ratings and it is hoped help contributors find the confidence to provide useful material to the site.


I'm considering the following five criteria:


Power: How does it compare with other Charms, Artefacts etcetera in the same 'range' or style?


Utility: How general or specialised is the Charm? This is similar to power, but in Stillborn's Charm: Anvil-Crushes-the-Hammer Technique utility and power are quite clearly different. That Charm has lots of power, but a limited utility.


Build: Costs, pre-requisites, placement.


Description: Flavour text, comprehensible, appropriate, length.


Instinct: My feelings on the rated material, this is purely personal opinion.


Fwiw, I'm actually a bit a prick when it comes to critiquing work, in that I score fairly rigidly and won't hand out a '+' just because it's nicer than a '0'. For 'fluff' I'll just rate it using my 'instincts' and deal with flames as I see fit. I don't intend to really soften this part of myself, but I do want to make my ratings fair and this is intended to help me do that.


Thoughts?


Edit: capitalisation.

 

[memesis]

I think your criteria are fine.  However, I will say that our rating system hinges on people being willing to give -1s or 0s when a submission merits them, and if you stick to your standards I think you will do that.


Four Elements Required to Form a Wise Crowd


Not all crowds (groups) are wise. Consider, for example, mobs or crazed investors in a stock market bubble. Refer to Failures of crowd intelligence for more examples of unwise crowds. What key criteria separate wise crowds from irrational ones?


- Diversity of opinion: Each person should have private information even if it's just an eccentric interpretation of the known facts.


- Independence: People's opinions aren't determined by the opinions of those around them.


- Decentralization: People are able to specialize and draw on local knowledge.


- Aggregation: Some mechanism exists for turning private judgments into a collective decision.


(Source: http://en.wikipedia.org/wiki/The_wisdom_of_crowds)