About Ratings

Posted by
April 16th, 2012 3:41 pm

If you haven’t already, go read this post: http://www.ludumdare.com/compo/2012/04/16/a-quick-reminder-read-the-rules/  I’d like to expand a bit on what Hazard has already said by talking about the Rating Categories.

I have heard many quibbles about ratings in the past few LD’s.  (Even though I don’t always participate I do follow the blog fairly consistently, so I’ve seen what some of you have been saying.)  It seems to me that an increase in complaints about the rating system is correlated to the rapid increase of participants over the past year or so.  To me this indicates that those new to the competition haven’t necessarily taken time to understand the rating system.

First off, let me state that I do not believe Ludum Dare is really a ‘competition’.  The aim of this event is for personal improvement in a fun environment.  If you have taken Hazard’s advice and read the rules, you’ll have come across this statement: “Ultimately, our goal with Ludum Dare is to encourage people to sit down and make something. ”  This is not about who takes the prize, or who gets a better rating than whom.  It’s not even about how well your game is rated at all!  I believe the staff will agree with me when I say that Ludum Dare is about what you can learn from your experience and the fun that you can have doing so.

That being said, it is important to rate the games you play fairly.   The feedback from the ratings is one of the most important ways of analyzing your work, specifically, where you need to improve.   If someone receives a lousy rating in the ‘audio’ category, they know they need to work on their sounds, but giving a low audio rating to a game with no sound is not helpful to the creator.  Thus, to help us all improve at what we’re doing, make sure to give reasonable ratings, and leave comments for categories you rate poorly.  To do so, take Hazard’s advice and read the rules, even if you’re a competition veteran.  A couple things stood out to me this time that I hadn’t noticed before, and it never hurts to refresh your memory.

The rules are fairly explicit in breaking down the categories, but indulge me to expound on them.

Innovation – This category is about either entirely new ideas, or old ideas used in new ways.  If you’ve made a platform shooter, you haven’t innovated yet.  If you’ve made a platform shooter where you try to avoid shooting as much as possible, you may have innovated.  When rating this category ask yourself, “What is different, fresh, or unexpected about this game’s mechanics?”

Fun – This category seems straightforward, but don’t forget that fun can be derived from a large range of qualities.  I’ve had fun because a particular game is really challenging (eg. N); because it has addictive, rewarding gameplay (eg. Diamond Hollow -shakes fist-); because of a pleasurably unique style (eg. The Wager); because it is actually frustrating at times (eg. Dwarf Fortress – losing is Fun); or even because it is frightening or disturbing  (eg. Amnesia).  When rating this category ask yourself, “Have I enjoyed myself while playing this game?”

Theme – This category is there to make sure you’re on track.  Back in LD #19, I attempted to make a game for the theme ‘discovery’.  After assesing my final concept (which didn’t even get completed) I realized that my gameplay was no longer related to the theme – it was about finding specific items rather than new ideas.  The winner of that LD captured discovery very well: a caveman being introduced to modern concepts.  Others who entered the same compo took the theme in different clever directions.  I remember several plays on the ‘disco-very’ mutation of the word.  When rating this category ask yourself, “Has this game incorporated the theme in a solid, tangible manner (even though it may be obtusely interpreted)?”

Graphics – This is often a misleading category in my experience.  I think it ends up being almost more about style than about flair.  This category often tends to be more subjective than others.  Some may think that a dichromatic color scheme makes for a very good graphics score, while others may rate it poorly.  I personally tend to rate games higher that use their graphics effectively.  By this I mean simple things like perhaps using colors to achieve a mood effect (eg. 5 colors pandora), or just something as simple as having a clear definition between solid foreground platforms and background art in a platformer.  When rating this category I ask myself, “Does this game hold together stylistically, and does it put its art to good use?”

Audio – This is also a tough one for me, simply because some games have just sound, some have sound and music, and others have none.  Keep in mind that if audio is completely absent do not rate this category poorly, instead, give it a N/A vote.  If it has sound but no music, judge it by the merit of its sound alone.  Things of note when judging audio are the volume difference between each sound and the balance between sound and music; are you having to adjust your speaker volume between different parts of the game for instance?  Do the sound effects give good feedback for game events (a ‘boing’ when jumping or an ‘ouch’ when hurt are good examples)?  Most importantly, are any of the sounds annoying (a constant harsh ‘pewpewpew’ for instance)?  When rating this category, ask yourself, “Did the game’s sounds add to the experience?”

Humor – Here’s another category that can easily merit an N/A.  If a game is meant to be somewhat disturbing, it’s not likely to have much humor in it and should receive an N/A rather than a low vote.  Take for instance ‘The Republia Times’ from the warmup for this LD (#23).  There not much more to say about this category; ask yourself, “Did I laugh, snort or chuckle?”

Mood – A recent addition and my new favorite category.  This tackles how well the game created a ‘feel’.  Mario has a mood that is instantly vibrant and lighthearted thanks to its catchy music, cutesy graphics, and laid-back gameplay.  Amnesia creates the opposite mood through chilling music, disturbingly good storytelling, and creating tension in the player over whether or not to even light his lamp.  Perhaps my favorite games for mood though have been the Advance Wars series for GBA.  Everything from the music to the graphic design to the dialog yell, “This game is fun!”  Digression aside, ask yourself, “How well do the separate elements contribute to the feel of this game?”

Overall – The most subjective category.  Some people treat it as an average of all the other categories, I tend to treat it as separate.  When rating this category, I ask myself, “Was my experience playing this game positive, negative, or average?”

Community – Personally I dislike this category for its ambiguity.  Someone may have posted a bunch of things about their entry, timelapse, progress reports, blahblahblah, but not really done anything for the community.  This becomes more true as the number of participants increases. There’s no way any sane person could watch 1000 timelapse videos, so how exactly are those all contributing?  I strongly dislike downvoting this category because so-and-so did not post a timelapse or somesuch.  Rant aside, I prefer to rate this based on the specific content of participants posts; whether they are genuinely trying to get involved or just putting things online to maintain status quo.  When rating this category, I ask myself, “Does this person appear to value the LD community over their own work?”

As a final word, please don’t ever make a lazy vote.  I would rather receive a few well-thought out criticisms than a myriad of undeserved praise.  If you find that you are just trying to get the rating process over with, please just set that game aside and move on rather than giving an unthinking rating.  If time permits, you can come back and rate that game fairly at a later point.  Also, if you have given a game a poor rating in any category, please take a moment to leave a comment providing constructive advice in that category.

TL;DR:  Think about your ratings; make sure they line up with the meanings of the categories laid out in the rules.

Thanks for your time, and enjoy LD #23!

3 Responses to “About Ratings”

  1. AD-Edge says:

    A great post. Everyone voting should read this and TBH it should be stickied when the compo ends so others *can* read it.

    But yeh, I also dont like the community vote. Its far too ambiguous, and regardless its impossible/impractical to keep track of every member, not to mention going though their history before you vote on a game.

    Mood would have to be my favorite categorize as well, since its something I value most in a game.

  2. digital_sorceress says:

    When rating this category, ask yourself, “Did the game’s sounds add to the experience?”

    Because some games don’t necessarily need sounds, I find it is more helpful to ask the converse question: “Would the experience be improved if sounds were added/altered/done differently?”

    I suppose the same could be said of graphics.

    • huhwhozat says:

      If the game does not have sound, my question is moot. The rules indicate it is proper practice in this case to rate the category as N/A; I tried to make that clear. Your converse question seems to imply that games which have no sound should be rated poorly if they could have benefited from sound. I do not think this is the case.

Leave a Reply

You must be logged in to post a comment.

[cache: storing page]