This post was originaly written for the 29th ludumdare. There is a slightly revised version that better suits my views after 10 participations here

 

Hello community!

This is my 8th participation in a row and, life being life, I had a lot of time on my hands in the past 2 weeks. This means that I was able to have several big sessions of rating. While this allowed my game to be featured in the “most coolness” section of the rating page, I was also confronted with a problem I never faced before: rating consistency.
Like most of you, when I find a game that I REALLY like I give a lot of stars. Even in categories where the game doesn’t deserve much stars. And yes, the opposite is also true: I will give less stars than you deserve if there is something I really don’t like about your game.
Those bias are often unconscious, but after a few sessions I found myself questionning my past ratings. Things like “Did that gorgeous game really deserved 5/5 overall?” or “Wait… I gave 4 stars to that other game, and this one is clearly better. But not 5/5 better”
With the amount of game I had already rated, and considering the amount of games I still intended to rate, I felt that I needed some kind of scoring system if I wanted to be fair with everyone.

 

I am NOT telling you how I think you should rate games. I post this for those who feel like their rating is not consistent. Or for those who find themselves giving either 1 or 5 stars in some categories. Or even as a tool to analyse your ratings if you like.

 

MOOD
[N/A] No intention from the author to create a mood (mystery, fear, relaxing…), and nothing special felt on my side
[1] Expressed intention from the author to create a mood (either in the description or in a text early in the game), but nothing in the game to support that.
[2] Clear intention from the author to do something, but poorly executed in my opinion (If you want to make a scary game your player must be affraid of the monster, not of the loud and unpleasant sound effect)
[3] Not very well done, but you tried. I also give a 3 when there is some kind of an ambiance in the game, but I don’t think it was intentionnal
[4] Well made, but something feels “off”. (don’t have a better way to describe it, depends on the game)
[5] Anything that passes the previous points

HUMOR
[N/A] No intention from the author
[1] You tried to make me laught, but at best it annoyed me (yes, I played a game like that)
[2] Something in the game made me genuinely smile, but I don’t think it was intentionnal
[3/4] I smiled/laught in multiple occasions
[5] I kept playing because I knew it would make me laugh

AUDIO
[N/A] No sounds AND I don’t think the lack of sound harms the game
[1] Either a lack of sound that harms the game, or if I muted the game
[2] Sound effects that are not unpleasant (basically any game that use exclusively bfxr sounds)
[3] Some music and/or sound effects of good quality or well integrated (ie. appropriate bfxr sounds with a well balanced volume). Basically you get at least 3 if I think the audio was a real concern in your game.
[4/5] Depends on the game, but the game must have REALLY good music and sounds. Otherwise, it’s a 3

GRAPHICS
[N/A] I always rate this category. Even if it is a text based game (in that case I rate based on the clarity of the interface, with bonus points if you have some effects)
[1] I don’t understand what is happening on the screen. Seriously, some games do just fine with colored squares!
[2] I understand what is going on
[3] You took the time to create some assets and/or have some effects here and there
[4] Good assets OR a nice aesthetic (style, colors…) OR some notably cool effects
[5] Everything listed in point 4, or REALLY beautiful art.

THEME
[N/A] I always rate this category.
[1] You did nothing with the theme (Come on… LudumDare is a theme-based jam) OR the link to the theme was explained in the description but I felt it was just bad.
[2] A take on the theme that is not original (under water, under earth), or the theme is only mentionned at the begining/end of the game
[3] A not-so-common take on the theme, but used only to set the context of the game.
[4] The theme is USED in someway in your game
[5] A fondamental mechanic of the game is built from the theme, or the mechanics of the game all revolve around the theme

FUN
[N/A] I always rate this category.
[1] I was forcing myself to play the game (Frustrating game: no clear objective, bad controls, bugs…)
[2] I didn’t feel the need to keep playing after a game over or after a few levels (I kept playing only to get a proper idea of the game)
[3] I enjoyed it
[4] I kept playing for a while
[5] I want to come back to this game later

INNOVATION
[N/A] I always rate this category.
[1] A tetris/pong/match3/arkanoid clone, or a “classic game” (platformer/shooter/infinite runner) with nothing unique
[2] Classic game type whith a not-so-common feature that is not well integrated
[3] Any “abstract” game, or a classic game with a nice twist
[4] Multiple cool features, or an unusual mix of mechanics, or a clever mechanic (often associated with the theme)
[5] Unique gameplay

OVERALL
[N/A] I always rate this category.
[1] Neither a game nor an interactive experience. I also give 1 when there is just a mechanic with nothing built around it (a guy running around)
[2] A game I won’t remember
[3] Well… anything between a 2 and a 4
[4] Great game overall (often have multiple 5 in other categories) but don’t feel like a “complete” game
[5] Well… anything better than a 4
Now, a few random thoughts:

  • Read the description. Sometime you don’t want to read a wall of text (especially if it is a big thesaurus of all the entities in the game), but the description is the only medium for the author to communicate with you.
  • Post a comment. It’s not always easy to post something nice, but for some people comments ARE more important than the actual rating. I suspect I am not the only one to check my page at least once a day. In my case I actually won’t be able to see my ratings for a few month since I won’t have access to internet, so the comments I am getting NOW are what really matters to me.
  • Be constructive. I have seen multiple games with comments that were not far from “nice art, shitty game”. Seriously, wtf!
  • Don’t get your expectations too high : The goal of LudumDare is to make a game and to get feedbacks on it, not to win it

 


5 Responses to “Rating consistency: How I rate games”

  1. goerp says:

    great job, I might take some parts of this system.

  2. Gabriel says:

    Hey, this is an AWESOME post! With the amount of newcomers around and even for us veterans, a clear and fair guidance like this is bound to generate more fair ratings and more useful feedback.

    LD admins, I think you should seriously consider making this into a sticky post, and pull it back to the top whevever there is judging going on! This is really, really helpful, and a truly high quality guide.

    PapyGaragos, I’m gonna rate your game and leave some useful feedback as I always try to do- it’s the bare minimum I can do! Would also love to receive a rating from you, but that part is obviously not required – you already made a major service here! :)

    • The post was indeed very insightful, and i wouldn’t mind seeing it again next time a rating is in progress. It’s pretty much what I rate like, too. 1 is rather poor, 2 a little better, 3 just an average rating, if it’s good it gets a 4, if it’s very good, a 5… 😉
      Thing is, I had to think about this post for a while, as I generally don’t like being told what to do in such a straight manner (I know that wasn’t the intention, but still…). I had a feeling we should concentrate more on rating games in a relation to each other, putting them in perspective in a way. But as I thought about this, I realized that concidering the small amount of games a single person can rate, this idea dosn’t make much sense. So maybe, sticking to some general rules when rating games could help make ratings more objective. I guess there is no other way…
      So : Good idea, let’s think about it some more! 😉

  3. vilya says:

    Great post! Here’s how I think you would have rated my game, it’d be interesting to see if you agree:

    Mood: 2 or 3
    Humor: N/A
    Audio: 3 or 4
    Graphics: 3 or 4
    Theme: 2
    Fun: 4 (I hope!)
    Innovation: 1 or 2
    Overall: 3 or 4 (if I’m lucky).

Leave a Reply

You must be logged in to post a comment.

[cache: storing page]