How many people actually got ratings

Posted by (twitter: @LiamLimeGames)
January 10th, 2016 3:48 pm

ALL YOUR PLACEMENT PERCENTAGES ARE WRONG! ;_; YOU DID WORSE 😀

Uh… did I get your attention? Hi! So I did a thing and found out how many people actually got ratings in each of the categories during Ludum Dare 34.

So I know everyone loves to just calculate their percentage with the assumption that everyone got a placement, but it’s not really the case, unfortunately – some people opted out of ratings in some categories, others simply didn’t receive enough votes to get a rating. So prepare to have your calculated percentages fall quite a bit when you consider just people who actually got ratings. The number of people with ratings are as follows:

Number of people who received a rating (Jam):

Humor: 853
Theme: 1101
Fun: 1096
Audio: 721
Mood: 1005
Graphics: 1024
Overall: 1121
Innovation: 1051

Number of people who received a rating (Compo):

Humor: 697
Theme: 966
Fun: 962
Audio: 684
Mood: 860
Graphics: 907
Overall: 973
Innovation: 941

How were these numbers acquired? Literally by scraping the whole site. The full Ludum Dare 34 results can be found in this table. You can click on a column header to sort them.

For previous jams: See LD33LD32LD31LD30LD29LD28LD27LD26, etc. I think I did all of them, just edit the URL accordingly.

Rating distribution (green = Jam; Blue = Compo):

Humor:

Theme:

Fun:

Audio:

Mood:

Graphics:

Overall:

Innovation:


8 Responses to “How many people actually got ratings”

  1. Codexus says:

    But people without a rating most likely performed much worse on average than those with a rating. If they didn’t bother rating enough games to get a rating themselves, it’s likely that they had a low effort entry in the first place.

    So I don’t think using the actual number of rated entries in a category for your percentile necessarily gives a more accurate result.

    • YinYin says:

      It most definitely gives a different result. Both are accurate for what they are.
      This is ranking among all ranked. What everyone looked at before was ranking among all submissions.
      Nothing wrong with either.

      Also it does make a huge difference for audio and humour because a lot of entrants opted out of those (additionally to those who didn’t get any rating).

    • Liam :D says:

      You cannot make assumptions on missing data, especially assumptions like that, which are completely incorrect. For example, one of my favourite games, which I played in LD32, did not end up getting a ranking. I made a comment on it in the vein of “Please rate more games so others see this, because it is awesome.”. I can’t say how many other “high quality, low number of received ratings” games there are, but just the fact that I found one makes me think that there are more.

      There is technically something you can do with missing data, which is to assume its value fits the same distribution of available data, but then you still run into the problem that people can opt out of rating categories, so you don’t know how many actually made it possible to rate their games. I didn’t collect that data, so I don’t know.

  2. Jezzamon says:

    I generally agree with the current way the site does it, where games that didn’t get enough votes (usually games that didn’t rate many other games) receive a 100% penalty in every category, putting them to the bottom 😛

  3. Jezzamon says:

    This is useful too, but it also isn’t perfect.

    For example, many games don’t try to do humour, and so disable that category, So your humour percentile could be based on how well you did compared to all the other games that tried to do humour. But the problem is not everyone disables the humour category.

    This way, you get your percentile of all the games, whether they tried to do humour or not, and it’s only slightly affected by the games that didn’t get enough ratings.

  4. Jezzamon says:

    P.S. This data is useful! I was actually in the process of writing my own scraper but I guess I don’t need to any more, I’ll just scrape your data instead :)

    one question: Did you filter out comments that were made after the competition ended?

  5. Liam :D says:

    The comments are at scrape-time, the data isn’t refreshed. I’ve tweaked the scraping algorithm, so not all pages are identical and not all JSON files are identical. For Ludum Dares < 16 or something, the JSON file includes the entire text of the posted comments, but I realized this made the files too big. For LD34, comments are logged in two lists: The first is a list of comments on an entry (with a decent amount of information logged, without the comment's content though), and the second is the list of entry IDs the owner of that entry has made comments on.

    The JSON file is available for (I think) all LDs by changing the .html to a .json.

Leave a Reply

You must be logged in to post a comment.

[cache: storing page]