LD34 Visualisation and Analysis

Posted by (twitter: @jezzamonn)
January 23rd, 2016 7:01 am

TD;LR? Just look at the pretty pictures.

Hi all!

Before I begin, lets just remember Correlation != Causation

So, using the data that “”¬†scraped from this Ludum Dare, I created some plots showing how each of the different categories correlate with the overall category. Here’s the lot of them (It’s a big image, so click to see it full size). Compo games are blue dots, and Jam games are red.

ld34 correlations

There’s a quite few interesting things there, but here’s a few little points.

As we’ve seen from previous analyses (Google told me that’s the plural of¬†analysis) of Ludum Dares, the fun category has the highest correlation with the overall category, and humour correlates the least.

Another interesting thing is that the audio category is split for¬†Jam and Compo games. You’re more likely to get a better overall score with the same audio score if you entered a Compo game, perhaps suggesting people are more forgiving for average audio in Compo games.

The ID plot may seem meaningless, except that IDs are given sequentially, and so it roughly shows what score people got in relation to how long they’ve been around Ludum Dare. It’s slightly skewed in favour of veterans, but not much, showing that¬†newbies have just as good of a chance of making a great game.

 

But what I wanted to focus on is how the number of votes you got relates to the overall rating you get.

ld34 votes vs score

Now, this plot is a little hard to read because there’s so many people clustered up in the left, which hides the significance a little bit. You can see a slight¬†upward trend as you get more votes, but it’s that clear. If¬†you compare it to the plot of Votes Given vs Overall, you can understand it a bit better.

ld34 votes given vs score
Because there are so many people that cast/received between 20 and 50¬†votes, you would expect to see more extreme results in that area, just because there are more games. This is what we see with the Votes Given vs Overall¬†plot — as the votes get larger there are less scores near the top and the bottom, mostly¬†because there are less games there and it’s unlikely to get a really good or really bad rating. (That being said, there are some relationships here, but they’re not as significant as in the votes received plot)

In the Votes Received vs Overall plot, we have games that got high ratings with a large number of votes. This would be pretty unlikely if they were unrelated, just because less games got that many votes, indicating that there is a correlation.

But please remember¬†CORRELATION IS NOT CAUSATION!!!¬†It’s totally wrong to say that this means that if you want to do better you should try to get more ratings, because more ratings = higher overall. Instead, we have to say: Ok, there’s a relationship, what theories can we come up with that might explain it.

When you think about it, it would make sense that games that are really good would tend to get more votes, because people share them more.

Even though the general trend is upward, we can also see that there are games that get a lot of votes in a way that’s unrelated to how good the game is, such as people who are hugely popular or do a lot of publicity.

 

Finally, an issue that often comes up is the concerns that games that didn’t get many votes could sneak a high score¬†just by being lucky with the ratings they got. If you look at the plot, there aren’t that many games that didn’t do well that didn’t also get quite a few votes.¬†There are a few, but as there are a lot of games that got a relatively small number of votes, there would also be a lot more if¬†it was entirely up to chance.

This doesn’t mean it’s perfect, just not all that bad.

That’s all for now! Thanks again to Liam for the useful data!

Tags: , , ,


One Response to “LD34 Visualisation and Analysis”

  1. Liam :D says:

    Awesome! This is exactly what I was hoping someone would do with the data scrape!

    The hypothesis I wanted to test myself, which you beat me to, is that games with higher numbers of received votes, tend to get more average ratings.

    I had a suspicion that the randomness factor for ratings is quite high. When I made the scrapes, I also made graphs, which show the number of games at each rating in 0.1 step intervals. What I’d expect to see would be some sort of function which peeks at the lowest rating and then decreases in some way as the rating goes up. Instead it’s a normal distribution with a peek somewhere around the 3.0 rating, depending on the category – this is what you’d expect to see if the ratings were random. So, going with the assumption that there is a large degree of randomness in each rating, receiving more of them would mean your result will tend more towards 3.0.

    But, as you show, this doesn’t seem to be the case. There seems to be some sort of relationship between the received votes and at least the overall category, which indicates that receiving lots of votes correlates with getting a higher rating. This is very apparent in the 3.5+ rating, 20-30 received votes region, where there are surprisingly few games, even though that for lower received ratings, the 20-30 received votes region is packed with games.

    Also, to address your point about the audio category: That one’s different from the others. While there tend to be some slight, easily identifiable relationship between graphs, the audio category may as well be two categories in terms of jam audio and compo audio. See here to see how different the jam and compo ratings for audio are: http://liamlime.com/dump/laptop/2016-01-10/liamlime.comexperimentsldscrapeout-ludum-dare-3_2016-01-10_22-47-48.png . So while your conclusion is correct (that getting an equal audio score in compo will likely get you a higher overall score than the same audio score in the jam), the reason is that, for example, a compo audio score of 3.1 means your game is well above average, while a jam audio score of 3.1 means your game is well below average.

    For similar graphs, refer to my previous post, which also contains the links to the data for the post above and this comment: http://ludumdare.com/compo/2016/01/10/how-many-people-actually-got-ratings/

Leave a Reply

You must be logged in to post a comment.

[cache: storing page]