TD;LR? Just look at the pretty pictures.
Before I begin, lets just remember Correlation != Causation
So, using the data that “Liam 😀” scraped from this Ludum Dare, I created some plots showing how each of the different categories correlate with the overall category. Here’s the lot of them (It’s a big image, so click to see it full size). Compo games are blue dots, and Jam games are red.
There’s a quite few interesting things there, but here’s a few little points.
As we’ve seen from previous analyses (Google told me that’s the plural of analysis) of Ludum Dares, the fun category has the highest correlation with the overall category, and humour correlates the least.
Another interesting thing is that the audio category is split for Jam and Compo games. You’re more likely to get a better overall score with the same audio score if you entered a Compo game, perhaps suggesting people are more forgiving for average audio in Compo games.
The ID plot may seem meaningless, except that IDs are given sequentially, and so it roughly shows what score people got in relation to how long they’ve been around Ludum Dare. It’s slightly skewed in favour of veterans, but not much, showing that newbies have just as good of a chance of making a great game.
But what I wanted to focus on is how the number of votes you got relates to the overall rating you get.
Now, this plot is a little hard to read because there’s so many people clustered up in the left, which hides the significance a little bit. You can see a slight upward trend as you get more votes, but it’s that clear. If you compare it to the plot of Votes Given vs Overall, you can understand it a bit better.
Because there are so many people that cast/received between 20 and 50 votes, you would expect to see more extreme results in that area, just because there are more games. This is what we see with the Votes Given vs Overall plot — as the votes get larger there are less scores near the top and the bottom, mostly because there are less games there and it’s unlikely to get a really good or really bad rating. (That being said, there are some relationships here, but they’re not as significant as in the votes received plot)
In the Votes Received vs Overall plot, we have games that got high ratings with a large number of votes. This would be pretty unlikely if they were unrelated, just because less games got that many votes, indicating that there is a correlation.
But please remember CORRELATION IS NOT CAUSATION!!! It’s totally wrong to say that this means that if you want to do better you should try to get more ratings, because more ratings = higher overall. Instead, we have to say: Ok, there’s a relationship, what theories can we come up with that might explain it.
When you think about it, it would make sense that games that are really good would tend to get more votes, because people share them more.
Even though the general trend is upward, we can also see that there are games that get a lot of votes in a way that’s unrelated to how good the game is, such as people who are hugely popular or do a lot of publicity.
Finally, an issue that often comes up is the concerns that games that didn’t get many votes could sneak a high score just by being lucky with the ratings they got. If you look at the plot, there aren’t that many games that didn’t do well that didn’t also get quite a few votes. There are a few, but as there are a lot of games that got a relatively small number of votes, there would also be a lot more if it was entirely up to chance.
This doesn’t mean it’s perfect, just not all that bad.
That’s all for now! Thanks again to Liam for the useful data!