Ludum Dare 31
December 5th-8th, 2014

Some voting stats…

Posted by
September 7th, 2010 2:08 pm

After getting slammed with a couple of troll rating votes (people who rated an entry with 1s across the board, while all other votes had decent ratings), I was curious about how much this was happening overall in the voting.

I ran some numbers, and put a report here: http://spampudding.com/troll.txt

The format is username and how many troll rating lines they received (1 or 2), followed by their average ratings in each category, followed by what their average ratings would have been without the troll votes.

These five entries were hit the hardest, each losing a quarter of a point or more in at least one category due to troll votes: http://spampudding.com/trolltop.txt

Assuming all votes were logged, I’m wondering if maybe the admins could go through the database and see if it was the same couple people making these votes.


7 Responses to “Some voting stats…”

  1. Thanks for putting this together. I also noticed the trolling with a solid row of 1s for my entry and some others.

    I was thinking maybe it would make sense to show not only the scores that an entry received but also the average score in each category that person judged other games. This would make it plainly obvious if people were giving scores way below or above what they should have. Of course this system could be gamed as well but it might lead to a little more honesty.

  2. ExciteMike says:

    I think it’s best to just count the votes as they are cast. Maybe the row of ones was someone being a jerk, but it’s also quite possible it was what they thought the game should get. I don’t think there’s a good way to distinguish between the two.

    Let’s keep voting anonymous, and if we are going to let the community vote, let’s let the community vote how they want to. If you are going to throw out outliers, make sure you do it on both the positive and negative sides, and that you do it for everybody.

    My suggestion: Don’t even show the rows, just show the average and number of votes in each category.

  3. bluescrn says:

    Or the alternative, make the voting more open, actually show name, so we can see who’s being overly harsh (or overly generous) – then we can maybe pester them for some more in-depth feedback if they havew genuine constructive criticism to make?

  4. mrfun says:

    Side note: IIRC the voting calculation discards the highest and lowest scores to help with the fairness. Phil/Pov, please chime in if that is wrong! I could swear we were doing that though.

  5. pekuja says:

    @mrfun: Yes, apparently it’s supposed to do that, but it’s not. I checked the numbers on my game and it seems like nothing was removed.

  6. Dan C says:

    I voted all ones for one of those games, and I don’t think my vote was particularly unfair. In fact it reflected the general consensus. Nobody gave it above 2 in any category, except one person voted 3 for audio.

    That same game is near the bottom in nearly every category. It shouldn’t be a surprise that if you remove all the ‘all ones’ votes from the worst rated games, their overall score increases significantly.

    It seems a bit odd to complain about people giving games low scores. If the voting system is to have any meaning at all, then something has to get the lowest score. Otherwise we might just as well do away with the competition and declare everyone the winner.

  7. shockedfrog says:

    In that particular case (MetroDriver’s game, to save others the hassle of looking) it was deserved – the game didn’t seem to have anything to do with the theme and looking at it closer I think it might have been intended as a somewhat subtle joke entry. I don’t think anyone’s suggesting removing the ability to give all 1s where appropriate. The issue is that there are some people who are doing it for other reasons, people who clearly haven’t taken the time to judge a game fairly. Anyone with common sense can take a quick look at Martoon’s game and say that, even if they don’t particularly like it as a whole, it has a number of things going for it and no reasonable judge would give it all 1s. The only explanations I can come up for those scores in this case are (in no particular order):

    1 – they dislike Martoon for some personal reason and allowed that to spill over into the judging.
    2 – they were offended by the game’s use of religious/political themes and wanted to make some kind of protest vote.
    3 – they disliked the game so much in one category that they wanted to give a punishing score in other categories where it wasn’t deserved.
    4 – they’re trolls who are giving all 1s perhaps even without playing the game.
    5 – they were unable to get the game to run (in which case they should have judged the community score and left everything else N/A).
    6 – the voter considered the game to have broken the rules (in which case they should have left an appropriate comment, at least unless someone else had already posted one).

    Without actually knowing who gave the scores and their own reason why I guess we’ll never know.

    Of course, the system is never really going to be entirely balanced – every game is rated by a different set of judges, each with a different way of judging, a different interpretation of scores. If your game happens to get the attention of people who are mostly very fussy, or mostly very generous, then your scores will suffer/benefit as a result. Even amongst the handful like myself who do try to review all (playable) games, your scores may be subject to whim – this particularly applies to humour, what’s 5/5 funny one day might fall flat the next. It’s best not to take the whole thing too seriously, though I am curious about the effects different systems of judging might have.

Leave a Reply

You must be logged in to post a comment.

[cache: storing page]