Ludum Dare 31
December 5th-8th, 2014

Interpreting feedback and building a backlog

Posted by (twitter: @jayrob_in)
May 11th, 2012 11:06 am

This was my first Ludum Dare and I thoroughly enjoyed it. Not only because I learned a lot in a short time, but also because I made (what I believe is) a pretty good first iteration of a game in just 48 hours. I have plenty of ideas for what I could do with the game, but I’d much rather find out what the players want and build that into my backlog. That is one of the great charms of Ludum Dare – building a prototype and getting guaranteed feedback from real players.

As of writing, my game has 34 comments. I could go through them one at a time and tick off each feature request, bug report, etc, but that would be long and stupid. So, here’s what I did instead:

1. Gather all feedback
I copied and pasted each comment from my entry page into Excel and cut out any that very clearly didn’t provide any useful feedback (e.g. “Awesome game!!!1″ or “I didn’t like it”)

2. Split feedback into negative and positive key points
The point of this step is to understand what each piece of feedback is actually telling you. I found it helped to make a list of negative and positive bullet points for each comment. Take this example:

“I like this, it has a lot of potential. Here are some of my random thoughts…

It’s rather slow paced. It feels less like action than “Line up with an enemy, start shooting, wait, dodge it, wait, wait, dodge it, aim at next enemy”. Also, because there are so few enemies in each stage, it feels empty.

It’s hard to keep track of where you are. It would be awesome if there were a minimap during the shooting bits.

I got frustrated with placing pieces, but then I figured out, you can’t place a piece if it doesn’t match up? That does make sense, but with a little work it could be better. How about if the game checks if there’s a valid path, and it won’t let you place a piece that makes the path invalid? Also, what happens if you run out of room? Perhaps you could place pieces over previously placed pieces to delete them.

I’ll be looking forward to later versions! ;)”

If we extract the key positives and negatives we get:

Positives:

  • Good concept

Negatives:

  • Slow-paced ship section
  • Too few enemies in a room
  • No in-game map
  • Difficult to figure out level building (tile placement)
  • Unintuitive level building (e.g. just place junk pieces elsewhere until right piece appears, potential to run out of room)
  • Build stage lets you proceed to ship stage even if a path doesn’t exist from start to goal

3. Group feedback points and sum totals to rank them
At this stage, you’re still left with a long ‘to do’ list of overlapping or contradicting items. You should group similar points together and rank them according to the totals

You can see below the tally of my groupings:
table

My interpretation is that people really enjoyed being able to create the levels before flying through them, but the level building could be better implemented, explained and introduced. The other major feedback points were around the ship sections being slow-paced but occasionally too difficult (green rooms are too easy and red rooms are too hard), and enemy ships starting too close to the door when entering a room.

4. Take the top negative items and determine how you can resolve/improve them
You should now have a list of items, ranked by the number of people who commented on that aspect of your game (I took anything that 2 or more people commented on). The idea is to remove the pain points by taking the most commented negatives and working out how you can turn them into positives. So, another example:

“I don’t fully understand the level building section”

Improvements:

  • First level goes straight into ship section, second introduces level building with short tutorial
  • Prevent level start until route exists between start and finish
  • Remove restriction on tile placement (can place anywhere)
  • Replace random tile selection with ability to choose which tile to place next

Note that you may occasionally need to refer back to some of the comments if your items are too general.

You should now have a pretty full to do list of changes that will make the biggest positive impact to players, which you could merge with your previous list of new features/fixes to cater for the few things that players may not have thought of.

Tags:


4 Responses to “Interpreting feedback and building a backlog”

  1. AlwaysGeeky says:

    Interesting article… I guess we all do a similar thing mentally without realizing, but its nice to see you write the steps to do this in a logical recordable way.

    I am not entirely convinced that the time commitment to do this sort of analysis is always worthwhile though.

    For example, my game might be a totally different kettle of fish to yours, but all I have to do to come up with the same output as you (what needs to be improved, a todo list of improvements) is read each of my comments and mentally make the connections, join the dots.

    Still whatever works for you post-project to get metrics and areas for improvements is good. It is always worthwhile going back to work afterwards just to take a second more detailed look at things like this :)

  2. Casino Jack says:

    That’s a very fair point, particularly if the feedback is limited or very similar. To be honest I think that being a project manager my love of planning and writing up lists tends to bleed through into all areas of my life.

  3. Jedi says:

    Good read. I’m going to try tallying up my comments and see if it’s very different from my overall perception.

    It’s easy to put off fixing things that aren’t a problem for you in favor of doing things that are more fun to develop. I think this kind of exercise is a good way to combat that.

  4. Keirua says:

    Interesting approach to the problem of dealing with feedback.

    I like the idea of sorting a comment into positive and negatives is interesting, however I’m not sure it scales well (would you do the same work if there were, say, 3000 feedback ?). Anyway there are not many ways to get the kind of diagrams you show.

    What is more it can be a good way to have metrics on contradictory wishes ( “I’d love the player to go faster” vs “the player is too fast !”), and as you say, for tweaking the gameplay details or functionnalities.

    I’m not sure however at which moment in the game developpement process one should get rid of the engineering-style approach and… think real, in a pragmatic way, about what one wants with its game (and if actually, one should, since it might not be what players want)

Leave a Reply

You must be logged in to post a comment.

[cache: storing page]