Ask your engaged users for feedback

What kind of user spontaneously leaves a rating on a game? In my experience, there are primarily two kinds:

• Someone who’s really delighted and pleased with what you’ve made, so much that he or she takes the time to open up the Google Play app or go to the website, and leave you a rating and maybe a comment
• Someone who finds something about your game that they don’t like. Too easy, too difficult, too stupid, too many ads, too many crashes, whatever. These people will almost always leave you a comment telling you why your game sucks

In my experience, the second kind of user is much more likely to leave a rating than the first. And this isn’t just related to app and game reviews, you’ll see it all over the internet. Look at user reviews on anything – electronics, hotels, lawn mowers, it doesn’t matter. Most of the people who’ve put in the effort to rate something, have done it because they have a complaint. This doesn’t mean the product is generally bad, it just means that people who are satisfied with it don’t leave reviews.

You can create an app or a game that will amaze everyone, and then you’ll get lots of spontaneous feedback. But in the real world, that doesn’t work so well. I’m sure you’ve been in this situation. You have a fairly decent game, your analytics are showing that people are playing it and enjoying it, but the only people leaving ratings are the ones who hate your game.

So what can you do about it?

The answer is simple – ask your engaged users to leave you feedback. The key concept here is ‘engaged users’. You don’t want to ask everyone for feedback, you want to identify the people who are using your app and presumably are happy with it, and ask them for feedback. This seems so obvious, and the first time I came across this idea, it was one of those head-slapping *duh* moments.

Here is an article that talks about this, and provides some sample code. This code is for Android, there’s a link to the original article which was for iOS. This uses a fairly simple heuristic for determining whether a given user is an ‘engaged’ user:

• It’s been at least 3 days since the first time the app was launched
• The app has been launched at least 7 times

Both numbers are adjustable, of course. But this simple check covers a few different scenarios. For instance, if your game crashes every time it’s opened on a phone, you might get 3-4 sessions one after the other on the first day. If you ask that user for feedback, you’ll definitely get a negative review.

Does it work?

Bus Jumper is a weird game when it comes to user feedback. It’s a silly, funny game, with very little gameplay and no depth at all. The spontaneous feedback has definitely been bi-modal. Some people get the humor and like it, and the positive reviews almost always comment on how it made them laugh. Some users don’t find it funny, and if you don’t, then it’s a pretty dumb game. And the negative reviews always point out that the game is stupid.

Sometime in October 2011, I put out a Bus Jumper release that had this ‘app rater’ code in it. I took a snapshot of the user ratings at that time, to see if this code had any effect. This is what the ratings looked like on October 19th:

The 1-star reviews are more than half the count of the 5-star reviews. I don’t see too many games like this. Good games have reviews that are tilted upwards, bad games have reviews that are tilted downwards. Bus Jumper seems to follow the typical distribution for 5-4-3-2, and then there are all the users who didn’t find the game funny, and took the time to leave 1-star reviews.

I realized that today is April 19th, which is exactly 6 months since I took the last snapshot. So that seemed like a good time to take another snapshot, and write this article, which I’ve been meaning to write for some time. Here’s the snapshot from today:

43 new 5-star reviews, 15 new 1-star reviews. 3- and 4-star reviews almost doubled, 2-star reviews didn’t change much. What this tells me is that I still have people who download the game, don’t find it funny, and take the time to tell me (and the world) that it’s a dumb game. But I think some of the people who wouldn’t have otherwise left a review, are now doing so. These usually don’t have any comments attached. These users aren’t really excited about the game, but they like it enough that if you ask them nicely, they won’t mind leaving you a rating.

So… there you have it. If you don’t have something like this included in your game or app, I think you should. It makes a difference.