Begging For App Ratings

This story originally appeared in The Loop Magazine Issue 18

John Gruber recently suggested that users who are annoyed by “Please Rate This App” panes should leave one-star reviews.

While lots of users rightfully hate these panes, this incendiary suggestion naturally resulted in fallout from the developers I know. For those who make our living on the App Store, a one-star rating is an existential threat. The equivalent would be for someone like Justin Bieber to suggest that people bothered by seeing ads on tech blogs like Gruber’s boycott their advertisers.

Small OS X and iOS software companies live and die by their App Store ratings. If an app is rated one star, nobody will look at it or buy it. If it’s rated five stars, the company will make a decent living.

This system seems fair and good, but there are three big problems with the current rating system. The first is that Apple hasn’t built bug reporting into the App Stores, so customers use one-star ratings as a way of reporting bugs.

“One star: didn’t install for me.”

What can we do for this customer? Well, I can’t contact the people who post ratings, so there is no way for me to actually help the customer with the problem. No, “Okay, sir, I’m sorry you’re having an issue. Is there a chance it’s something with your setup?” just an unhappy customer who feels ignored.

The second problem is that people are much more likely to complain than to praise. If you have an app that crashes for 0.01% of your users, those users are generally not going to quietly ask Apple for their money back. They’re going to post a negative review on the App Store. However, the 99.99% of people for whom it works great are not going to post five-star ratings for the software working as expected. You don’t write a letter to Starbucks every time they don’t burn your coffee.

Now, if an app has 100,000 users, and even 0.01% of them have a problem, the app ends up with ten one-star reviews. Is ten a big deal? Yes, it turns out, because of the third problem: the App Stores only show the ratings given to the latest version of an app. Every time a minor update is released, the app’s ratings history is effectively wiped out. If you look at the reviews of Delicious Library 2 right now, you can see that our latest version has four one-star and one two-star reviews. Since Apple resets the rating of apps with every minor version, our composite rating is now one and a half stars.

One and a half stars means starvation for us developers.

The total number of users who’ve rated this particular version is only six. Never mind that 113 people have rated our app before—if you look at the “all versions” rating, our rating is a much more acceptable four stars. But the “all versions” rating is hidden below the “current version” one. The “all versions” rating isn’t the one shown in the results matrix when you search the App Store. Nobody is ever going to click through to an app that’s showing one and a half stars to discover that its real rating is four stars.

Five ratings can be death for a company.

The App Store’s version of Delicious Library 2 did indeed have a crasher we introduced as we tried to make it sandbox-compliant. So three of those reviews appear to be on the money. Again, it only happens for a handful of people, we’ve fixed it, and the new version has been submitted to Apple and is currently on our website.

But those one-star reviews aren’t going anywhere.

And what if the review is just wrong? If you look at the topmost review on the Delicious Library 2 page (as of right now), you can see we didn’t cause the crasher—our customer put his SQLite database file in Dropbox and eventually it got corrupted. Of course he’s upset. But this is the number one review, and it’s not actually our fault. We recommend against sharing our data files because it’s a crapshoot—sharing a database that you are actively modifying without any locking protocol sometimes works, but sometimes leads to file corruption. You have to have some technical knowledge to even get Delicious Library 2 to look in the Dropbox folder. Unfortunately, when it went wrong this user blamed us.

And we get one star.

Further, we can’t contact the user to tell him that he’s corrupted his file, and he needs to go to his backup, because the Mac App Store doesn’t allow us to respond to reviews.

We have a “contact us” menu item under our Help menu, but if users decide to post reviews instead, we cannot provide any tech support. So we also can’t tell the other four people that their crashers should be fixed now, and they should update to the version on our website (or wait for the App Store approval). Apple’s customers are Apple’s customers, except we’re supposed to support them, but we aren’t provided the tools by Apple to do so.

From four stars to one and a half stars because of five users whose problems we really want to fix (or have already fixed).

But we can’t contact them.

Luckily for us, Delicious Library 2 isn’t where we’re making most of our money —we launched Delicious Library 3 this year. Except: when we first released it, it had the same rating problems. Our earliest customers were unsurprisingly the users of version 2, and a percentage of them liked the look of 2 better. Now it’s no insult to me if you like my old work better. But several of these customers bought Delicious Library 3 without trying the free version that’s on our website. This was easy to do, since we aren’t allowed to have trial versions directly on the App Store.

Then, after buying from the App Store, these customers wanted to go back to version 2. Which would be fine if they’d bought directly from us, or if they’d used the trial version. In the App Store case, we’re in an awkward situation. While we happily refund customers’ money—in this case we only made $17 from the $25 they spent. If we refund $25 directly to the customer, we’re actually losing $8. Furthermore, since Apple’s customers are Apple’s, we don’t have a list of who has actually bought our software; we have no way of knowing if someone complaining is actually our customer, or just someone who heard he can get $25 from us by writing us saying, “I want to go back to version 2.”

Many of these customers don’t even talk to us, or ask the App Store for a refund. They just post one-star reviews: “I don’t like the look, preferred version 2.” We ended up with a rating that was around three and a quarter stars. We were losing money.

In desperation, I stuck in a rating pane when we did version 3.1. It reads:

Ok, we hate these stupid “Please rate our app” panels as much as you do, but since we’re in the Mac App Store we live and die by your ratings, and of course you can’t rate an app the moment you buy it, since you don’t know what it’s like yet.

So, in the politest way possible, we’d like to ask—just this once—that if you like our app you go back and rate it, so we can stay in business and continue making cool software.

The panel only kicks in for people who’ve been using Delicious Library 3 for a while and have scanned a number of items. It defaults to never showing itself again if you dismiss it, and (because it’s on OS X) it doesn’t have to quit Library to open the App Store if you do say yes.

Our rating shot up to five stars within a week of releasing this version. They’ve now settled at a very respectable four and a half stars. We’re making money, and we’ve started work on our next app.

None of this is to say I like being bothered to rate an app—I resent it strongly when it feels like coercion. But, as Daring Fireball itself noted in followups, a lot of this is in how you ask, not that you are asking.

Part of the problem is that once developers get it in their head they should ask users for ratings, they lose perspective on what’s polite.

Some ask specifically for five star ratings, which I think is incredibly presumptuous. Some ask multiple times, and have confusing (or missing) options to opt out of them asking again. And some ask even after you’ve rated the app. For iOS apps, all of them require that you leave the app to rate it, which is kind of a pain. You obviously launched the app because you want to, like, actually use it. All of this creates enmity in the users, and, yes, makes us want to rate their apps with one star just to shut them up.

I assume, by and large, developers have never been waiters. They don’t know that if they go to a table and say, “Hey, I’d really appreciate it if you guys would tip me 20%, but, also, you can’t do it here—can you stop eating for a minute, come out back, and give me a tip now, before your meal is over?” they’re just going to end up with customers that resent them. If they keep doing it, every time they bring another course? If, after the customer has paid the check, they still do it? Well, they’re fired.

So good sense is required here. Part of the problem is that Apple hasn’t provided developers with any APIs to tell if users have rated our apps, or to make it possible for users to rate apps inside the app itself. This would obviously make it less intrusive for users.

Since Apple resets apps’ ratings with every tiny release, developers find themselves wanting to ask users over and over for ratings, since they get wiped out all the time. But no customer should be asked to rate an app twice—it’s like that clueless waiter asking a customer to tip twice.

With the App Stores, Apple has created a micro-economy where ratings are king. However, as with any tightly controlled economy, tiny flaws can blossom into huge problems. With App Store ratings getting reset on every minor version, we are actively discouraged from updating our apps once they have good ratings. I’ve just submitted a new version of Delicious Library 3, and I’m scared out of my head that the first two or three people who review it won’t like it, which will tank it to the extent where nobody discovers it any more, so there won’t be any positive reviews to balance them out.

This system also sucks for customers, who put the time into reviewing an app that—as soon as a new minor version comes out—is thrown away. Only the people who rate the very latest version of any app have their vote count.

Both the Mac App Store and iOS App Store need to allow secondary links to trial versions, show the “All versions” rating on search results, give us a way to engage customers who are having problems (even if it’s totally anonymously), and make it clear how customers can get refunds on software so we don’t get one-star ratings instead. For the iOS App Store, Apple also needs to provide a standard way for users to rate an app without leaving the app, and let developers know if their app has been rated so they won’t bug the user again.

In the meantime, developers need to treat their customers with respect: ask for a rating only once, don’t suggest that five stars are good (I mean, duh?), apologize for being gauche, and don’t ask until the user has used the app enough to get a feel for it.

Wil is the Chief Monster at Delicious Monster, creators of Delicious Library. He started programming when he was 12, taking college courses in BASIC and Pascal and generally making himself unpopular with the undergrads. He founded The Omni Group while a student at the University of Washington, and helped design and write OmniWeb, OmniPDF, and OmniGraffle among others. Under his leadership Omni’s products won a record five Apple Design Awards, and Delicious Monster has won another three.

Wil’s Website | Wil’s Twitter



  • Jwcorey

    As you’ve indicated in your final paragraph, there’s requesting a review and then there’s “requesting a review.”

    I don’t mind: An app (especially if it’s free) asking for a review after I’ve used it for a while. It also helps if the app happens to be a great app and not just an attention-trapping device that constantly steers the user towards IAPs.

    What I don’t like: An app that doesn’t give me the option to say “no thanks” (i.e. just offering a “remind me later” button and not a “no thank you”); an app that asks for a review every time I start it up; when an app costs money AND weights the experience towards IAPs AND asks for a review (because, frankly, after a certain point I feel like I’ve done quite enough for you, thankyouverymuch).

    Honest people who do good work should be allowed to modestly ask for a hand up, and good people shouldn’t mind giving it. But when a developer doesn’t respect the experience of the user and sees them only as sheep to be sheared, you can smell it from a mile away. In those situations, I’ll one-star the review without hesitation or remorse, regardless of what the app is or how it works.

    The way to avoid the fallout of Gruber’s one-star campaign isn’t to plead to users about how much money you need/want/deserve to make. It’s finding a way to get users to give you what you need/want/deserve by creating something so good that they see it as their pleasure to give it.

    • Piyush

      So let’s settle it at that – If an app asks for Review, without a permanent dismissal option, and/or messages too early even without trying the app for some time, we will definitely give it a 1 star rating. App developers, you are listening?

  • Odi Kosmatos

    Flappy Bird has a ton of 1-star reviews. It’s super popular. [REDACTED, my game] got a ton of 4.5 and 5 star reviews and nobody ever noticed it.

    • reallyronswanson

      The redacted game in question is Pickpawcket, it’s a great game! FIVE STARS

  • Herding_sheep

    Gruber obviously understands the difference between politely asking for a review, and obnoxiously badgering for one.

    Those are the applications that deserve one star. Several apps continue to ask me for reviews on a regular basis, even AFTER I’VE RATED IT. That’s annoying, and worthy of a 1 star. I don’t mind a single pop up that encourages a review, and then lets me decide. I do mind something that constantly interferes with my experience.

    You developers live and die by making great software that people enjoy using.

  • def4

    I don’t mind being asked for a review. If I use the app heavily then it deserves my appreciation. If I don’t then I don’t really see the review request anyway.

  • Dad

    Great post!

  • http://ouriel.typepad.com OurielOhayon

    I think the first step of solving the issue you describe the right is for developers to provide a way to collect feedback correctly. After all asking for rating is just one form of feedback and unfortunately you can t prompt randomly a pop up asking for reviews because it is more than likely you will piss off happy users or just stumble upon unhappy users.

    The right approach is to create (or use) an elegant well integrated feedback system and possibly reroute the users to the app store for reviews.

    Asking for feedback or proposing a clear call to action for feedback is already going to take away of the undeserved 1 star rating from the store and route them to the developer so he can get a chance to handle them

    This is a system we created and invented 3 years ago. It is available for anyone to use for free on ios and android

    http://blog.appsfire.com/app-store-boost-your-ratings/

  • http://www.kevwil.me/ Kevin Williams

    “Small OS X and iOS software companies live and die by their App Store ratings.” – That is a stupid problem to have; a problem Apple created and Apple should fix.

    “…. We ended up with a rating that was around three and a quarter stars. We were losing money.” Is it really that fewer stars = less money, or an app which had problems and/or dissatisfied users = less money? I suspect the author thinks the former, which goes to my point that Apple has created this “we live and die by ratings” mindset, which created the nagware problem & the resulting backlash.

    As the author points out, the many problems in the App Store ratings system are creating many false assumptions and poor decisions to be made. With a proper feedback/support system and a removal of nagware prompts, the ratings might take on actual meaning.

  • Joe Chin

    I find the per version rating system more effective. If you release a 1.0 product with lots of bugs you’re dead. Release an update that fixes the bugs and your clock resets. Should you be dragged down by old buggy versions? Should you abuse the credibility of previous ratings to introduce nagware and fullscreen ads? I don’t think so.

  • Dev Dutt

    Every app developer wants his app reviewed on playstore, as it’s probably one of the best way to gain a lot of exposure and get downloads on playstore. So how does one get such a review? Well, if you’re not one of the big guys yet you need to ask for it. And you need to do it the right way.

    If you’ve tried it before, you know it’s far from easy to get a free review (don’t even bother with paying for one – it’s proven in most cases).

    So we have build a review platform for the android app developer’s

    Go to http://www.reviewmyap.com

  • Rnni

    Great article Wil. I couldn’ agree more. There are users that love you app, lots of them obviously. However, only the ones who don’t go (almost) go and rate it. Resulting in bad reviews, less downloads, less revenue.

    I’ve been reading an interesting article on how to naturally boost your 5 star ratings, the keypoints where the followings: 1. Identify true fans 2. Identify moments of joy 3. Ask them to rate, at the right moment 4. Show the benefit the rating can bring to the developer (new updates, more features, etc) 5. Reward them 6. most important, as you mentioned, ask it the right way. The wording is very important, use humour, fun, etc.

    Check out the full article : https://polljoy.com/blog/improve-app-store-rating