How to Increase Your Mobile App Sales With A/B Testing

Marketing your mobile app can often feel like a dark art.

With over 600,000 apps competing for sales in each app store, an app’s icon acts like a mini banner ad, so choosing the right one can mean the difference between success and failure.

But when you’re presented with 3 icon options, how can you be certain which one will perform the best?

If you drop your price from $1.99 to $0.99, will your sales double? Or triple? Or more?

Should your description start with your ranking “#3 Best Selling Zombie App!” or with a list of features? Which will convert better?

Many app marketers will do their research, compare competitors and best sellers and then answer these questions based on gut feeling and intuition.

The iTunes app store and Google Play don’t allow for proper, scientific A/B testing, so marketers are forced to make their choices this way.

How to “Micro Test” Your App

There is a better way to make these decisions. Let me introduce you to “Micro Testing,” a method that lets you test all of these marketing elements — your icon, price, description and more — and then apply your findings to the app store.

Although some of the bigger app development companies are already using similar methods, the practice is still relatively unknown in the app marketing community.

I’m going to explain here, step by step, how you can use free and inexpensive tools to create your own micro tests and optimize your app store marketing today.

Micro testing is based on this simple premise: We can’t perform A/B testing within the app store, so we’ll mimic the app store experience and run our experiments there.

How to Find the Perfect Icon

TapTapTap, a well known app development company, shared the story about how they picked the wrong icon for one of their apps, which lead to a disastrous launch.

During development they had 3 design concepts to explore. Each person on the team had a favorite. Everybody had their own beliefs about which icon would perform best.

taptaptap app icons thumbs

Unfortunately, they had no clinical, objective facts on which to base a decision.

Everyone had their own opinions on what concepts they liked the best and what should be changed or tweaked. This lead to some pretty heated arguments.

The story goes on to explain how, in the absence of objective data, design-by-committee took hold and produced a generic, middle of the road icon that performed very poorly. This is where micro testing should come into play. To help you avoid making the same mistakes, we’ll show you how to create your first micro test in 3 steps:

  1. Create an A/B Landing Page
  2. Send Traffic to Test Your Pages
  3. Analyze Your Results

This same process can be used for all the app’s marketing elements. We’ll use icons to illustrate the steps in detail, and then touch on pricing, description and screenshots.

Step 1: Create a Mobile Landing Page

A landing page is a single web page that will showcase your app with the same elements that a customer might see in the app store “an icon, description, screen shots and buy now button. Even if this is a new app, you should create your landing page as if the app is available to purchase, with a main Call-to-Action (i.e., a big button) saying Available on iTunes “Install Now”.

app store CTA

The goal here is to replicate the experience users will have when they view your app in the app store in order to test how many actually will try to buy it when presented with the different icons.

Landing Page Resources

You have 3 main options for building your mobile landing page:

  1. Custom code it or hire a developer on elance or oDesk. This is the most time/cost intensive, but it gives you good flexibility.
  2. Use a template from a site like Themeforest. They cost about $10, but this requires some basic coding knowledge.
  3. Use a mobile landing page builder like Pijnz, Landr, Atmio, Convrrt or SparkPage.

In this guide, we won’t go into detail on how to modify html templates or upload them to your server; you should pick those options only if you know the basics of coding and have the time to invest.

building a landing page

Building a landing page

Set Up an A/B Test

Now that you have a landing page built, you need to create several variations of that page. Each variation will display a different icon, but the rest of the page content will remain exactly the same. In the screenshot below, you can see an example of 4 different landing pages used to test 4 slightly different logo styles.

kissmetrics app logos split test

A/B tests will randomly split traffic among these test pages to show what percentage of visitors click on “Get it on iTunes.”

If you’ve custom built your landing page using a template or html, you can use a service like Optimizely or VWO to create your variations.

Right now, Optimizely has mobile A/B testing in only their higher price plans; but if you do choose to use them, you can integrate directly with KISSmetrics to track your results.

Step 2: Find Testers for Your Test

Once your landing page is ready to test, you need to start sending people to it. You can do free promotion, such as sharing the link on Twitter or Facebook, but we recommend a small ad campaign for a reliable, non-biased flow of users to test your icons.

By running a small ad campaign, even as low as $50, you can send hundreds of visitors to your landing pages.

This is another reason it’s important to keep your variations to a minimum. Testing 2-4 icons is optimal; but for any more, you need to send larger and larger amounts of traffic to get significant results (which is fine, of course, if you have the budget).

You can use AdMob, Google Adwords or any other mobile ad network to run this small campaign.

You can use a network like AdMob for a quick and easy set up, but a more targeted network like Google Adwords can have its advantages, too. For example, if you’re testing the icon for your latest Zombie Game, you can target ads at people searching “Download Zombie Games” on Google, iPhone and Android. This will ensure that you test your icon with the right kinds of users.

AdMob

Step 3: Get Results

As your ad campaign sends traffic to your page, you should begin to get a clear picture of which icon is resulting in the most clicks on the “download” button.

If you build the landing page with custom code or a template, you also will need to use a service like KISSmetrics or Optimizely to track the number of clicks on your “download” button. To set up tracking with KISSmetrics, you just need to create a new event, as described in this help page.

variants results close

A sample micro test showing “Icon 1″ as the winner

You should run the experiment over a couple of days until you get at least 100 clicks on the download button to give you an accurate picture.

Although the test can’t make you 100% certain, you’ll quickly get a sense of which icons are resonating with users and which are turning them off.

Even small increases from one icon over another can have a huge effect. The extra sales they encourage can mean the difference between making it into the Top 10 chart or being noticed and featured by Apple.

Both of these would skyrocket your app’s sales. This is a much more scientific approach than relying on your gut or intuition to pick the best icon. This is how to set up your first micro test to help you pick the perfect icon for your app. Using the same method, we’ll now run through the other main elements you can test — pricing, description and screenshots.

What Is Your Most Profitable Price Point?

Here’s an example of a common conversation I have with app developers before they launch.

One person was torn between a $0.99 and a $1.99 price tag for their next app.

“I know,” the developer said, “that I’m more likely to buy an app if it’s $0.99 rather than $1.99.”

“But how much more likely?” I asked. “Twice as likely? Three times as likely? Or 50% more likely?”

“Hmmm… I don’t know!” was the response.

This is the key question you need to answer before setting a price. If you double your price from $0.99 to $1.99, will your sales fall by more or less than half?

I’ll illustrate this with an example:

If you sell 100 apps in a day at $0.99, you’ll earn $99. So what happens when you increase your price to $1.99?

  • If sales drop to 70 per day, you will make $139
  • If sales drop to 55 per day, you will make $109
  • If sales drop below 50 (half) to 45, you’ll make only $89

If you double the price, how far will sales fall? Economists call the answer to this question “elasticity”, and knowing your app’s elasticity is vital for picking your most profitable price point.

As with icons, micro testing can be used to find the most profitable app price.

For testing your price points, you can follow the same steps as before, but make sure that you include the price as a prominent element on each page. All other elements should remain constant, so that you’re seeing only the effect of pricing.

split testing price points

Making Sense of Your Pricing Results

As your test is running, you should begin to get a sense of the conversion rate for each price point.

Although KISSmetrics, SparkPage and Optimizely will calculate this for you, here’s the simple formula if you need it:

(Number of Clicks / Number of Visitors) x 100 = Conversion Rate.

I’ll run through some example figures to demonstrate.

Let’s imagine you tested two price points, $0.99 and $1.99. Each landing page had 1,000 visitors.

Page A ($0.99) had 250 clicks on the download button. This gives us a conversion rate of 25%.

Page B ($1.99) had 150 clicks on the download button. This gives us a conversion rate of 15%.

But what does this mean for your overall revenue?

The simplest way to compare the two is to multiply the price by the conversion rate and see which is bigger.

  • Page A: $0.99 x 25% = 24.7c
  • Page B: $1.99 x 15% = 29.8c

This tells us that every visitor to Page A earns us 24.7c, but every visitor to Page B earns us 29.8c.

If an app marketer uses micro testing to pick Price B over Price A, as in this example, the marketer would stand to increase total revenue by 20.6%

(Note: This doesn’t show you what your conversion rate will be in reality; it just shows you the difference between price points. Use micro testing to pick the optimal price, but not to forecast sales.)

Discover the App Description that Sells

Of all the elements in an app store listing, the description still seems the least understood, even though it occupies most of the page’s real estate.

If you open an app store now and look through the top selling apps, you’ll see a wide variety of description styles and tactics. Some start with a marketing message and some with user reviews or worldwide rankings. Others list app features, and some start with the newest features in the most recent update.

descriptions a/b app testing

There’s no clear “best practice” when it comes to descriptions, which can leave app marketers stumped.

Should it be a really long description to convince anybody who wants to read about the app?

Should it be short and sweet? Just enough to get them interested? What about bullet point feature lists — they seem popular?

Fear not; once more, data driven decision making will come to the rescue!

Running Your Description Micro Test

For this micro test, you can build your landing page as before, but make this one very light on imagery. We want to isolate and test the effect of descriptions on conversion rates, so excess imagery can distract from this goal.

Unlike the other experiments, you should place your Call-To-Action above the description text here. This may seem paradoxical at first — after all, you want to test clicks after people read — right?

Not quite! You want to assess what description style works best in the app store, so you want to mimic the iTunes store as closely as possible. In the iTunes store, the download CTA is above the description.

cta above description

As with the other tests, you want to keep the number of variations relatively low so that you don’t have to spend too much sending traffic to each page.

Because of this, you might want to test your descriptions in two phases:

First, start with broad description styles. For example, start with rankings on one page, features on the next and a catchy tagline on the third.

Then drill down into the best style, and test different variations of content. For example, if a features list works best for your app, run a second round of testing to experiment with different features in the list, different list lengths, etc.

Find the Screenshots Users Love

At this stage in the guide, hopefully, you see a pattern forming!

Screenshots are often underrated when it comes to app marketing. They aren’t as prominent as the icon or the price, and they appear “below the fold” in both app stores.

But they can be a vital part of encouraging a user to purchase your app. They are the closest a user can get to trying out the app before downloading it. They are your window display.

A screenshots micro test operates much like the others, but with one key difference. As we want to replicate the app store experience as closely as possible, screenshots are best presented in a side scrolling list, where the user “swipes” through them one by one.

swipe screenshots

Other Uses and Caveats

Ideally, this guide has given you a good introduction to the concept of micro testing to optimize your app store presence.

If you try these tests yourself, there might be other elements you can experiment with or different ways to test the elements we described. If you do have any success or ideas for other tests, please leave a comment and let us know.

Finally, it is important to consider the results from these tests in the wider context of your app strategy.

For instance, testing could show that a higher price might result in higher profit, but you could still choose to keep the price low in order to build up your initial user base, if social sharing is a big part of your strategy.

Micro testing can be an incredibly powerful optimization tool. We trust that this guide will help you get more scientific with your decision making and that you will be able to promote your apps even more effectively.

About the Author: Peter Tanham is the co-founder of SparkPage, a platform for creating mobile-optimized landing pages. At the time of publishing SparkPage is in private beta, you can request an invite here.

  1. Excellent ideas and strategies:)

    ~ Clint
    @Cazoomi

  2. I don’t understand how the pricing one can be tested if eventually they go to your store to purchase and may see another price.

    • Hey Anna,

      What you’re really trying to measure here is the difference in click through rate at each price point.

      Because Apple or Google won’t let you do A/B testing with the actual pricing, this method lets you test by proxy. So if you get double the number of people clicking on 99c vs $1.99, you can deduce that roughly double will purchase at 99c vs $1.99

      Of course 100 cilcks at 99c won’t mean 100 sales at 99c, but it does show you the relative difference between prices, which is what’s important here.

      I’ve mostly run these experiments before an app is even launched, so instead of visiting an app store, when the person clicks “Download Now” they see a page that says “This App is Coming Soon – Enter Your Email Address to Be Notified When We Launch” with an email signup form.

      Remember, this isn’t a marketing activity with a goal of making sales, it’s a research activity with the goal of finding your most profitable price point.

      Hope that helps!
      Peter

  3. These are amazing tips. Thanks for sharing them with us. I would have never though of such a strategy to increase the sales of my mobile apps. Looking forwards to trying them on my own.

  4. Beny Schonfeld Oct 01, 2012 at 8:55 am

    I like the methodology. It’s simple and covers 3 of the major factors that affect App Store conversions (testing Screenshots would be interesting as well).

    Any suggestions on how to test freemium model pricing prior to launching in the App Store? I think the data would be severely skewed if we amply tested a “free” option along with the possible prices for the in app upgrade to full version. Simply becuase the test would not account for people downloading and testing the app and then being called to upgraded to the full or pro version.

    Any thoughts on how to test this scenario in order to optimally set the upgrade price?

    Thanks

    • Beny Schonfeld Oct 01, 2012 at 8:58 am

      Correction: not “amply”… Meant “simply”. Fat thumbs…

    • I didn’t want to go too in depth on this in the “caveats” at the end, because the post was already really long, but you’re correct, it’s not directly applicable for freemium models.

      It is possible, though, to apply the same principles to experiment with the pricing of upgrades.

      I haven’t done this, but here’s how I’d go about planning the experiment:

      1. Build a similar landing page, as if it were an upgrade screen within your app, with upgrade features and pricing.

      With a SparkPage page you’d have a URL, so this will just open in the browser on the app.

      2. Now you need to send a small number of your free users in your app to this page.

      You could do this with a push notification (e.g. Urban Airship), or if you want to get more in depth you might check out http://swrve.com/ or https://clutch.io/

      Or if you already have free & premium apps in the market, change it slightly so that when somebody hits “upgrade” on the free app they’re brought to the SparkPage which is running the test.

      Using the same techniques in the post above, you could split test pricing with a sample of your existing free users. I imagine doing 500 out of 10,000 free users would get you some good data on the optimum upgrade pricing.

      Does that sound like it might help test your upgrade price?

  5. Your screenshots show the full iPhone. I assume that when visited on a desktop computer, you encounter a normal looking web page with the elements discussed and you are not trying to mimic the look of the App Store.

  6. Great tips for app icon, pricing and screenshots!

    We’re working hard on Pathmapp, a Real-time Mobile A/B Testing platform. Check out our TechCrunch Disrupt Video and sign up for the beta at http://www.pathmapp.com. Would love to get any feedback on the platform.

    Thanks!

  7. ӏ leаѵe a rеѕponse ωhen I espеcially еnjoy a poѕt
    οn a blog or I have sоmеthіng
    to valuable to сontribute to the conveгsation.

    Іt іѕ a гeѕult οf the paѕѕion cоmmunісated in the post I browsed.
    And afteг this artіclе How tο Incrеase Your
    Mobilе App Sales Wіth Α/B Τesting.

    I wаs actuаlly movеd enοugh to pοѕt а
    thought ;) I aсtuаlly do have a few questiоns
    foг уou if уοu ԁο nоt mind.
    Іs it juѕt me or do somе of thе commеntѕ come across like they are written by braіn dead indіѵiduals?

    :-Ρ Αnԁ, if you аre wrіting on οthеr placеs, I ωοuld liκe to keep up with уοu.
    Woulԁ уοu make a lіst all of аll
    yоur social pages like уour Fаcebook
    page, tωіtter fеed, or linkеdіn pгofile?

  8. Hi,

    Excellent work man!
    Keep on working.Thanks for sharing your idea with us.

    Thanks

  9. Thanks a lot for sharing this with all people you really understand what you
    are talking about! Bookmarked. Please additionally seek advice from my web
    site =). We could have a link trade arrangement between us

  10. Hy

    Excellent man!I love your idea your tips and the way you write this article.

    Thanks

  11. Thank you for any other informative website.

    Where elѕe could I am getting that κind of informаtiοn writtеn in such an іdeal manneг?

    I’ve a challenge that I’m just nοw runnіng on, and I have been at the glance out for ѕuсh іnformation.

  12. Excellent post man!
    I really like the way you write this post,thanks for sharing.

  13. Hy,

    Great article man!Nicely written,easy to understand.thanks for sharing keep on doing.

    Thanks

  14. Hey Peter !
    Really great post. I just want to ask that, Does micro testing helps to determine a successful name??
    Over all i really liked this post.
    Thanks
    Matt

    • Hey Matt,

      Yes, the same principles all apply for micro testing an App’s name – see which of your options resonates more with users and encourages them to download.

      The one caveat with App names (and the reason I left it out) is that they also play a big role in the App Store SEO.

      For example, if you call your App “Find a Taxi” but most people are searching “Find a Cab” on iTunes, it could lower your ranking in the search results, thus impacting sales.

      So short answer yes; long answer yes but be mindful of the search impact.

      Hope that helps,
      Peter

  15. Very nice article. I will sure try it with my new app project

Comments are closed.

← Previous ArticleNext Article →
Buffer