How to manage mobile devices for maximum test coverage

By | May 23, 2016

One of the biggest challenges when testing mobile applications and games is the selection, sourcing, and management of devices. The smartphone market is buoyant, with countless new models released every year. The software running on those smartphones is varied. For most developers the idea of maintaining a full inventory of mobile devices is simply not realistic.

How do you ensure good test coverage for your app without breaking the bank or creating a logistical nightmare? There are a few different broad strategies that you can employ. For most developers a hybrid approach is going to be the best fit.

Identify target devices

This is a simple numbers game. What are the most popular devices? What devices is your app most likely to run on? Take a look at the market. Samsung accounted for 22.2 percent and Apple accounted for 16.1 percent of the total market in 2015, according to Strategy Analytics. Digging deeper, you can find the most popular models and filter according to your target demographics and geographical markets.

You want to draw up a shortlist of the most important devices, covering as much of your target market as possible. These are the devices that it’s vital your app can run perfectly on. You won’t be running the full suite of tests on every device that might run your app, so target your resources carefully to get the best from them.


Real devices vs emulation

To keep the burden down, you’ll probably want to use an emulation service. You can test on a much wider set of devices using emulation, at a fraction of the cost, but remember that nothing is a complete substitute for having a device in hand. From a usability perspective, always make sure you have at least couple of target devices physically in the office for testing.

Manual vs automated testing

Just as you might employ emulation to keep the device burden down, you should consider automation to prevent your manual testing burden from getting out of hand. Anything that’s going to be retested frequently is a good candidate for automation.

When you’re deciding what to automate and which tools to use, consider what will integrate best with your device management strategy. It will be easier to employ automation with emulators than on physical devices. Just watch out for test automation pitfalls.

Internal vs outsourced

If you have internal testing expertise and a high volume of projects then it might be worthwhile working out how to handle mobile testing yourself, but for many developers outsourcing is a more sensible option. Companies with experience in mobile testing will be equipped with the latest devices. They’ll be able to tell you where you should be targeting your resources and they’ll handle the process for you.

To get maximum test coverage within budget you have to find a good balance. Focus your efforts on the most important mobile devices. Find the right blend of physical devices with emulation, manual testing with automation, and internal testing with outsourced testing. A carefully planned mix will garner the best results and deliver the best quality mobile app.



Category: Quality assurance testing Usability testing Tags: ,

About Simon Hill

Simon is an experienced freelance technology journalist covering mobile technology, software, and videogames for a wide variety of clients in print and online. He regularly contributes to Digital Trends, Tech Radar, and Android Authority, and he ghostwrites for CEOs in the technology space. After completing a Masters in Scottish History at Edinburgh University, he began his career as a games tester, progressing to lead tester, game designer, and finally producer, before leaving the industry to write full time. He is passionate about the potential for good software and hardware to improve our lives, and strongly believes that thorough testing is a vital prerequisite for greatness.

  • Satyam

    Nice write-up Simon. Just to add on couple of points
    -When identifying target devices ofcourse the popularity in your target market is the big criteria. But depending upon the needs of testing of your application identifying devices based on factors like OS versions, size and also aspects like processor (arm and x86/64) etc is also beneficial. You might want to test on these different configs as well.
    -Also yes the option of emulation does help in increasing the device covergae. And options like emualtors on local machine and device clouds are there.