One of the biggest challenges when testing mobile applications and games is the selection, sourcing, and management of devices. The smartphone market is buoyant, with countless new models released every year. The software running on those smartphones is varied. For most developers the idea of maintaining a full inventory of mobile devices is simply not realistic.
How do you ensure good test coverage for your app without breaking the bank or creating a logistical nightmare? There are a few different broad strategies that you can employ. For most developers a hybrid approach is going to be the best fit.
Identify target devices
This is a simple numbers game. What are the most popular devices? What devices is your app most likely to run on? Take a look at the market. Samsung accounted for 22.2 percent and Apple accounted for 16.1 percent of the total market in 2015, according to Strategy Analytics. Digging deeper, you can find the most popular models and filter according to your target demographics and geographical markets.
You want to draw up a shortlist of the most important devices, covering as much of your target market as possible. These are the devices that it’s vital your app can run perfectly on. You won’t be running the full suite of tests on every device that might run your app, so target your resources carefully to get the best from them.
Real devices vs emulation
To keep the burden down, you’ll probably want to use an emulation service. You can test on a much wider set of devices using emulation, at a fraction of the cost, but remember that nothing is a complete substitute for having a device in hand. From a usability perspective, always make sure you have at least couple of target devices physically in the office for testing.
Manual vs automated testing
Just as you might employ emulation to keep the device burden down, you should consider automation to prevent your manual testing burden from getting out of hand. Anything that’s going to be retested frequently is a good candidate for automation.
When you’re deciding what to automate and which tools to use, consider what will integrate best with your device management strategy. It will be easier to employ automation with emulators than on physical devices. Just watch out for test automation pitfalls.
Internal vs outsourced
If you have internal testing expertise and a high volume of projects then it might be worthwhile working out how to handle mobile testing yourself, but for many developers outsourcing is a more sensible option. Companies with experience in mobile testing will be equipped with the latest devices. They’ll be able to tell you where you should be targeting your resources and they’ll handle the process for you.
To get maximum test coverage within budget you have to find a good balance. Focus your efforts on the most important mobile devices. Find the right blend of physical devices with emulation, manual testing with automation, and internal testing with outsourced testing. A carefully planned mix will garner the best results and deliver the best quality mobile app.