Smartphone sales are expected to top 1 billion and tablet sales will push past 220 million for the second year running in 2014. The Android and iOS app stores offer more than 3 million apps between them. When we look at how people spend time on mobile devices, we find that apps are dominant and app usage is still rising.
Mobile apps are here to stay, but they represent some serious challenges for testing departments. There are difficulties surrounding fragmentation, with platform versions and multiple devices featuring different hardware. You naturally have to cover functional, usability, performance, and security testing. But none of that is new.
What’s different about mobile devices?
It’s not just the mobility that really sets mobile devices apart from a testing perspective. There are a number of important things you need to consider that wouldn’t enter into the equation for a desktop application.
Interruptions – they are expected to deal with interruptions, like incoming calls, by switching function and then to switch back again seamlessly to the same point at the end of the call. They also have to deal with notifications for incoming messages or emails gracefully.
Networks – they have to switch networks in the background, sometimes between different cellular bands, sometimes from cellular to Wi-Fi, without impacting your app experience. The app also has to handle different signal strengths or the complete lack of a signal.
Battery – mobile devices have limited battery life and so efficiency is at a real premium. If your app makes the device work over time and guzzles all the battery juice it’s an absolute disaster for a smartphone.
Security – a device that could be exposed to risk in a variety of locations needs a little extra security consideration. If your app syncs to the cloud you need to ensure private data stays that way. How is it stored on the device, on the server, and in transit? It’s not just the risk of malware, but also close-range attacks that exploit features like Bluetooth and NFC.
Storage – just like battery life, storage is at a premium on mobile devices, so you need to take extra care to make sure your app isn’t generating junk files. What happens when the storage is full? Is temporary data being cleared correctly?
Real world versus testing lab
The majority of testing is traditionally conducted by a QA department in a fixed location. If you test mobile apps in a lab, then you never find out how your app handles signal strength fluctuations or changing network connections. Do you make a point of testing incoming calls and messages? Emulators make it possible to cover a wider variety of hardware, but they aren’t fully representative of the actual device.
What’s really desirable is to get as close to real world conditions as possible in order to uncover potential defects. You want the app tested on a wide variety of different devices and ideally in a wide variety of different locations in order to build a good picture of its stability and performance.
If you’re testing the app in isolation then you may be getting a false picture.