The importance of real world testing for mobile apps

By | October 20, 2014

Smartphone sales are expected to top 1 billion and tablet sales will push past 220 million for the second year running in 2014. The Android and iOS app stores offer more than 3 million apps between them. When we look at how people spend time on mobile devices, we find that apps are dominant and app usage is still rising.

Mobile apps are here to stay, but they represent some serious challenges for testing departments. There are difficulties surrounding fragmentation, with platform versions and multiple devices featuring different hardware. You naturally have to cover functional, usability, performance, and security testing. But none of that is new.

mobile app testing


What’s different about mobile devices?

It’s not just the mobility that really sets mobile devices apart from a testing perspective. There are a number of important things you need to consider that wouldn’t enter into the equation for a desktop application.

Interruptions – they are expected to deal with interruptions, like incoming calls, by switching function and then to switch back again seamlessly to the same point at the end of the call. They also have to deal with notifications for incoming messages or emails gracefully.

Networks – they have to switch networks in the background, sometimes between different cellular bands, sometimes from cellular to Wi-Fi, without impacting your app experience. The app also has to handle different signal strengths or the complete lack of a signal.

Battery – mobile devices have limited battery life and so efficiency is at a real premium. If your app makes the device work over time and guzzles all the battery juice it’s an absolute disaster for a smartphone.

Security – a device that could be exposed to risk in a variety of locations needs a little extra security consideration. If your app syncs to the cloud you need to ensure private data stays that way. How is it stored on the device, on the server, and in transit? It’s not just the risk of malware, but also close-range attacks that exploit features like Bluetooth and NFC.

Storage – just like battery life, storage is at a premium on mobile devices, so you need to take extra care to make sure your app isn’t generating junk files. What happens when the storage is full? Is temporary data being cleared correctly?

Real world versus testing lab

The majority of testing is traditionally conducted by a QA department in a fixed location. If you test mobile apps in a lab, then you never find out how your app handles signal strength fluctuations or changing network connections. Do you make a point of testing incoming calls and messages? Emulators make it possible to cover a wider variety of hardware, but they aren’t fully representative of the actual device.

What’s really desirable is to get as close to real world conditions as possible in order to uncover potential defects. You want the app tested on a wide variety of different devices and ideally in a wide variety of different locations in order to build a good picture of its stability and performance.

If you’re testing the app in isolation then you may be getting a false picture.

Category: Quality assurance testing Tags: ,

About Simon Hill

Simon is an experienced freelance technology journalist covering mobile technology, software, and videogames for a wide variety of clients in print and online. He regularly contributes to Digital Trends, Tech Radar, and Android Authority, and he ghostwrites for CEOs in the technology space. After completing a Masters in Scottish History at Edinburgh University, he began his career as a games tester, progressing to lead tester, game designer, and finally producer, before leaving the industry to write full time. He is passionate about the potential for good software and hardware to improve our lives, and strongly believes that thorough testing is a vital prerequisite for greatness.

  • MobileDevice

    Your blogs are so very useful information written in your blog. You explain one by one its really look like user friendly blog. Thanks for the information. mobile device testing provides comes with powerful features such as record and playback, object recognition, and scripting capabilities that allow you to test on multiple mobile devices using a single test.

  • Mobile Device

    Its really informative page. mobile device testing application testing basically includes the normal testing types used for any other application and few things which are specific to the mobile environment.