Best practices to leverage crowd testing services

By | November 19, 2014

Since it started 7 years ago, crowd testing has entered many innovative companies to become a part of their standard testing process.


Although its benefits are undeniable, some users have also criticized crowd testing because they have had bad experiences with it.

Like any service, crowd testing has benefits and limitations.

Here are a few tips to ensure that you make the most of your testing dollars and that the products you ship are flawless.

Clear documentation is a must

Testers, especially external testers, do not have the comprehension that you do of your products.

They only understand its functionality to the extent of the information you’ve provided.

Make sure that you provide enough documentation about the inner-workings of your product so that testers don’t have to guess its supposed to work.

The more complex your product is, the more important it is that you provide documentation so that testers understand its subtleties.

Provide detailed instructions

If a particular aspect of your project is what you’d like to have tested, make sure it is clear in your test request.

Without instructions, testers can only use their best judgement to use your product as a normal user would.

If need be, consider providing a test plan. Testers will follow them and wander off on their own for exploratory testing.

Make sure someone is available to answer questions during the testing period

When you are working with a crowd of individuals, they engage with your project in their own time. You can maximize your return on investment if you can answer their questions as they are working on your product to keep up the momentum.

Given that most crowd testers are working from home to supplement their income, they often test your product outside of normal work hours.

If there is a problem in your product that prevents them from continuing, it’s unlikely that they will be available to resume testing exactly when you would want them to.

So keep your eye on your mobile phone and be prepared to answer questions quickly while testers are engaged.

Pay attention to testers’ actions

One of the most appealing aspects of crowd testing is that it provides insights on the functionality and usability of your product.

When you request the services of a crowd of testers, the results that you get reflect an audience’s perception of your website or mobile app.

Crowd testers might act in ways that you hadn’t accounted for. They may wander off into directions you wouldn’t have wanted them to.

When this happens, it is an opportunity to wonder why. “Why didn’t they do what we expected them to do?”

You should pay close attention to the behavior of crowd testers on your site; it most likely reflects the future behavior of your target users.

Choose the right crowd testing provider

Before you select a crowd testing provider, make sure you ask the following questions:

Are the results peer-reviewed or reviewed by a project manager?

Peer-reviewed crowd testing sites are sometimes less expensive than managed crowd testing services, but the difference in the quality of the results is significant.

Who will have access to my project’s information?

Some crowd testing sites publish open calls on their websites and social media pages for anyone to see.

Given that your product’s test environments may contain sensitive information, you want to be ensured that it won’t be available for the world to see.

On what platforms will my product be tested?

To ensure the best quality, your products need to be tested on a wide array of different devices, operating systems and browsers.

Your crowd testing provider should be able to tell you upfront on what platforms your product will be tested.


Category: Crowdsourcing

About Simon Papineau

Simon is the founder of Crowdsourced Testing. After 10 years in interactive software development, he set his sights on building a world-class crowdsourcing platform to facilitate the software testing process for developers.

  • Jack Richard

    Crowd sourced testing is an emerging trend in software testing which exploits the benefits, effectiveness and efficiency of crowd sourcing and the cloud platform. The objective is to get the work done from an open-ended crowd in the form of an open call. Crowd source testing allows more individuals to participate, often at a reduced cost and with better testing quality.

    Thanks for the article.
    Cheers Jack!