The testing industry has been significantly shaken up by the rise of crowdsourced testing. It can be tremendously useful for some developers, but it’s not going to be suitable for every project. How do you weigh up the pros and cons of having a dedicated in-house QA department, versus engaging crowdsourced testers? That’s what we’re going to take a look at today.
Pros of in-house QA
You can hand-pick a skilled team that meets your needs when you have an in-house QA department, but that creates an HR overhead, and it can be difficult to take future projects into account when seeking skills today. Like most of the pros on our list, there’s usually another side that can be interpreted as a con.
- Direct accountability – Your in-house team reports directly to you. You can guide them, teach them, and reprimand them when they get something wrong.
- System knowledge – Working internally, it’s likely your QA department will amass specialist knowledge about your systems that may enable them to do a better job.
- Relationships – Having established relationships makes communication easier, and generally serves as a shortcut to getting what you need from your testers.
- Proximity – If a developer can’t reproduce a bug, they can call the tester to their desk and have them show it. Face-to-face communication is usually more effective and can be a real time-saver.
Crowdsourced testing offers a seemingly irresistible menu of potential advantages, but it’s not a panacea. To get maximum value, you really need the right circumstances.
- Lower costs – You aren’t housing testers, you aren’t paying for their upkeep, and you aren’t paying them during downtime. In short, crowdsourced testing should work out cheaper, if you were to assign a cost to each defect discovered.
- Scalability – You don’t need any testing for three months and then you want 200 testers working on your project for a month. Only crowdsourced testing can give you that kind of scalability.
- Round-the-clock – With global coverage, crowdsourced testing can provide true 24/7 testing when you need it. The alternative is outsourcing and a lot of overtime.
- Diversity – You can find testers from different localities, different demographic groups, and with different set-ups in terms of their devices and environment. You could argue that crowdsourced testing is closer to testing in the wild, and provides a better emulation of end users.
Double-edged swords and other considerations
There are a lot of other considerations to weigh up and many of them are far from clear cut. You could argue that crowdsourced testers care less about your product. On the one hand that might lead to expectations that they won’t do such a thorough job, but with no vested interest they’re also more likely to give an honest opinion.
For some projects crowdsourcing will be considered a leak risk. If you have sensitive data to protect, or valuable IP, you may not feel comfortable going out of house.
Your own QA department will probably have set standards and a common technique. Crowdsourced testing is going to result in more variability in the quality of defect reports, but it could also lead to a more rounded picture of your software health because you have a wider set of different points of view.
There’s no right answer
We can’t unequivocally say that crowdsourced testing is better than having an in-house QA department, or vice versa. For small companies it makes a lot of sense, because a dedicated team will be prohibitively expensive, but larger organizations need to assess carefully. You really need to weigh up the requirements of your particular project.
It’s worth bearing in mind that you don’t necessarily need to choose. The best scenario will often be a small in-house team that can oversee crowdsourced testing resources when they’re needed. That way you can reap the benefits, and mitigate the risks.