The Test and QA Wales group regularly meet to discuss pressing software testing and QA subjects. Recent discussions around certification and automation have brought some interesting insights to the fore.

Tester certification

The subject of tester certification has been the subject of significant debate. Certification can broadly be split into two subsets, the course and the certification process itself. These are different in that a course can be taken without certification, there are many good courses that do not provide a certification element at the end of them.

The two general certification types that seem to be prevalent in the Welsh Testing and QA community appear to be the International Software Testing Qualifications Board (ISTQB) and the Certified Agile Testing (CAT) routes. The ISTQB testing syllabus is relatively long established compared to the CAT syllabus. The ISTQB have added an Agile Tester qualification which can be conducted after the foundation level certification.

Value of tester certification

The general opinion of the various courses was that they provide a good grounding and starting point for new testers. They provide a common language within an organisation and set a basic level of understanding.

There was much debate over courses that simply taught ‘book knowledge’ as opposed to the broader practical exercises that appeared to be more prevalent in the CAT course. It was felt that the ISTQB course was really taught to enable delegates to pass the certification exam as opposed to providing practical knowledge that could be taken back into the work place.

It was felt that courses taught by individuals who were actually performing the role tend to be of more value than those provided by trainers who lack current real world experience. The capability of the trainer to relate to the working situation of professionals, coupled with the ability to draw on recent experiences means that they are more likely to engage with delegates.

Certification is widely seen as a CV building activity. Whilst a number of years ago becoming a certified tester may have been a good guide to managers when recruiting, this is no longer the case with nearly all managers within the Test and QA Wales group agreeing that certification on CVs is largely ignored on the basis that it cannot be a reliable indicator of capability. There is concern within the Welsh Testing and QA community that being certified may be seen as a requirement for those coming into the industry.

Despite the contrary view among test managers, many larger organisations feel that there is value in certification, as do smaller companies that are in the process of establishing testing and QA functions.

Broader certification and stricter conditions

A broader certification of testers would hold greater value than the existing certifications for software testers. This would include, soft skills, practical elements, theory and not be a multiple choice based. Certification would have more value if there were re-assessments of individuals at regular intervals to ensure that the knowledge gained was being used in practice and that any claims of malpractice can lead to de-certification.

Whilst certification of testers is not entirely without value, it is clear that there are limitations to the advantages that are offered by the certifying boards. Of the two certification schemes discussed, the Test and QA Wales group felt that the CAT course and assessment process offered the most value to individuals and organisations.

The benefits of automated testing over manual software testing techniques

With teams moving towards faster deployment to decrease time to market and increase return on investment, testing and QA functions have increasingly turned to test automation as part of their solution. The promises of automation are manifold including reduced execution time, reduced testing cost, increased coverage; and better test accuracy to name a few, but is it really a silver bullet?

Many teams have an assumption that it is necessary to automate their test cases. However, there is often a lack of will in the development team or skills within the existing QA and testing team to drive changes through in a meaningful way.

The automation process that is employed will only be as good as the team performing the automation.

Complementary efforts

Manual testing is a thoughtful process and allows a tester to dive deeper into the application than automation typically does. It enables critical review of user experience delivery. Often in manual testing a tester will notice something and investigate the issue, for instance, image colour changes or UI component moves. Such findings are lost with automation as this will only perform the checks that it has been specifically told to do.

Automation and manual testing ideally complement each other, both adding equal value to the understanding of quality. The amount of effort that will be spent producing automated tests and maintaining them can make it difficult to realise the benefits if a robust framework is not created. This coupled with the additional humanistic testing benefits that manual testing can bring mean that finding the right balance between the two is an essential element of project success for many within the Test and QA Wales group.

Many teams find that manual testing can get stale. This is particularly true of teams using formalised manual test scripts. These test scripts are regularly identified as candidates for automation, which then frees teams to perform exploratory testing which is more pro-active and engaging.

End user input

Manual testing with a group of end users is also considered to be useful. It generates a fresh pair of eyes, which may drive both new requirement and defect identification. Here the value is in interacting with the user although it is often at the end of the software lifecycle. Teams across the Test and QA Wales group have found that it is better to draw on the user expertise earlier, including at the story writing stage.

Test automation is seen as a fast way to perform a significant number of validations against new builds of software. There is little doubt that if and when done well it greatly enhances the capability of software delivery. It does however continue to pose challenges to organisations as they adopt the new processes and procedures to put in place such a solution.

Whilst the Test and QA Wales group see huge value in automation, they very much see it as complementary to the thought processes involved in manual software testing.

The Test and QA Wales group is sponsored by Acorn Recruitment and Cloud Services by Conduit Global.

Edited for web by Cecilia Rehn.