The results from the first European Software Testing Benchmark Report 2016 survey on test automation are in, highlighting that automation is not yet as commonplace as firms want; there are still numerous challenges hindering some; and sourcing test automation specialists needs to start in‑house.

TEST Magazine, in partnership with its online news portal, SoftwareTestingNews, carried out the survey research and has now published the findings.

Key findings from the automation report include:

Only 5% of survey respondents said they currently carry out 0:100 manual:automation testing. The majority (66%) are either at 75:25 or 50:50, and 9% said they are only doing manual testing.

When asked what they’d like to achieve in the next five years, the majority (73%) said they’d like to see 50:50 to 25:75 manual:automation testing. 14% said they’d like to have no manual testing at all.

The majority of respondents reported seeing a return on their test automation investment immediately (24%) or within the first 6 months (24%). The remainder saw a result within 6‑12 months (28%) or after one year (15%). Only 9% reported never getting an ROI.

The majority of respondents (94%) said they use test execution tools and automation to support testing efforts. Other popular answers include generation of test data (57%) and deployment of environments (49%).

The survey respondents reported that the primary users of their testing tools are test automation specialists (55%) and testers (27%). Developers make up just 6%. Other responses included offshore contractors software engineers, and product owners.

When asked what automation currently exists in organisations, there were a wide variety of responses. UI (82%) is the most popular; APIs (62%), cross browser/platform (55%), test data/environment (55%), performance (53%) and integration (47%) are all very even. Interestingly, automation in the ecosystem scored the lowest at just 6%.

The biggest challenges facing automated testing in the future

The survey also asked what the biggest challenges facing automated testing in the future are. Common responses included maintenance overhead and continuity of ownership; automation in sprint; deciding what to automate; skilled resources; tool selection; budget/funding; updating automation scripts; mind set; buy‑in from stakeholders; legacy projects; and lack of experience.

The sheer number of different responses underscores the complexity of implementing test automation and how solutions are never ‘one size fit all’ due to differentiating sets of circumstances and resources at hand.

Sourcing automation testers

The survey showed that test automation is overwhelmingly carried out in‑house, primarily by in‑house resources (62%) but also by contractors working in‑house (12%). A few (9%) responded ‘other’, and specified a combination of in‑house and outsourced efforts.

A majority (51%) said that they prefer to retrain existing staff to meet automation testing resource requirements. Another 38% said they prefer to hire skilled permanent staff. Only 8% said they’d hire skilled contract staff and even fewer said they would engage consultancy services (3%).

When asked about what attributes/characteristics are sought after in team members who are to be retrained in test automation, it was clear from the volume of responses that a combination of technical (often coding) skills and soft (primarily communication and motivation) skills are attractive.

Conclusion

For more in depth insight and to see all the survey results, download the full report.

TEST Magazine will be running more surveys throughout 2016, if you are interested in participating please visit: www.softwaretestingnews.co.uk/survey/

Written by Cecilia Rehn.