Andy Robson, Owner, Testology Ltd, addresses the changing QA challenges facing those working in the gaming sector.

The industry has seen a few notable QA challenges in the last few years. The first has been focused around platform diversification and growth within these platforms, with mobile being a significant contributor to these challenges. Work is now focused on mobile devices, consoles, PC/Steam platforms, browser‑based products on desktop and mobile, physical toys, hardware that collaborates with applications, VR, and others I’ve missed, I’m sure. And these platforms are no longer limited to singular industries; they’re shared across a multitude.

Platform diversification

So, for example, we can test browser‑based video games one day, and then slots or websites using the same browser considerations the next. Platforms are expanding in number, as well as the type of product they support. QA has to be considerate of all of these things.

Some 23 years ago, PC and a couple of consoles (Nintendo and the Master System) were all we had to work with, so the industry is diversifying through platform growth and sharing more synergy with other product types and industries. But this means that we have to be more aware of time, of resource distribution, and of focus when planning test phases for our clients, or suggesting how we can support and supplement their existing QA departments.

Increased size and complexity of games

Secondly, one of the toughest things that the industry has faced would be the ever increasing size of games that are now being released on console, PC and, at times, mobile too. Many products are incredibly open ended, expansive, and feature rich beasts, that are both online/offline and heavily reliant on complex and large‑scale multiple options. The testing coverage is significantly greater than the early years of games development and this does complicate the testing process considerably. And I personally think this has had huge ramifications on the industry and the amount of time during development dedicated to the QA process.

QA teams are simply not getting enough coverage on titles and games are being released before they’re ready to meet imposed deadlines. This really frustrates QA teams across the gaming industry, as scope is simply inaccurate during periods of planning and comprehension of how involved a proper QA period is. It’s longwinded and repetitive and time consuming. Huge games need huge amounts of testing.

The growth in mobile

And with mobile being the most significant platform to develop in recent years, the products have grown in scope in the some way. Applications are being consistently released with successful IPs releasing new features and updates and content bi‑weekly or monthly. Title development seems to be endless on this platform for many of our clients. So, with the sheer quantity of titles and developers, we are now testing 30‑35 projects a day, as opposed to three or four major titles a few years ago. Some require 1 day of testing every few weeks, some require hundreds of man‑days every month. There’s diversity in the requirements but success for a title means the IP needs to be sustained to satiate the user base. Our management structure, scheduling model, and client liaisons need to be spot on to make sure we manage our own QA departments while managing the dynamic expectations of our clients and their products.

A lot of QA departments I know have struggled with this adjustment. We’ve managed to create a flexible QA business that’s suited to changes in requirements, changes in industry, and fluctuations in workload. But it’s always exciting to see the industry being affected and influenced by technology and developments in platforms. It’s only natural that our business should be influenced by it too!

Mobile is not just about the products, applications, or games, either. It’s a convoluted and saturated market from a hardware and software perspective. We face the challenge of making sure we’re market representative and supportive of our clients’ compatibility needs. With iOS, we manage the limited hardware releases, software updates, and BETA releases relatively painlessly. It’s the Android market that can be the tricky one.

We currently have over 300+ physical devices and would never consider emulation as an option. The scale of procurement was a challenge, initially, but now it’s more of less making sure we’re expert consumers. Developers just don’t have the amount of physical devices we have, or the desire to purchase them, so companies like Testology are a time and cost‑effective solution for ensuring full compatibility of their products.

Where we’re headed

For me, I don’t really think technology like virtual reality (VR) will be hugely impactful on the gaming market. It’s too expensive and the player experiences leave a lot to be desired from what I’ve seen. But, mobile VR might the answer… Generally, I think platforms will continue to grow and influence the type of products we test. Mobile will get bigger as a platform and, perhaps, even become the gamers’ console – it’s an attractive proposition when most daily requirements are on someone’s mobile, anyway. Some smartphones are more powerful than a console and, with improving network speeds and better battery life, games of console quality will be neatly stored on your favourite mobile device. Portability won’t compromise quality or overall experience and AAA games will be ‘mobile.’

This article first appeared in the March 2016 issue of TEST Magazine. You can subscribe to the digital edition for FREE here.

Edited for web by Cecilia Rehn.