Testing strategies for mobile and web

Ahead of their industry roundtable at The TEST Focus Groups, Eran Kinsbruner, Director, Product Marketing, Perfecto, discusses the importance of testing for real user conditions.

If your company is taking steps to be a more digitally engaged business, congratulations – you’re on the path to innovation and success. But when trying to deliver high quality digital products across mobile, web (mobile web, responsive web, desktop web), and IoT on a variety of operating systems and networks, it’s important to remember to stay focused on the all-important user experience.

Everything is driven by a winning UX: Business goals, customer satisfaction, positive app store reviews, word-of-mouth uptick and, of course, revenue. But in complex digital environments, testing just basic mobile app features under ideal conditions will fall short of delivering great UX.

For instance, above is a recent search for Starbucks stores in Miami using both web search and the Starbucks native mobile app (on the right). Clearly, the mobile app testing team did not test against real conditions in this case.

In today’s world, it’s essential to test for the real user conditions that your customers experience every day. Here are three key tips to keep in mind when making user conditions part of your testing.

Define the right users for your products

Step one is to identify your target users (aka ‘personas’). While this sounds easy, accurate personas can only be created through market research and data analytics; defining personas relies on consistent communication between engineering and business teams.
The next step is integrating these users and their traits into the test lab. This is usually the toughest task for engineering teams because mimicking real environment conditions is hard and requires ongoing lab maintenance.
Some of the core requirements for sustaining a lab that tests for user conditions:

  • Ability to insert into the lab various network conditions, apps running in the background, phone call interruptions, and more.
  • Support for different geographies to cover testing of relevant users in relevant countries.
  • Support for any device and any operating system to sustain lab coverage as market matures.

Add user conditions to existing functional tests

Once the lab is ready, DevTesters can add the relevant user conditions to existing functional tests in various ways, including manual testing, data-driven testing or through automated test code. We believe the most efficient path is through automation.
Implement user condition testing on top of functional test automation code by adding test code lines that define the target. The test code will automatically inherit user traits such as their name, the platforms and devices they use, their background apps, their location, their network conditions, and more.
Integrating these user traits into an existing continuous integration (CI) workflow and running many scenarios simultaneously will pay off in shorter release cycles, increased test coverage and, ultimately, better UX.

Use reports to find and fix issues

Now that your team has defined the target personas and implemented the test code correctly, they’ll now need a detailed and actionable rich media report (see below). This report should include network logs, device and app memory and battery consumption details, and timers showing how long it took certain actions to take place.

Example of a user condition report (Perfecto proprietary content).
Example of a user condition report (Perfecto proprietary content).

Such reports should include results for every test scenario and should allow teams to take action and fix the issues. It’s also worth noting that to consistently address bug fixes, the report should compare the specific user condition tests to a “happy path” to show the contrast between real world conditions and ideal conditions.

Discussing mobile testing at The TEST Focus Groups

Perfecto are session sponsors at the upcoming TEST Focus Groups. To learn more about testing strategies for mobile applications, network and share your mobile testing peaks and pitfalls with other enterprise companies, be sure to join Perfecto’s roundtable.

Edited for web by Cecilia Rehn.