Michael Fritzius, Arch DevOps, LLC, reveals how to avoid test automation pitfalls and traps.

Ever since software became a thing, there’s been a need for testing, and then test automation. It didn’t take long to realise that testing software manually is not scalable, and so we’ve been using the same hardware to assist us in writing test automation as we do for writing the actual software.

Today, there are multitudes of tools available, each setting out to solve a subset of the problems that we encounter in both spheres of development and test automation. With such a wide selection of tools in the automation space, there are some traps I want to talk about, regarding choosing the correct tool.

And I enjoy using word pictures to illustrate points.

Choosing your tools well

My wife and I recently contracted our Pastor to build a custom chair for us. He’s very experienced in carpentry and does it on the side as a hobby. He’s made some impressive pieces over the years.

The arrangement was: We’d give him a concept, pay him for time and materials, and he’d take care of the design and construction.

There were a few trips he and I made to the hardwood store. He would spend quite a bit of time deciding what kind of wood to get, and distinguishing between certain pieces.

When I saw him overlook a plank for a seemingly identical one, he said, “The cuts I’d have to make in that piece would go right through a knot, and that’s not good structurally.” He had the chair designed in his head so well that he was making mental cuts in the wood, and that foresight led him to choose pieces of wood that wouldn’t be a problem later.

The end result was a beautiful piece which will probably outlive all of us.

Do I know how to use all the tools he used? Yep. Could I have gotten the same result? Not a chance! I don’t have the full understanding of how the chair goes together. I don’t know how to construct the best joints for what type of wood. I don’t understand carpentry. The fundamental knowledge is lacking, so my attempt would’ve been catastrophic.

My wife and I also like making things from scratch. Things most people wouldn’t think twice about buying, we make. We’re funny that way.

Baby wipes are a great example. We make our own at a fraction of the cost of store‑bought ones. We take a roll of paper towels, and my job is to cut the roll exactly in half, with a straight edged kitchen knife.

A couple years ago, I’d bought a power miter saw for some house projects. It wasn’t long before I realised that, hey, this would cut rolls much faster.

It does cut quickly. It also shoots paper towel fuzz everywhere, and the cut itself, although straight, isn’t very clean.

The problem I’d set out to solve was to decrease the time to cut these rolls. Some of the time was spent sharpening the knife – paper towels tend to dull blades quickly – but much of the time was spent with actual cutting.

A better tool ended up being a straight edge knife, but not made of steel since that dulls too quickly. A ceramic knife was a much better choice.

And, it does work well. Although it’s slower, the cut is much cleaner, and as a bonus, our baby doesn’t get fuzz all over her backside.

Two lessons come from these two word pictures, respectively:

• Knowing when and why to use a tool is more important than knowing how.
• A tool that can do the job quickly, does not mean it does the job well.

The test automation craft

There are traps that both you, and a company, can fall into when choosing a test automation tool.

We of course want to remove the pain that comes from having to test manually. But a danger is that we can latch onto tools that make us too reliant on them, and require us to use them as a solution, simply because it’s too costly or tedious to retool when needed. Or, we can adopt tools that have a large learning curve, which leads us to keep using them when they’re not the best solution anymore, due to how much time and effort we’ve committed.

And I think the reason we reach for the kind of tools we do – the ones that advertise quick pain relief – is because many of us don’t know, or have forgotten, that there’s a craft to test automation.

There’s a disparity between knowing how to use an automation tool, and understanding the craft of test automation. What we end up doing is exchanging knowledge of the craft for an increase in speed.

This ends up hurting us as testers and automators, because we’re so insulated  from knowing how the problem is being solved. With our tools, we often don’t
know how something works, we just know that it works. Until it doesn’t. Then we’re stuck either coming up with clunky workarounds, or avoiding the automation of certain tests.

This is why I’m a huge advocate of open source tooling, particularly ones that we have to write ourselves.

As we build them to solve our company’s problems, we learn about the craft, and have better understanding of what’s going on, and why the tools work the way they do.

Benefits

We can also visualise solutions better, since we don’t have to think of the automation pieces as ‘black boxes’. We know what’s in the box, and have a pretty good idea how they fit in to accomplish the goal.

There are other great benefits to this approach:

Talent retention

If you send the message to your employees that you’ll spend time and money on marketable skills for them, you will foster loyalty. Coupled with their domain knowledge, you’ll have an incredibly powerful team with the strategy to automate just what’s needed, for the most pressing problems. No tool on the market can provide that for you.

Talent attraction

Many people know that test automation skills are highly sought after. If you offer a chance to those who are driven to learn, to come to your company to do so, you will be tapping a market which is unavailable to those companies who only have a tool to offer.

Change immunity

Hey, things happen. Jobs get lost, companies get bought, decisions get made. Investing time and brain space on certain tools limits your next job to those tools. Investing in learning the craft teaches you about the ‘what’ behind the ‘how’. If you master the craft, you have mastered every tool. Go forth and conquer.

Independent solutions

What I said above, about a tool working, until it wasn’t, is a very common thing. If you have control of the tool’s source code, you’re in a powerful position. The whole tool is probably not broken, and the broken part can either be fixed or swapped out. Instead of scrapping and re‑platforming your automation, you can repair and keep going. Your competitors who rely on costly third party tools can’t.

Innovation increase

Test automation is by no means a solved problem. There are innovations just waiting to be made by smart people who share ideas from their variegated backgrounds, and apply them to your company’s unique problems. Sometimes it can put your company on the map, simply by how cleverly the work is being done. That probably won’t happen if they’re constrained to using solutions someone else thinks will solve their problems.

So I encourage you – employer or employee – take the harder path. Don’t just learn about a bunch of tools, learn about the craft.

Go build some stuff! Try writing something to test a webpage. Send some data to a webservice and parse what comes back. Make a database query. See if you can interact with a Windows application.

Then ask: How does my code look? Is it maintainable? Readable? Stringy? Heavy? Adapt it, share with others, solve problems with it, and constantly improve – both it, and yourself.

I’ve already started the journey. Will you?

This article first appeared in the November 2016 issue of TEST Magazine. Edited for web by Jordan Platt.