According to Gartner, scriptless testing tools aim to reduce the amount of scripting involved in the creation of tests using a model or keyword-driven approach. The goal is then to allow business user testing and to reduce maintenance costs.
We have then recently talked to Eran Kinsbruner, Chief Evangelist, Perfecto by Perforce, to find out more about scriptless testing, including some of its challenges and how these can be addressed.
There is a difference between modern scriptless testing and legacy scriptless testing. In the past, we used to have tools that could have created test automation. These tools, however, were very high maintenance and static.
Says Eran Kinsbruner, “The modern scriptless test automation tools are now driven by artificial intelligence (AI) and machine learning (ML) and thus, allow you to not just create test automation without writing any line of code but also to benefit from self-healing and ease of maintenance. Hence, with these tools, you can create more reliable test automation as well as gain more trust in your test cases.”
These modern scriptless tools can give manual business testers the trust they need from the other partitioners, developers, and test automation engineers.
The need for scriptless testing
According to Eran, scriptless testing has been needed for many years now.
“Test automation can be too long, too unreliable, too hard to maintain, and not always integrated into the companies’ process. Organisations need modern and intelligent test automation to help solve these problems.
Testers have now the opportunity to automate the manual testing without writing code and use tools that are very easily integrated into the CICD (continuous integration continuous delivery) toolchain. These tools can also be scheduled to run upon every and each trigger in the pipeline.
Scriptless testing helps to shift business testers into the automation cycle and the CI (in many cases they are not part of the process currently). This enables faster feedback from business testers who typically have great business flows understanding.”
Moreover, Eran continues, it automates regression, exploratory, and manual high-value tests. Indeed, people usually run these tests manually once, but from the 2nd and 3rd executions, these are regression/repetitive flows, so they are worth automating to reduce workload. It also reduces test flakiness by generating more reliable tests using AI/ML and using self-healing tools.
Finally, Eran points out that scriptless testing allows faster time to create new tests compared to coding new tests.
The benefits
The first benefit of scriptless testing, Eran tells me, is that it helps to reduce to the minimum the amount of money and manual testing done by the business testers. The business testers are spending way too much time running these scenarios. Reducing the number of manual test cases boosts the velocity and productivity of the entire team.
The second benefit is that it complements test automation coverage driven by developers and test engineers.
“If you complement code-based testing with scriptless automation, you could actually get to around 90% of test automation coverage upon each and every regression cycle. It’s time-saving, cost-saving, and above all, more efficient.”
The challenges
However, Eran highlights that scriptless testing introduces its own challenges…
“As a new and growing technology, it cannot automate every scenario that can be automated using code. The solution is to combine additional methodologies to complete full high coverage suites. The software development test engineers can manage more complex and code-based scenarios, while business testers automate the process using scriptless to cover other use cases.
Scriptless testing is more friendly to business testers rather than developers who usually prefer IDEs/codes for creating test automation. However, it still requires the use of multiple testing technologies throughout the organization. There is not a one size fits all testing solution from both persona, skillsets, and objective perspectives.”
Eran emphasises that the challenge is to make scriptless fit and complement the ongoing practices in the organization.
“You don’t need to throw away all the other scripts. On the contrary, you want to complement them in a way that doesn’t cause any trouble or lose the productivity of the different team members. It relates to the maturity of the tools, the maturity of the skills of the team, and it will take time.”
Moreover, he continues, they are very sophisticated scenarios that scriptless simply cannot cover. These will always be done by coding. It is then about making the right balance between what fits scriptless and what remains code-related. Once you decide on something, you need to maintain it.
“Getting the balance right is challenging but it will benefit organisations in the long run.”
Start small!
Eran tells me when people ask him how to overcome challenges, regardless of the challenge, he always says: ‘start small and then grow’.
He underlines that you cannot control large volumes if you’re not mature enough. His recommendation is to start small, small always wins! For instance, take the ten most time-consuming test cases that are going to be repetitive, and start automating them and see how it works for you. Then merge them into Jenkins and into your CICD toolchain and see if they work and if they bring back value. Then do the next ten.
You need to build the foundation of reusability. Everything you do with coding; you can do with scriptless. There is a test management layer in scriptless that can allow you to gather and manage as well as maintain all the test scenarios.
“Start small and build the foundation right so you can reuse it and easily maintain it.”
Eran also advises making sure that you are communicating and making it a win-win situation for the entire team. When you talk about scriptless, developers think that it’s going to steal their jobs but it’s not the case. It’s all about balance. You need to explain the strategy behind these tools to allow this new technology to be more of a success than a challenge or a threat.
The right culture
Based on market trends, organisations are soon going to have no choice but to implement scriptless testing.
Eran states that the entire industry is experiencing test automation that is not reliable enough and has no sufficient skillset. They need tests to run upon each code changes out of the box with zero maintenance. “If it breaks, you break the entire pipeline.”
He believes that scriptless is needed and can complement some of the pains that developers and test automation engineers have today. So, they can actually spend more time stabilizing their test automation scenarios, leaving time for the manual testers to be more comfortable with the tools and tests.
The future of scriptless testing
Like any mature technology, especially when it’s built on top of AI and ML, we want to be aiming towards autonomous testing. Right now, what we have is a good solution but it’s a control solution as you need the humans to point the tools in the right direction to automate.
For Eran, scriptless for a web or mobile application will create a very reliable test automation suite that can run again in parallel on different platforms, without stealing the jobs of any practitioners! They need to be behind the tools and moderate the testing.
“This is just the beginning of scriptless and we’re going to see a gradual stepping forward towards a more autonomous way of creating test automation. It will take a few more years to get something that really works but this is a good start!”
