The need for Blockchain Testing

While hackers are becoming more and more sophisticated in their attacks, it is essential to be able to secure, store, and manage digital identity and data in a safe platform. Organisations all over the world are then turning towards Blockchain technology.

Blockchain collects data into blocks, and chains the data together using cryptography, improving the overall data security. According to a recent World Economic Forum report, it is expected that 10% of Global Gross Domestic Product (GDP) will be stored on Blockchains or Blockchain-based technology by 2025.

Hence, there is a growing need for businesses to start adopting blockchain testing and technology. To tackle this topic, we have asked experts in the industry to share their insights.


What is Blockchain Testing?

Blockchain Testing is different from regular software testing, as it consists of many components like wallets, transactions, blocks, consensus, mining, gossip, etc. which require special focus and specialized tools to test.

As Raj Shekhar Bhardwaj, Blockchain Research DevOps at Clearmatics points out, each action(write) on Blockchain is a transaction and to test Blockchains basically involves testing the transaction lifecycle from its creation, validation, block creation, mining, and finally commit/save in state immutably.

After this, we can move further with performance and security testing.

Santu Maity, Enterprise architect at IBM also underlines that Blockchain is a digital ledger where records are stacked together in small blocks forming a continuous chain, which cannot be updated or deleted. It is decentralized, and that means each transaction that is added onto the chain has to be replicated so that each node has the same copy.

Hence, all these unique features make blockchain applications complicated so proper end-to-end testing of each feature must be done. Some of the core tests that should be run include functional testing, non-functional testing, performance testing, security testing, and integration testing. Other than these core testing, we also need to perform specialized testing for blockchains such as Smart Contract testing and Peer/Node Testing.

Rutvik Mrug, Associate Director at Cognizant, adds that blockchain is a new technology, hence many organization are apprehensive to adopt it. Yet, Blockchain testing helps to reduce these apprehensions by ensuring the highest quality of end product created using the Blockchain framework. Thus, Blockchain testing helps to build quality various stages including functional applicability, Blockchain principle adoption, as well as the performance of the system to the security of Blockchain network.

Blockchain testing is more about Blockchain-based application testing rather than Blockchain platform (like Ethereum, Hyperledger, etc..) based testing which is a proven platform.


The best practices

According to Santu, the best practice of Blockchain testing involves testing the entire environment. This includes reviewing web and mobile applications that interact with the blockchain technology, APIs, access controls, ingress and egress points in the blockchain, public key infrastructure (PKI), user certificates, configuration, smart contract and blockchain nodes, and network.

Raj also states that the blockchains are mainly classified into two categories:

  1. Public Blockchains
  2. Private/Enterprise Blockchains.

Public Blockchains are easy to set up (but hard to manage) as the whole network is already created and we can just plug in our node and deploy the smart contract i.e., our use case. It runs on an individual economic incentive strategy driven by its cryptocurrency.

While, on the other hand, private blockchains are permissioned. Hence, we first need to create the network with all the components like peer nodes, miners, certificate authorities, smart contracts, etc. It does not require any cryptocurrency as fuel to secure it as all parties are known to each other.

The Blockchain Testing approach should be focussed on one of these types of Blockchain. If it is a public blockchain, then the focus should be more on network security, smart contract vulnerabilities, private key management, wallet security.

In the case of Private permissioned Blockchains, you should focus on the component level tests, smart contract, identity management, and infrastructure security.

Furthermore, Rutvik points out that Blockchain testing should be aligned to the technology layer of Blockchain-based application. UI/end user-based testing should be minimal (E2E flow) while middleware (API) should be the majority of your testing. Blockchain layer should have only testing related to network and Blockchain basic principle (block creation, resiliency, etc…). Hence, blockchain is mostly run through Agile mode, which mandates as much automation as possible.


Types & tools

Raj explains that we can divide Blockchain into two types:

  1. Functional Blockchain Testing.
    1. Smart Contract Testing.
    2. Transaction lifecycle
    3. Peer gossip and block sync.
    4. Finality
  2. Non-Function Blockchain Testing.
    1. Performance Testing – Load, transaction latency, system performance
    2. Security Testing – Identity theft, Network penetration, and integrity.
    3. Recovery Testing – Failure recovery, Crash tolerance, Fault tolerance.

Currently, there are not many specialized tools available for Blockchain testing. However, there are some projects working in this direction such as Hyperledger Caliper.

Testing a Blockchain application ensures its quality and maximizes the test coverage and minimizes the risks that are associated, Santu adds.

There are standard tools for application and APIs functional and non-functional testing. The following are a few tools that help in testing blockchain and guarantee that they are functioning properly:

  • Ethereum Tester: Ethereum Tester is reliable for Ethereum testing include Web3 Integration, API, Smart Contracts. This helps both developers and testers to simulate a production-like blockchain based on Ethereum.
  • Ganache (Testrpc): This tool is mainly used to test Ethereum contracts locally. It creates a simulation of a blockchain that allows anyone to use multiple accounts for testing.
  • Hyperledger Fabric testing tools: Operator tool and PTE: The Operator tool is used to deploy Fabric networks. It can be used to deploy Fabric to your local machine using Docker, or to Kubernetes. PTE or Performance Traffic Engine is used to invoke and query chaincode through a network deployed using the Operator tool.
  • Exonum Testkit: Testing the activity of the entire service of the blockchain application is the specialty of Exonum Testkit. The tool allows us to do API testing and transaction execution without the need for the network operation and consensus algorithm.
  • Corda Testing Tools: Corda is a blockchain-based and open-sources distributed ledger platform. It has a built-in testing tool to help with: writing contract tests, integration testing, writing flow tests, load testing.

Blockchain testing is aligned to technology layers of blockchain-based application, Rutivk notes. On top, it may have UI / IoT layer, with user interface-related testing needed. The majority of testing is done at the API layer as every Blockchain functionality is exposed at the API level. Blockchain applications are mostly data-driven applications (due to smart contract and transaction storage) and hence, automated testing at the API layer will play a big role.

At the Blockchain layer, testing will be around Blockchain concepts like block creation, ledger synchronization, node resiliency, etc… For UI and API, a traditional toolset (including rest assured, Selenium, etc…) will be adequate. For the Blockchain layer level, it is recommended to go for the framework (Ethereum, Hyperledger) related SDK or framework aligned to it (e.g., Ethereum tester/truffle for Ethereum).

The benefits…

Testing a blockchain is vital to ensure the defects in a decentralized ledger are eliminated, Santu tells me. Indeed, this ultimately protects a business against the negative consequences of blockchain poor operation. Implementing blockchain testing help to validate all the different entities that the blockchain system involves for its operation and ensuring that the developed blockchain ecosystem is functioning as expected.

Blockchain Testing helps organizations build and use blockchain technology and the connected infrastructure securely.

For Rutvik, blockchain testing will help reduce the risk of adding new applications well as help revalidate the implementation smoothly. According to Raj, Blockchain testing ensures high-quality product delivery and enhanced customer experience. In the case of public Blockchains, it is vital as real money is involved and a single bug in smart contracts could cost huge monetary losses.

…And the challenges

The biggest challenge of Blockchain Testing is the lack of Blockchain expertise among developers and a smaller number of specialized tools available, Raj underlines. More time should be dedicated to training developers in blockchain testing and expertise so as to achieve the best results possible.

He points out that Blockchain testing can also be an expensive and time taking task.

Rutvik emphasises that Blockchain testing is still evolving and we have barely any sophisticated tool/strategy. This means every team ends up inventing a new tool/framework for them. Moreover, there are very so few adoptions of blockchain so we hardly get a full-fledged testing opportunity. Another challenge is the variety of frameworks, as most of the time, one testing solution won’t work on all the frameworks (Ethereun, Hyperledger, etc…).

Santu points out the common challenges seen in testing Blockchain applications such as:

  • Lack of standardization in Blockchain testing: Lack of technical, functional, legal expertise is also crucial for Blockchain testing adoption. Developers without a strong ability to conceptualize, standardize, and abstract complex concepts in Blockchain lead to a number of issues in Blockchain testing.
  • Lack of Blockchain Testing Tools: If you don’t have the right set of tools, you are likely to fail. Testing Blockchain applications requires a mixed toolset that comes with blockchain end-to-end testing. There are very few available for each blockchain framework.
  • Performance and load: Failing to test for performance and load testing gives little or no insight into how the Blockchain application performs in both production as well as under specific workloads and network conditions.


Should businesses adopt blockchain testing?

Blockchains are now entering our day-to-day lives, Raj emphasizes. Indeed, they are everywhere – from supply chains to Trade Finance, from Defi to NFTfi. With this increase in adoption, the need to ship high-quality products cannot be met without investing in creating Blockchain and Blockchain testing expertise.

Hence, Raj thinks this is the right moment for organisations to adopt and develop blockchain expertise and blockchain testing strategy. The demand is huge, and the availability of the right solutions is scarce.

Any new technology, specifically the ones that change software drastically, will have challenges to prove its credential/quality, Rutvik adds. Blockchain testing will then help to boost that confidence in the quality. Blockchain-based applications (also called DAPP – Decentralized Application) will allow businesses to have a completely new way of storing and processing data.

Moreover, Santu underlines that one of the main features of Blockchain is trust. Hence, to guarantee trust, we must ensure all the components of a blockchain are working perfectly and all applications are interacting with it in a trusted manner. The key to a successful implementation of Blockchain technology is based on a well-balanced approach, plan to implement the design, and proper testing of the entire Blockchain network and connected application. This requires a full test strategy plan.

Therefore, organizations should consider the full test strategy plan involving use cases map, test execution, and result verification as well as the methodology to conduct standard and specialized testing of Blockchain applications.


The future of blockchain testing

In 2017, the blockchain market was valued at $708 million, Santu states. Two separate reports have estimated that by 2024–2025, the market could be valued between $20 to $60 billion. Hence, this significant growth represents up to an 8,300 percent increase in the span of fewer than 10 years. Santu then believes that we are still in the early stages of exploring this technology, and it will take time to fully realize its applications and potential.

He also thinks that there is immense scope for blockchain testing that requires extensive expertise in testing blockchain platforms. There will be bigger connected blockchain networks, where attackers could exploit identified vulnerabilities. This requires the entire testing of connected blockchain networks.

Raj believes that Blockchain testing, as a separate segment of the QA stream, will be in high demand and will be a must-have offering for the Testing/QA companies. He thinks that it will be a very interesting space to watch in the coming few years.

Rutvik underlines that Blockchain adoption is still at an early stage, with many of the organizations still doing pilot. Yet, there are many industry research forecasting higher and more mature Blockchain adoption. Blockchain testing will thus be on the rise. With no standard tool available currently for Blockchain testing, there is potential to have more standardize tooling and solution around Blockchain testing.


Special thanks to Raj Shekhar Bhardwaj, Santu Maity, and Rutvik Mrug for their insights on the topic!