Appium TestNG Framework and Multi-Device Automation Execution

The rapid development mobile app development eco-space brings up the pressure to quickly launch the apps. The growing testing needs cannot be met without parallel testing.

Mobile automation becomes a strategic initiative for an enterprise. After the organisations decide to automate their tests, the next item on their priority list would be to determine a suitable test automation framework. An automation framework is a comprehensive set of guidelines used to produce beneficial results of the automated testing activity. It helps to reduce the maintenance efforts and so the organisations should opt for a framework that enables even a manual tester to execute automation scripts. Any organisation that starts with automation, generally follows a typical journey and at each stage of evolution of testing framework, engineers face numerous challenges. Let us understand the typical evolution of automation test framework and challenges that engineers have to face in each stage.

Stage-1: Driver Script + Tool = Results in Console

Description: This stage can be considered as the initial automation script or a tester’s ‘First Automation Script’. During this stage, the organisation initiates the automation and there is no properly aligned automation framework available.

Challenges: Since it is an initial automation script, engineers might have to face multiple challenges.

Stage-2: Driver Script + Data + Tool = Execution Report + Snapshots

Description: We try to overcome the shortcomings of Stage1 in Stage 2. Keeping rest of the framework same, we just create a boiler plate code to read data from the external file.

Challenges: Although, this stage overcomes a lot of shortcomings of the initial stage, still more requirements come up to make testing smoother and faster.

Stage-3: Driver Script + Data + Objects + Tool = HTML Report + Snaps + Logs

Description: The challenges and the requirement that arose during stage 2 are all covered in stage 3. The basic framework has ultimately got some promising shape.

Challenges: Now comes the need to run the test on different mobiles in parallel with no or minimal change in the automation code.

Stage-4: Multiple processes in parallel with the same automation code

Description: In stage 4, you can take the same automation script for multiple automation code and every process will run in parallel. So parallel automation is achieved to some extent with minimal or no changes in the script.

Challenges: With an advancement in stage, challenges that engineers face also advance in nature.  

Stage-5: Multiple ‘threads’ in parallel to the same automation code.

Description: So, finally we come to Stage 5 of the automation framework where multiple processes have been replaced by multiple threads. It actually resolves the challenges of Stage 2 and Stage 3. There will be a single process for parallel execution that invokes multi-threads i.e. one thread for each browser.

Challenges: !!Code not Thread Safe!! You might be wondering what this means. In this stage, the driver object & many other things like reports, reporting frameworks etc. are kept static which means, they can be accessed without instantiating a class and are like shared variables.

Now let’s try to understand this with an example: Create Firefox driver in one thread and create Chrome driver in another thread. The first thread keeps the Firefox in public static Webdriver driver; variable. The second thread which has the Chrome driver object keeps it in the same variable. As it is static, there is no instance of the driver object that needs to be created and it can be accessed from any other class. Therefore, it is not THREADSAFE. So, the long-term solution could be to remove all the ‘static’ keyword from the code.

Most of the organisations accept to live with the problems of Stage 4. But, unfortunately, this is not the right approach. Let’s try to work out a comprehensive framework through Appium and TestNG.

“TestNG is a testing framework which has powerful functionalities and is easy to use, inspired by JUnit and NUnit.” TestNG is similar to JUnit, especially when testing integrated classes. TestNG eliminates most of the problems of an older framework and gives the developer the ability to write more flexible and powerful tests with the help of easy annotations, grouping, sequencing & parameterizing.

TestNG Annotations

Let us get some idea of Annotations hierarchy or Annotations levels in TestNG.

  • @Test is the smallest annotation.
  • @Method will be executed before and after the execution of @Test.
  • The same way @Class will be executed before and after the execution of @Method and so on.
  • It is clearly visible that the @Suite annotation is the very first and the very lastly
  • Then @Test followed by @Class.
  • Now if you notice, the @Method has been executed twice. As @Test is a method in the class, hence @Method will always executed for each @Test 


Test Case Grouping

‘Groups‘ is one more annotation of TestNG which can be used in the execution of multiple tests.

We use groups in TestNG when,

  • We don’t want to define test methods separately in different classes (depending upon functionality) and at the same time, we want to ignore (not to execute) some other test cases as if they do not exist in the code.
  • So to carry out this we have to Group them.

Sequencing & Prioritizing



There will be situations when you want to put number of tests under a single test class and like to run all in a single shot.   You need to use the ‘priority‘ parameter, if you want the methods to be executed in a defined order. Parameters are keywords that modify the annotation’s function.


Data Parameterisation

Data Parameterisation is a very common term these days and most of the engineers know quite well about it. Still, let us know something more about it. As you know it allows to automatically run a test case multiple times with different input and validation values.

There is another application of this functionality where you can mention the data provider named Authentication and one more data provider named Browser and can return an instance of the new object(). The first browser is Chrome. Second, the browser is Firefox. The third browser is Internet Explorer, so the same tests will be invoked three times

The design and the structure of solution highlighted in this article is an example that demonstrates how TestNG framework with Appium can help engineers generate real value and accelerate their testing multi-folds. The solutions pitch to most of the organisation’s needs and addressed many of their issues associated with mobile automation. As the app continues to evolve, the significance of a stable and faster automation solution is growing tremendously.

About pCloudy

pCloudy (, the key product of Smart Software Testing Solutions is revolutionising the testing environment in mobile space. pCloudy offers multiple cloud-based mobile app testing solutions from small and medium-sized businesses (SMB) to large enterprises. It offers on-demand public cloud-based mobile testing over more than 500 Android and iOS devices and private cloud and on-premise setup offerings for enterprises.

pCloudy has also been recognised by top industry analysts including Gartner that included the products in the latest report on mobile testing tool landscape.