Retry executing only Failed Tests using TestNG

There may be many reasons for a Test case getting failed, may be due to element not found or time out exception or stale element exception etc. Normally in automation after executing scripts/tests, we will check for the results and if the test fails because of above reasons we will re-run then again.

Instead of that we can ask testNG to execute the failed test cases again for X (we can define) number of times and check for the updated results.

To achieve this we need to implement TestNG IRetryAnalyzer. Below is the simple code snippet:

package com.pack.test;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class Retry implements IRetryAnalyzer {
    private int retryCount = 0;
    private int maxRetryCount = 1;

// Below method returns 'true' if the test method has to be retried else 'false' 
//and it takes the 'Result' as parameter of the test method that just ran
    public boolean retry(ITestResult result) {
        if (retryCount < maxRetryCount) {
            System.out.println("Retrying test " + result.getName() + " with status "
                    + getResultStatusName(result.getStatus()) + " for the " + (retryCount+1) + " time(s).");
            retryCount++;
            return true;
        }
        return false;
    }
    
    public String getResultStatusName(int status) {
    	String resultName = null;
    	if(status==1)
    		resultName = "SUCCESS";
    	if(status==2)
    		resultName = "FAILURE";
    	if(status==3)
    		resultName = "SKIP";
		return resultName;
    }
}

Create an other class 'RetryListener' by implementing 'IAnnotationTransformer'. We need to setRetryAnalyzer for iTestAnnotation. In the example below, add the above class name.

package com.pack.test;

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;

import org.testng.IAnnotationTransformer;
import org.testng.IRetryAnalyzer;
import org.testng.annotations.ITestAnnotation;

public class RetryListener implements IAnnotationTransformer {

	@Override
	public void transform(ITestAnnotation testAnnotation, Class testClass,
                          Constructor testConstructor, Method testMethod)	{

                    testAnnotation.setRetryAnalyzer(Retry.class);

    }
}
}

Let us see the example by executing simple tests below. In the below test verifyForgotPasswordPage, We are trying to verify the wrong text which will return False and the Test fails.

Now the second test will be executed for the value that we defined for 'maxRetryCount' in Retry Class.

package com.pack.test;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class TestNGExampleTests {
	WebDriver driver;
	String baseURL = "https://www.linkedin.com/";

	@BeforeClass
	public void setup() {
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
	}

	@Test(priority=1)
	public void verifyLoginPageText() {
		driver.navigate().to(baseURL);
		System.out.println("Verify login page test started");
		WebElement element = driver.findElement(By.cssSelector(".header>h2"));
		String headerText = element.getText();
		Assert.assertEquals(headerText, "Get started – it’s free.");
	}

	@Test(priority=2)
	public void verifyForgotPasswordPage() {
		driver.navigate().to(baseURL);
		System.out.println("Verify Forgot password page test started");
		WebElement element = driver.findElement(By.linkText("Forgot your password?"));
		element.click();
		WebElement pageTextElement = driver.findElement(By.cssSelector(".flow-login-content>fieldset>h1"));
		String pageText = pageTextElement.getText();
		Assert.assertEquals(pageText, "Wrong text");
		//Assert.assertEquals(pageText, "Change your password");
	}
}

After executing the above program, We need to add the Listener to testng.xml file. Below is syntax to add listener for RetryListener.

<listeners>
        <listener class-name="com.pack.test.RetryListener"/>
  </listeners>

After executing the above program, the output should look like below. The verifyForgotPasswordPage test will be executed for two times as we have defined 'maxRetryCount = 1', hence when the test fails, it will execute again for one time. Now the total count of tests it will show as 3, Failures 2. But actually we have only two tests. To handle this we need to adjust the count by implementing 'ITestListener' and update the count from 'onFinish' method.

[TestNG] Running:
  D:\Selenium\TestNgSample\testng.xml

Verify login page test started
Verify Forgot password page test started
Retrying test verifyForgotPasswordPage with status FAILURE for the 1 time(s).
Verify Forgot password page test started

===============================================
Parallel test runs
Total tests run: 3, Failures: 2, Skips: 0
===============================================
Test Frameworks: 

Comments

I need to make the test report as pass, but the test executed twice and 1 pass and 1 fail so entire suite is fail.
How to make entire suite as pass. How to achieve this ?

How to remove the duplicate entry of the failed test cases from the testng report?

As per above logic, after failing test case is skipping, its not reruning

Something doesnt seem right with the above code.
The test scripts that are re-executed get shown as skipped in the reportNG report!

Can anyone suggest the correction needed to be done??

I am executing the test cases using testng . I need to run the testcases for two times i.e the first time all the testcases and second time only the failed testcases. How to rerun the failed testcases and need the logs in separated folder each time.It means the first time results in folder and second time results in some othe folder

I have class in which Tests(@test) are sequentially dependent on previous ones . In this case if any of test fails , I want to re run all of its previous parent tests . With above only failing test gets executed again & not the entire sequence .
Does anyone know how to achieve that ?

Add new comment

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.