TestNG Tests Run Count after Retry.

We have seen an example to Re-run failed tests using iRetryListener. In the previous example, the count of the tests run was changing based on the maxRetryCountvalue. To over come that, we need to add the below code snippet for onFinish method by implementing 'ITestListener'

package com.pack.test;

import java.util.Set;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;

public class TestListener implements ITestListener {
    @Override
	public void onFinish(ITestContext context) {
		Set<ITestResult> failedTests = context.getFailedTests().getAllResults();
		for (ITestResult temp : failedTests) {
			ITestNGMethod method = temp.getMethod();
			if (context.getFailedTests().getResults(method).size() > 1) {
				failedTests.remove(temp);
			} else {
				if (context.getPassedTests().getResults(method).size() > 0) {
					failedTests.remove(temp);
				}
			}
		}
	}
  
    public void onTestStart(ITestResult result) {   }
  
    public void onTestSuccess(ITestResult result) {   }
  
    public void onTestFailure(ITestResult result) {   }

    public void onTestSkipped(ITestResult result) {   }

    public void onTestFailedButWithinSuccessPercentage(ITestResult result) {   }

    public void onStart(ITestContext context) {   }
}  

Now please add the above code to the previous example to get the Test count based on the number of tests, Not based on the maxRetry count. And also we need to add the listener to the testng.xml file

<listeners>
       <listener class-name="com.pack.test.TestListener"/>
  </listeners>

After adding the above code to the previous example, the output should look like below:

[TestNG] Running:
  D:\Selenium\TestNgSample\testng.xml

Verify login page test started
Verify Forgot password page test started
Retrying test verifyForgotPasswordPage with status FAILURE for the 1 time(s).
Verify Forgot password page test started

===============================================
Parallel test runs
Total tests run: 2, Failures: 1, Skips: 0
===============================================
Test Frameworks: 

Comments

Nice example. It doesn't cater for the tests which gets skipped

Thanks for the nice blog. I able to generate TestNG reports as expected. In my project we have Allure report integration with TestNG. In the allure report the result is showing "cancelled" for failed & retried test case. ie. 2 Cancelled 1 Passed. Do you have any clue how can this be resolved?

If your test result is displaying Skipped and the total test count is mismatching, then you may have to replace the line "Set<ITestResult> failedTests = context.getFailedTests().getAllResults();" with "Set<ITestResult> failedTests = context.getSkippedTests().getAllResults();". This worked fine for me.

can u please share the code.... what i need to changed.... it is not working for me

Thank you providing such valuable information. I was able to easily update my reportng report.

It's a nice example. It works perfectly for me. The best solution for avoiding mismatch the count in TestNG report and it is showing only after retry result. By this implementation, it would not be showing duplication of test cases in the TestNG report

Hi, how can i integrate this with surefire reports?

Hi, Is there any way to implement the same in emailable report?? In emailable report it is showing total number of failed count as
maxRetryCountvalue.

Which version of TestNG are you using?

Please use version 7.x.

I did this and it worked for failed and skipped test cases

@Override
public void onFinish(ITestContext context) {
Set<ITestResult> failedTests = context.getSkippedTests().getAllResults();

for (ITestResult temp : failedTests) {
ITestNGMethod method = temp.getMethod();
if (context.getFailedTests().getResults(method).size() > 0) {
failedTests.remove(temp);
} else {
if (context.getSkippedTests().getResults(method).size() > 0) {
failedTests.remove(temp);
}
}
}

}

Add new comment

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.