TestNG Tests Run Count after Retry.

We have seen an example to Re-run failed tests using iRetryListener. In the previous example, the count of the tests run was changing based on the maxRetryCountvalue. To over come that, we need to add the below code snippet for onFinish method by implementing 'ITestListener'

package com.pack.test;

import java.util.Set;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;

public class TestListener implements ITestListener {
    @Override
	public void onFinish(ITestContext context) {
		Set<ITestResult> failedTests = context.getFailedTests().getAllResults();
		for (ITestResult temp : failedTests) {
			ITestNGMethod method = temp.getMethod();
			if (context.getFailedTests().getResults(method).size() > 1) {
				failedTests.remove(temp);
			} else {
				if (context.getPassedTests().getResults(method).size() > 0) {
					failedTests.remove(temp);
				}
			}
		}
	}
  
    public void onTestStart(ITestResult result) {   }
  
    public void onTestSuccess(ITestResult result) {   }
  
    public void onTestFailure(ITestResult result) {   }

    public void onTestSkipped(ITestResult result) {   }

    public void onTestFailedButWithinSuccessPercentage(ITestResult result) {   }

    public void onStart(ITestContext context) {   }
}  

Now please add the above code to the previous example to get the Test count based on the number of tests, Not based on the maxRetry count. And also we need to add the listener to the testng.xml file

<listeners>
       <listener class-name="com.pack.test.TestListener"/>
  </listeners>

After adding the above code to the previous example, the output should look like below:

[TestNG] Running:
  D:\Selenium\TestNgSample\testng.xml

Verify login page test started
Verify Forgot password page test started
Retrying test verifyForgotPasswordPage with status FAILURE for the 1 time(s).
Verify Forgot password page test started

===============================================
Parallel test runs
Total tests run: 2, Failures: 1, Skips: 0
===============================================
Test Frameworks: 

Comments

Nice example. It doesn't cater for the tests which gets skipped

Thanks for the nice blog. I able to generate TestNG reports as expected. In my project we have Allure report integration with TestNG. In the allure report the result is showing "cancelled" for failed & retried test case. ie. 2 Cancelled 1 Passed. Do you have any clue how can this be resolved?

If your test result is displaying Skipped and the total test count is mismatching, then you may have to replace the line "Set<ITestResult> failedTests = context.getFailedTests().getAllResults();" with "Set<ITestResult> failedTests = context.getSkippedTests().getAllResults();". This worked fine for me.

can u please share the code.... what i need to changed.... it is not working for me

Thank you providing such valuable information. I was able to easily update my reportng report.

Add new comment

Image CAPTCHA