Extent Reports using TestNG Listeners in POM

As we know all know that ExtentReport is an HTML reporting library (open source) which can be integrated with Selenium WebDriver.

In this article, we will teach you how to develop advanced extent reports / integrate Extent Reports with your existing test automation framework.

We assume you have a working knowledge on Selenium,TestNG and Maven.

Let us start now !!!!

Importance of Reporting in a Test Automation Framework :-

Even if you have the best automation framework, without reporting it's difficult to know just what went wrong and where should we fix if it is an automation issue. These reports help us to make important decisions as per the release requirement. Automated reporting provide valuable insights into not only the big picture of the tests executed, also the details of each test case passed and failed with screenshots.

Page Object Model is a design pattern which is very popular in test automation for enhancing test maintenance and reducing code duplication. POM is best for the applications which has multiple pages.

If your not familiar using page object model, please refer to understand how page object design patterns works. We will be using page object model example from previous article where we already created page objects and tests. And will add few changes to integrate extent reports using TestNG ITestListeners.

Let’s look at page object model example and add ExtentReports : -

1. Create a maven project and add below dependencies to pom.xml

<!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-server -->
	<dependency>
	    <groupId>org.seleniumhq.selenium</groupId>
	    <artifactId>selenium-server</artifactId>
	    <version>3.14.0</version>
	</dependency>
	<!-- https://mvnrepository.com/artifact/org.testng/testng -->
	<dependency>
	    <groupId>org.testng</groupId>
	    <artifactId>testng</artifactId>
	    <version>6.14.3</version>
	    <scope>test</scope>
	</dependency>
		 <dependency>
	<groupId>com.aventstack</groupId>
	<artifactId>extentreports</artifactId>
	<version>3.1.5</version>
</dependency>

Now right click on folder and add new Class file in it and name it to TestBase i.e. TestBase.java to create a driver, where all our tests and page classes use this driver.

package com.vsysq.base;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Parameters;

public class TestBase {

	private WebDriver driver;
	private static String driverPath = "D:\\chromedriver\";

	public WebDriver getDriver() {
		return driver;
	}

	private void setDriver(String browserType, String appURL) {
		switch (browserType) {
		case "chrome":
			driver = initChromeDriver(appURL);
			break;
		case "firefox":
			driver = initFirefoxDriver(appURL);
			break;
		default:
			System.out.println("browser : " + browserType + " is invalid, Launching Firefox as browser of choice..");
			driver = initFirefoxDriver(appURL);
		}
	}

	private static WebDriver initChromeDriver(String appURL) {
		System.out.println("Launching google chrome with new profile..");
		System.setProperty("webdriver.chrome.driver", driverPath + "chromedriver.exe");
		WebDriver driver = new ChromeDriver();
		driver.manage().window().maximize();
		driver.navigate().to(appURL);
		return driver;
	}

	private static WebDriver initFirefoxDriver(String appURL) {
		System.out.println("Launching Firefox browser..");
		WebDriver driver = new FirefoxDriver();
		driver.manage().window().maximize();
		driver.navigate().to(appURL);
		return driver;
	}

	@Parameters({ "browserType", "appURL" })
	@BeforeClass
	public void initializeTestBaseSetup(String browserType, String appURL) {
		try {
			setDriver(browserType, appURL);

		} catch (Exception e) {
			System.out.println("Error....." + e.getStackTrace());
		}
	}

	@AfterClass
	public void tearDown() {
		driver.quit();
	}
}

Now let us create Page Objects for the required classes one by one.

First will create BasePage.java to add locators and methods of this page object class which can be used to interact with the UI of that page.

package com.vsysq.pages;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;

public class BasePage {
	protected WebDriver driver;
	private By signInButton = By.linkText("Sign in");

	public BasePage(WebDriver driver) {
		this.driver = driver;
	}

	public SignInPage clickSignInBtn() {
		System.out.println("clicking on sign in button");
		WebElement signInBtnElement = driver.findElement(signInButton);
		if (signInBtnElement.isDisplayed() || signInBtnElement.isEnabled())
			signInBtnElement.click();
		else
			System.out.println("Element not found");
		return new SignInPage(driver);
	}

	public void clickImagesLink() {
		// It should have a logic to click on images link
		// And it should navigate to google images page
	}

	public String getPageTitle() {
		String title = driver.getTitle();
		return title;
	}

	public boolean verifyBasePageTitle() {
		String expectedPageTitle = "Google";
		return getPageTitle().contains(expectedPageTitle);
	}
}

Let us create an other page CreateAccountPage.java :-

package com.vsysq.pages;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;

public class CreateAccountPage {

	private WebDriver driver;
	private By headerPageTxt = By.cssSelector(".signuponepage.main.content.clearfix>h1");

	public CreateAccountPage(WebDriver driver) {
		this.driver = driver;
	}

	public String getPageTitle() {
		String title = driver.getTitle();
		return title;
	}

	public boolean verifyPageTitle() {
		String pageTitle = "Create your Google Account";
		return getPageTitle().contains(pageTitle);
	}

	public boolean verifyCreateAccountPageText() {
		WebElement element = driver.findElement(headerPageTxt);
		String pageText = "Create your Google Account";
		return element.getText().contains(pageText);
	}

	public void createAccount() {
		// need to write steps for creating an account
	}
}

Create other page class 'SignInPage.java'

package com.vsysq.pages;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;

public class SignInPage {

	private WebDriver driver;

	private By headerPageText = By.cssSelector(".hidden-small");
	private By createAccountLink = By.id("link-signup");
	private By emailTextBox = By.id("Email");
	private By passwordTextBox = By.id("Passwd");
	private By loginBtn = By.id("signIn");
	private By errorMsgTxt = By.id("errormsg_0_Passwd");

	public SignInPage(WebDriver driver) {
		this.driver = driver;
	}

	public String getSignInPageTitle() {
		String pageTitle = driver.getTitle();
		return pageTitle;
	}

	public boolean verifySignInPageTitle() {
		String expectedTitle = "Sign in - Google Accounts";
		return getSignInPageTitle().contains(expectedTitle);
	}

	public boolean verifySignInPageText() {
		WebElement element = driver.findElement(headerPageText);
		String pageText = element.getText();
		String expectedPageText = "Sign in with your Google Account";
		return pageText.contains(expectedPageText);
	}

	public CreateAccountPage clickonCreateAnAccount() {
		WebElement element = driver.findElement(createAccountLink);
		if (element.isDisplayed() || element.isEnabled())
			element.click();
		return new CreateAccountPage(driver);
	}

	public boolean verifySignIn() {
		enterUserName("test");
		enterPassword("pass");
		clickOnSignIn();
		return getErrorMessage().contains("incorrect");
	}

	public void enterUserName(String userName) {
		WebElement emailTxtBox = driver.findElement(emailTextBox);
		if (emailTxtBox.isDisplayed())
			emailTxtBox.sendKeys(userName);
	}

	public void enterPassword(String password) {
		WebElement passwordTxtBox = driver.findElement(passwordTextBox);
		if (passwordTxtBox.isDisplayed())
			passwordTxtBox.sendKeys(password);
	}

	public void clickOnSignIn() {
		WebElement signInBtn = driver.findElement(loginBtn);
		if (signInBtn.isDisplayed())
			signInBtn.click();
	}

	public String getErrorMessage() {
		String strErrorMsg = null;
		WebElement errorMsg = driver.findElement(errorMsgTxt);
		if (errorMsg.isDisplayed() && errorMsg.isEnabled())
			strErrorMsg = errorMsg.getText();
		return strErrorMsg;
	}
}

Now let us create TestCase classes in Tests package and name it according to the test.

First will create test class and name it as BasePageTest.java

package com.tests;

import org.openqa.selenium.WebDriver;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import com.aventstack.extentreports.Status;
import com.vsysq.base.ExtentTestManager;
import com.vsysq.base.TestBase;
import com.vsysq.pages.BasePage;

public class BasePageTest extends TestBase {

	private WebDriver driver;

	@BeforeClass
	public void setUp() {
		driver = getDriver();
	}

	@Test
	public void verifyHomePage() {
		System.out.println("Home page test...");
		BasePage basePage = new BasePage(driver);
		Assert.assertTrue(basePage.verifyBasePageTitle(), "Home page title doesn't match");
	}

	@Test
	public void baseTest1() {
		ExtentTestManager.getTest().log(Status.INFO, "Hellooo started base test1");
		System.out.println("Hey im in test1 test");
		ExtentTestManager.getTest().log(Status.INFO, "Hey im in base test1 1");
		ExtentTestManager.getTest().log(Status.INFO, "Hey im in base test1 2");
		ExtentTestManager.getTest().log(Status.INFO, "Hey im in base test1 3");
		ExtentTestManager.getTest().log(Status.INFO, "Hey im in base test1 4");
	}

	@Test
	public void baseTest2() throws InterruptedException {
		ExtentTestManager.getTest().log(Status.INFO, "Hellooo started base test2");
		System.out.println("Hey im in test2 test");
		Thread.sleep(3000);
		ExtentTestManager.getTest().log(Status.INFO, "Hey im in base test2 1");
		ExtentTestManager.getTest().log(Status.INFO, "Hey im in base test2 2");
	}

	@Test
	public void baseTest3() {
		ExtentTestManager.getTest().log(Status.INFO, "Hellooo started base test3");
		System.out.println("Hey im in test3 test");
		ExtentTestManager.getTest().log(Status.INFO, "Hey im in base test3 1");
		ExtentTestManager.getTest().log(Status.INFO, "Hey im in base test3 2");
	}

}

If you observe the above test class, we have used Extent report logs. We can input logs in the HTML report. To log the inputs to the report, we need to use the Status.INFO as first parameter and text to be displayed as second parameter for log() method. Also use Status.PASS / Status.FAIL in the test Methods when needed.

In our example, first we need to get ExtentTest instance from ExtentTestManager and then add log method in testng tests.

ExtentTestManager.getTest().log(Status.INFO, "Hellooo started base test1");

Now create a class and name it as CreateAccountTest.java

package com.tests;

import org.openqa.selenium.WebDriver;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import com.vsysq.base.TestBase;
import com.vsysq.pages.BasePage;
import com.vsysq.pages.CreateAccountPage;
import com.vsysq.pages.SignInPage;

public class CreateAccountTest extends TestBase {
	private WebDriver driver;
	private SignInPage signInPage;
	private BasePage basePage;
	private CreateAccountPage createAccountPage;

	@BeforeClass
	public void setUp() {
		driver = getDriver();
	}

	@Test
	public void verifyCreateAnAccountPage() {
		System.out.println("Create An Account page test...");
		basePage = new BasePage(driver);
		signInPage = basePage.clickSignInBtn();
		createAccountPage = signInPage.clickonCreateAnAccount();
		Assert.assertTrue(createAccountPage.verifyPageTitle(), "Page title not matching");
		Assert.assertTrue(createAccountPage.verifyCreateAccountPageText(), "Page text not matching");
	}

	@Test
	public void createAccountExample1() {
		System.out.println("Hey im in example1 test");
	}

	@Test
	public void createAccountExample2() {
		System.out.println("Hey im in Example2 test");
	}
}

Create an other class and name that as SignInPageTest.java

package com.tests;

import org.openqa.selenium.WebDriver;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import com.vsysq.base.TestBase;
import com.vsysq.pages.BasePage;
import com.vsysq.pages.SignInPage;

public class SignInPageTest extends TestBase {

	private WebDriver driver;
	private SignInPage signInPage;
	private BasePage basePage;

	@BeforeClass
	public void setUp() {
		//Get driver
		driver = getDriver();
	}

	@Test
	public void verifySignInFunction() {
		System.out.println("Sign In functionality details...");
		basePage = new BasePage(driver);
		signInPage = basePage.clickSignInBtn();
		Assert.assertTrue(signInPage.verifySignInPageTitle(), "Sign In page title doesn't match");
		Assert.assertTrue(signInPage.verifySignInPageText(), "Page text not matching");
		Assert.assertTrue(signInPage.verifySignIn(), "Unable to sign in");
	}

	@Test
	public void SignInPageTestOne() {
		System.out.println("Hey im in example test One");
	}

	@Test
	public void SignInPageTestTwo() {
		System.out.println("Hey im in example test two");
	}

	@Test
	public void SignInPageTestThree() {
		System.out.println("Hey im in example test three");
	}
}

Until here, we have create a POM style framework with few page object classes and test classes.

Now let's integrate Extent Report with Selenium WebDriver. if you are new to ExtentReports, please refer previous article which talks about Introduction to ExtentReports with basic example.

Let us create a class and name it as 'ExtentManager.java'. In this class we will create and instance for ExtentReports using createInstance() with HTML config params like the report location, Report name, Theme format etc.

Let's create a new class ExtentManager.java to allow us to explore this code as we talk about it :-

package com.vsysq.base;

import java.io.File;

import com.aventstack.extentreports.ExtentReports;
import com.aventstack.extentreports.reporter.ExtentHtmlReporter;
import com.aventstack.extentreports.reporter.configuration.ChartLocation;
import com.aventstack.extentreports.reporter.configuration.Theme;

public class ExtentManager {
    private static ExtentReports extent;
    private static String reportFileName = "Test-Automaton-Report"+".html";
    private static String fileSeperator = System.getProperty("file.separator");
    private static String reportFilepath = System.getProperty("user.dir") +fileSeperator+ "TestReport";
    private static String reportFileLocation =  reportFilepath +fileSeperator+ reportFileName;
  
 
    public static ExtentReports getInstance() {
        if (extent == null)
            createInstance();
        return extent;
    }
 
    //Create an extent report instance
    public static ExtentReports createInstance() {
        String fileName = getReportPath(reportFilepath);
       
        ExtentHtmlReporter htmlReporter = new ExtentHtmlReporter(fileName);
        htmlReporter.config().setTestViewChartLocation(ChartLocation.BOTTOM);
        htmlReporter.config().setChartVisibilityOnOpen(true);
        htmlReporter.config().setTheme(Theme.STANDARD);
        htmlReporter.config().setDocumentTitle(reportFileName);
        htmlReporter.config().setEncoding("utf-8");
        htmlReporter.config().setReportName(reportFileName);
        htmlReporter.config().setTimeStampFormat("EEEE, MMMM dd, yyyy, hh:mm a '('zzz')'");
 
        extent = new ExtentReports();
        extent.attachReporter(htmlReporter);
        //Set environment details
		extent.setSystemInfo("OS", "Windows");
		extent.setSystemInfo("AUT", "QA");
 
        return extent;
    }
     
    //Create the report path
    private static String getReportPath (String path) {
    	File testDirectory = new File(path);
        if (!testDirectory.exists()) {
        	if (testDirectory.mkdir()) {
                System.out.println("Directory: " + path + " is created!" );
                return reportFileLocation;
            } else {
                System.out.println("Failed to create directory: " + path);
                return System.getProperty("user.dir");
            }
        } else {
            System.out.println("Directory already exists: " + path);
        }
		return reportFileLocation;
    }
 
}

Now let us create ExtentTestManager class to get ExtentReports instance from ExtentManager class. This can be used to create instance of ExtentTest and put into extentTestMap with current thread id.

If you want to this ExtentTest instance in your TestListiner or Selenium webdriver test classes, we can use getTest() method, which returns ExtentTest instance from extentTestMap by using current thread id.

Let's create ExtentTestManager.java class like below :-

package com.vsysq.base;

import java.util.HashMap;
import java.util.Map;

import com.aventstack.extentreports.ExtentReports;
import com.aventstack.extentreports.ExtentTest;

public class ExtentTestManager {
	static Map<Integer, ExtentTest> extentTestMap = new HashMap<Integer, ExtentTest>();
	static ExtentReports extent = ExtentManager.getInstance();

	public static synchronized ExtentTest getTest() {
		return (ExtentTest) extentTestMap.get((int) (long) (Thread.currentThread().getId()));
	}

	public static synchronized void endTest() {
		extent.flush();
	}

	public static synchronized ExtentTest startTest(String testName) {
		ExtentTest test = extent.createTest(testName);
		extentTestMap.put((int) (long) (Thread.currentThread().getId()), test);
		return test;
	}
}

Now will create TestListener.java class and implement TestNG ITestListener - A listener for test running. You can check more about TestNG Listeners which we have discussed with multiple examples using TestListiners.

package com.vsysq.base;

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;

import com.aventstack.extentreports.Status;

public class TestListener implements ITestListener {


	public void onStart(ITestContext context) {
		System.out.println("*** Test Suite " + context.getName() + " started ***");
	}

	public void onFinish(ITestContext context) {
		System.out.println(("*** Test Suite " + context.getName() + " ending ***"));
		ExtentTestManager.endTest();
		ExtentManager.getInstance().flush();
	}

	public void onTestStart(ITestResult result) {
		System.out.println(("*** Running test method " + result.getMethod().getMethodName() + "..."));
		ExtentTestManager.startTest(result.getMethod().getMethodName());
	}

	public void onTestSuccess(ITestResult result) {
		System.out.println("*** Executed " + result.getMethod().getMethodName() + " test successfully...");
		ExtentTestManager.getTest().log(Status.PASS, "Test passed");
	}

	public void onTestFailure(ITestResult result) {
		System.out.println("*** Test execution " + result.getMethod().getMethodName() + " failed...");
		ExtentTestManager.getTest().log(Status.FAIL, "Test Failed");
	}

	public void onTestSkipped(ITestResult result) {
		System.out.println("*** Test " + result.getMethod().getMethodName() + " skipped...");
		ExtentTestManager.getTest().log(Status.SKIP, "Test Skipped");
	}

	public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
		System.out.println("*** Test failed but within percentage % " + result.getMethod().getMethodName());
	}

}

If you also want to capture screenshots for failed cases and attach them to the extent report, please replace below method in your TestListener. We have using Selenium FileHandler class to capture the screenshot and the add the image location with Extent Report by using MediaEntityBuilder.createScreenCaptureFromPath(targetLocation)

public void onTestFailure(ITestResult result) {
		System.out.println("*** Test execution " + result.getMethod().getMethodName() + " failed...");
		System.out.println((result.getMethod().getMethodName() + " failed!"));
		String targetLocation=null;
		ITestContext context = result.getTestContext();
		WebDriver driver = (WebDriver) context.getAttribute("driver");

		String testClassName = getTestClassName(result.getInstanceName()).trim();
		String timeStamp = Util.getCurrentTimeStamp(); // get timestamp 
		String testMethodName = result.getName().toString().trim();
		String screenShotName = testMethodName + timeStamp + ".png";
		
		try {
			File file = new File("TestReport" + fileSeperator + "screenshots"); // Set screenshots folder
			if (!file.exists()) {
				System.out.println("Screenshots directory created " + file);
				file.mkdir();
			}

			File screenshotFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
			targetLocation = reportsPath + fileSeperator + testClassName +fileSeperator+ screenShotName;// define location
			File targetFile = new File(targetLocation);
			 FileHandler.copy(screenshotFile, targetFile);
			
		} catch (Exception e) {
			System.out.println("An exception occurred while taking screenshot " + e.getCause());
		}
		
		//attach screenshots to report
		try {
			ExtentTestManager.getTest().fail("Screenshot", MediaEntityBuilder.createScreenCaptureFromPath(targetLocation).build());
		} catch (IOException e) {
		
			System.out.println("An exception occured while taking screenshot " + e.getCause());
		}
		ExtentTestManager.getTest().log(Status.FAIL, "Test Failed");
	}

OK, let's try to execute tests to make sure everything is working.

To start executing tests, create a new file called testng.xml, and add the following code to it:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Test Suite" verbose="1">
	<listeners>
		<listener class-name="com.vsysq.base.TestListener" />
	</listeners>
	<parameter name="browserType" value="chrome" />
	<parameter name="appURL" value="https://google.com" />

	<test name="Admin Tests">
		<classes>
			<class name="com.tests.BasePageTest" />
			<class name="com.tests.SignInPageTest" />
			<class name="com.tests.CreateAccountTest" />
		</classes>
	</test>
</suite>

After creating all the above classes, your project should look like : -

POM framework structure

Now you can run the test by right clicking on testng.xml file or, you can have maven surefire plug-in to execute your tests using maven.

Once you run, you should see an instance of chrome browser automatically open up! Google should automatically be loaded in a tab and rest followed.

Let us look at the extent report generated :-

Extent report detailed view

You can also view the Dashboard design by clicking on the waves icon in the left menu : -

Extent report dashboard view

There is also nothing to stop you running the test methods in parallel. Let's try this!

Below is the modified testng.xml to execute test methods in parallel :-

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Test Suite" parallel="methods" thread-count="2" verbose="1">
	<listeners>
		<listener class-name="com.vsysq.base.TestListener" />
	</listeners>
	<parameter name="browserType" value="chrome" />
	<parameter name="appURL" value="https://google.com" />

	<test name="Admin Tests">
		<classes>
			<class name="com.tests.BasePageTest" />
			<class name="com.tests.SignInPageTest" />
			<class name="com.tests.CreateAccountTest" />
		</classes>
	</test>
</suite>
Selenium Tutorials: 

Add new comment