Allure is an open-source framework designed to create interactive and comprehensive test report by Yandex QA Team

Each time when we run automation tests, we will have test results to view details about no. of tests passed, failed and failure details etc. And few reports also include test failure screenshots. We have discussed selenium examples on Extent report and testng custom report with screenshots.

Let us now look at Allure Reports

1. How does allure report works ?

When we run our tests, every popular test framework generates junit-style xml report which will be used by Allure to generate HTML report.

In the below example, we use maven surefire plugin which automatically generates xml test reports and stored in target/surefire-reports. And these XML files are transformed to an HTML report by Allure.

Allure reports has provided adapters for Java, PHP, Ruby, Python, Scala and C# test frameworks.

2. How to generate Allure Reports ?

To generate allure report, we have to first install Allure command line application.

The simplest way to do this For Windows, using Scoop commandline-installer. Scoop is a command-line installer for Windows.

Run below command from your PowerShell to install scoop to its default location (C:\Users\\scoop)

One of the easiest way to start PowerShell in Windows, is using Run window. Press Win + R keys on your keyboard, then type powershell and press Enter or click OK.

iex (new-object net.webclient).downloadstring('https://get.scoop.sh')

If you get an error, you might have to set the execution policy (i.e. enable Powershell)with Set-ExecutionPolicy RemoteSigned -scope CurrentUser

In the above case, it was already installed, so you a message as 'Scoop is already installed'. You can run 'scoop update' to get the latest version any time.

To install Allure, using Scoop, run the below command

scoop install allure

You can also do this manually by downloading the latest version as a zip archive from bintray.

1. Unpack the archive to allure-command line directory.
2. Navigate to bin directory and
3. Add allure to system PATH.

You can find more information on Allure documentation.

Below simple example will help you to implement Allure Reports using TestNG, Java and Maven. Later will see an other example using Selenium page object model framework and integrate with your existing test automation framework without any major changes.

Step 1 : - Create simple maven project like below

Step 2 : - Add allure maven dependencies to pom.xml. And we should also add AspectJ Weaver dependency along with allure.

Your pom.xml file should look like below after adding dependencies along with maven surefire plugin

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.test</groupId>
  <artifactId>AllureExample</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  
    <properties>
		<jre.level>1.8</jre.level>
		<jdk.level>1.8</jdk.level>
		<aspectj.version>1.9.2</aspectj.version>
	</properties>

	<build>
		<plugins>
			<!-- Compiler plug-in -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>3.5.1</version>
				<configuration>
					<source>${jdk.level}</source>
					<target>${jdk.level}</target>
				</configuration>
			</plugin>
			<!-- Added Surefire Plugin configuration to execute tests -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>2.19.1</version>
				<configuration>
					<testFailureIgnore>true</testFailureIgnore>
					<argLine>
						-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
					</argLine>
					<suiteXmlFiles>
						<suiteXmlFile>testng.xml</suiteXmlFile>
					</suiteXmlFiles>
				</configuration>
				<dependencies>
					<dependency>
						<groupId>org.aspectj</groupId>
						<artifactId>aspectjweaver</artifactId>
						<version>${aspectj.version}</version>
					</dependency>
				</dependencies>
			</plugin>
		</plugins>
	</build>
	
		<dependencies>
		<!-- https://mvnrepository.com/artifact/org.testng/testng -->
		<dependency>
			<groupId>org.testng</groupId>
			<artifactId>testng</artifactId>
			<version>6.14.3</version>
		</dependency>
		<!-- https://mvnrepository.com/artifact/io.qameta.allure/allure-testng -->
		<dependency>
			<groupId>io.qameta.allure</groupId>
			<artifactId>allure-testng</artifactId>
			<version>2.7.0</version>
		</dependency>
	</dependencies>

	<reporting>
		<excludeDefaults>true</excludeDefaults>
		<plugins>
			<plugin>
				<groupId>io.qameta.allure</groupId>
				<artifactId>allure-maven</artifactId>
				<version>2.9</version>
			</plugin>
		</plugins>
	</reporting>
  
</project>

Step 3 : - Create two test files in src/test/java folder

File name - Test1.java

package com.test;

import org.testng.Assert;
import org.testng.annotations.Test;

public class Test1 {
	
	@Test
	public void example1() {
		Assert.assertEquals(2, 3, "This test should be failed");
	}
	
	@Test
	public void example2() {
		Assert.assertEquals(2, 2);
	}
	
	@Test
	public void example3() {
		Assert.assertTrue(true, "This test should be failed");
	}
	
	@Test
    public void example4() {
        Assert.fail("This test should be failed");
    }

    @Test(dependsOnMethods = "example1")
    public void skippedByDependencyTest() {
    	
    }

}

File name - Test2.java

package com.test;

import org.testng.Assert;
import org.testng.annotations.Test;

public class Test2 {
	
	@Test
	public void test1() {
		Assert.assertEquals(2, 3);
	}
	
	@Test
	public void test2() {
		Assert.assertEquals(2, 2);
	}
	
	@Test
	public void test3() {
		Assert.assertTrue(true, "This test should be failed");
	}
	
	@Test
    public void test4() {
        Assert.fail("This test should be failed");
    }

    @Test(dependsOnMethods = "test4")
    public void skippedByDependencyTest() {
    	
    }

}

Step 4 : - Create testng.xml file like below to execute tests.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Test Suite" verbose="2">
	<test name="User Tests">
		<classes>
			<class name="com.test.Test1" />
			<class name="com.test.Test2" />
		</classes>
	</test>
</suite>

Step 5 : - Now, open command prompt, navigate to the project directory, and run tests using below command!

 mvn clean test

Or else If you are using any IDE, You have to run maven goal 'site' to generate a report.

Just right click on your project -> Run as -> Maven build... And then type 'site' as goal and run. Alternatively you can also type the goals 'clean test site', which executes all tests.

Once the execution is completed, By default, TestNG attaches a few basic listeners to generate HTML and XML reports and surefire generates TEST-*.xml files under ${basedir}/target/surefire-reports.

Step 6 : - Now let us run below command to generate allure reports

allure serve

The above command generates a report in the temporary folder from the data found in target/surefire-reports/ and then creates a local Jetty server instance, serves generated report and opens html report in the default browser.

To output Allure results to the build directory (the default is target/allure-results), We need to put the allure.properties file in src/test/resources/ and add below value :

File name : - allure.properties

allure.results.directory=target/allure-results

If you want to save html report on your project directory, run the below allure command.

allure generate

The above command generates 'allure-report' directory which has html report. But when you open the report, you just see Loading...

This is not the problem of Allure. The problem is related to default Webkit security settings which forbid doing Ajax requests on the local file system.

You have at least two possible solutions:

1. Serve index.html with some web-server like Nginx or Apache. This will emulate remote website and thus trick your browser.

2. With Allure CLI 2.0+ this can be done running the below command:

allure open

Above command will start the web server and opens the html report on your default browser. And the report looks like below -