Capture Performance Data Using BrowserMob Proxy and Selenium

BrowserMob Proxy is an open source tools which is used to capture performance data for a web applications in an HAR format. It also allows to manipulate browser behavior and traffic, such as simulating network traffic, rewriting HTTP requests and responses etc.

Manually, we can monitor performance of a web application by using browser tools like Firebug or Developers Tools etc. BrowserMob proxy helps us to capture client side performance data for a web application using Selenium WebDriver automated tests. In this example, we will configure BrowserMob proxy for Selenium WebDriver and capture the performance data in HAR format.

HTTP Archive format or HAR is a JSON-formatted archive file format for logging of a web browser's interaction with a web application.

Wikipedia

In the below example, we will be taking TestNG annotations help to start and stop proxy server before the tests and close after all tests completes execution. BrowserMob Proxy makes it easy to use a proxy in Selenium tests. You can look at the sample snippet provided by BrowserMob GitHub.

    // start the proxy
    BrowserMobProxy proxy = new BrowserMobProxyServer();
    proxy.start(0);

    // get the Selenium proxy object
    Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);

    // configure it as a desired capability
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);

    // start the browser up
    WebDriver driver = new FirefoxDriver(capabilities);

    // enable more detailed HAR capture, if desired (see CaptureType for the complete list)
    proxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);

    // create a new HAR with the label "yahoo.com"
    proxy.newHar("yahoo.com");

    // open yahoo.com
    driver.get("http://yahoo.com");

    // get the HAR data
    Har har = proxy.getHar();

Below are the methods that we use to start and stop BrowserMobProxy server. If you want to start server on specific port, we need to pass the port value

proxy.start() - Starts the proxy on port 0.
proxy.start(int port) - Starts the proxy on the specified port. Ff the specified proxy has already been started, it throws 'java.lang.IllegalStateException'.
proxy.stop() - shutdown the proxy server gracefully

We can use any of the below methods to collect performance data in an HAR format based on the requirement

Har newHar() - Creates a new HAR file with the default page name.
Har newHar(String initialPageRef) - Creates a new HAR file with the specified 'initialPageRef' as the page name and page title.
Har newHar(String initialPageRef, String initialPageTitle) - Creates a new HAR file with the specified 'initialPageRef' page name and 'initialPageTitle' page title.

You have to download BrowserMob Proxy and make sure that all the dependencies are imported in to the project along with selenium server. Once you unzip browsermob zip folder, Navigate to the lib directory and add. Latest released version is 2.1.2 (July 23, 2016). For more information on the new release, you can visit the BrowserMob Proxy page on Github

package com.easy;

import java.io.File;
import java.io.IOException;

import org.openqa.selenium.By;
import org.openqa.selenium.Proxy;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

import net.lightbody.bmp.BrowserMobProxy;
import net.lightbody.bmp.BrowserMobProxyServer;
import net.lightbody.bmp.client.ClientUtil;
import net.lightbody.bmp.core.har.Har;
import net.lightbody.bmp.proxy.CaptureType;

public class BrowserMobExample {
	
	String driverPath = "F:/drivers/chromedriver/";
	String sFileName = "F:/SeleniumEasy.har";
	
	public WebDriver driver;
	public BrowserMobProxy proxy;
	
	@BeforeTest
	public void setUp() {
		
	   // start the proxy
	    proxy = new BrowserMobProxyServer();
	    proxy.start(0);

	    //get the Selenium proxy object - org.openqa.selenium.Proxy;
	    Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);

	    // configure it as a desired capability
	    DesiredCapabilities capabilities = new DesiredCapabilities();
	    capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
		
	    //set chromedriver system property
		System.setProperty("webdriver.chrome.driver", driverPath+"chromedriver.exe");
		driver = new ChromeDriver(capabilities);
		
	    // enable more detailed HAR capture, if desired (see CaptureType for the complete list)
	    proxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);

	    // create a new HAR with the label "seleniumeasy.com"
	    proxy.newHar("seleniumeasy.com");

	    // open seleniumeasy.com
	    driver.get("https://seleniumeasy.com/");
        
	}
	
	@Test
	public void testCaseOne() {
		System.out.println("Navigate to selenium tutorials page");
		driver.findElement(By.linkText("Selenium"));
	}
	
	@AfterTest
	public void tearDown() {

		// get the HAR data
		Har har = proxy.getHar();

		// Write HAR Data in a File
		File harFile = new File(sFileName);
		try {
			har.writeTo(harFile);
		} catch (IOException ex) {
			 System.out.println (ex.toString());
		     System.out.println("Could not find file " + sFileName);
		}
		
		if (driver != null) {
			proxy.stop();
			driver.quit();
		}
	}
}

If you're using Maven, add below dependency to your pom.xml file along with Selenium and TestNG dependencies. MVNRepository - BrowserMob Proxy Parent Project

    <dependency>
        <groupId>net.lightbody.bmp</groupId>
        <artifactId>browsermob-core</artifactId>
        <version>2.1.2</version>
        <scope>test</scope>
    </dependency>

All the network traffic during this test will be captured and saved to the seleniumeasy.har file (specified location) and can be viewed by a HAR Viewer, some of the har viewer links are provided below.

Below is the HAR file generated :

Selenium webdriver to access Shadow Elements

Har Viewer By Software is hard Har Viewer
Har Viewer by Ericduran.github
By Google Chrome Ad on HTTP Archive Viewer

Hope this helps you. Let me know if you face any issues configuring BrowserMob Proxy with Selenium.

Selenium Tutorials: 

Comments

I got your script where i can get those scripts and if you have newsletter sign up, it can be useful for me to know about the concepts of selenium.

Hello,

How can we Manipulate browser behavior & traffic by using BrowserMob Proxy ?

Hi,
With the above setup i was able to capture content in HAR format when i run my tests locally but when tried to run tests on a VM or on cloud, the browser opens up but does not load anything because there will be no internet connection for that session. Reason being, i am opening up a port locally and since tests are running in cloud, it will try to access this port on cloud which will be connected to a different network. How do we resolve this ?

I want to you browsermobproxy in TESTNG such that in @BeforeSuite annotation/method , it will get started and in @AfterSuite annotation/ method, it will get closed.

How can I meet my requirement ?

Add new comment

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.