Compare Screenshots with Selenium WebDriver

Use case and motivation

Sometimes it‘s necessary to test the design or the correct position of one or more elements on a website. In this case, it could be useful to compare screenshots of the tested website with a reference screenshot.

And it’s a good idea, to do this kind of test automatically.

Below we will show how to compare a taken screenshot of any website with a reference screenshot and which tools are recommendable to do this smart.

We will point some pitfalls and defiance with this. E.g. how the test reacts with changing elements like advertising media.

Compare screenshots implementation

The example below shows how an image comparison with ImageMagick® and im4java can be implemented. The source code is not following coding standards, it serves only to illustrate the image comparison.

ImageMagick can be used to create, edit, compose, or convert bitmap images. The functionality of ImageMagick is typically utilized from the command line. Im4java is a pure-java interface to the ImageMagick command line.

The first source code snippet shows the method to compare images which we are using later. The different metrics which can be used to compare images are explained on the ImageMagic website. When the images are not equal the compare command will throw an exception.

import org.im4java.core.CompareCmd;
import org.im4java.process.StandardStream;
import org.im4java.core.IMOperation;
...
boolean compareImages (String exp, String cur, String diff) {
  // This instance wraps the compare command
  CompareCmd compare = new CompareCmd();
 
  // For metric-output
  compare.setErrorConsumer(StandardStream.STDERR);
  IMOperation cmpOp = new IMOperation();
  // Set the compare metric
  cmpOp.metric("mae");
 
  // Add the expected image
  cmpOp.addImage(exp);
 
  // Add the current image
  cmpOp.addImage(cur);
 
  // This stores the difference
  cmpOp.addImage(diff);
 
  try {
    // Do the compare
    compare.run(cmpOp);
    return true;
  }
  catch (Exception ex) {
    return false;
  }
}

The next source code snippet is using the just introduced method to compare images. At first a webpage is opened and a screenshot is taken.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
...
// Get the driver and open the page
WebDriver driver = new FirefoxDriver();
driver.get("http://testandwin.net");
 
// Take Screenshot
File scrFile = ((TakesScreenshot)driver).
getScreenshotAs(OutputType.FILE);
 
String current = "c:/temp/image.png";
FileUtils.copyFile(scrFile, new File(current));
 
// Compare the images
boolean compareSuccess =
  compareImages("c:/temp/expected.png", current, "c:/temp/difference.png");
 
// Close the driver
driver.close();

It is also possible to take a screenshot only from a certain web element. If you would like to do this you can include the following code snippet before the FileUtils.copyFile(…) call.

import java.awt.image.BufferedImage;
import javax.imageio.ImageIO;
import org.openqa.selenium.Point;
...
WebElement webElement = ...;
BufferedImage image = ImageIO.read(scrFile);
Point point = webElement.getLocation();
BufferedImage elementImage = image.getSubimage(
      point.getX(), point.getY(), 
      webElement[0].getSize().getWidth(), webElement[0].getSize().getHeight());
ImageIO.write(elementImage, "png", scrFile);

How to deal withing changing parts

Most websites contain dynamic elements like advertising media, version numbers, dates, etc. These elements make it almost impossible to compare screenshots. The solution we are using is to hide those web elements with the method listed below.

hideElement(WebElement e, WebDriver d) {
  ((JavascriptExecutor)d).executeScript("arguments[0].style.visibility='hidden'", e);
}

Compare Image Size

When compare screenshots it could be useful to compare first the image size because when the image size is different the comparison will fail. For example image sizes can differ when taking screenshots on different machines.

import java.awt.image.BufferedImage;
...
BufferedImage image = ImageIO.read(new File(path));
String size = image.getWidth() + "x" + image.getHeight();

Prerequisites

Before using  these examples, you have to install ImageMagick on the machine which is running the tests. The installation is described on ImageMagick website.

Additionally you have to include the im4java jar in your classpath.

Example images

Below you will find one example of the screenshot comparison. The first one shows the current image, the second one the expected image and the third one is the difference image.
Compare Screenshots - Current Image

Compare Screenshots - Source Image

Compare Screenshots - Difference Image

2 thoughts on “Compare Screenshots with Selenium WebDriver”

Comments are closed.