Test Case Documentation

Motivation

From my point of view the test case documentation of automatic tests is important. It helps to maintain the tests in the future and the documentation is useful for co-workers if they would like to understand the tests or if they would like to gain more information about a test. It is also important that the documentation is easy available, e.g. from the test report.

We distinguish between log outputs that are written during test execution and the documentation of the test case itself. This post will outline how we cover both of these topics.

Environment

We implement our tests for our internet portal with Java, TestNG and Selenium WebDriver. We execute our automatic tests with the continuous integration system Jenkins (with testng-plugin).

Test case documentation

To document the test cases we are using Javadoc. The Javadoc for each test method contains a brief description what the test is doing and which steps are executed. The Javadoc class summary section outlines which area of the software is covered with the tests of the class.

One of the steps during building and executing the tests is to generate the Javadoc, which could be for example done with a Gradle task (or Ant, Maven).

Test execution documentation

I would recommend adding log output statements in the source code of the test. The log statements are written during the test run in a log file. This helps in analyzing why a test failed. The log statements can be accessed from the Jenkins TestNG Results view. This includes:

  • Log statements,
  • Output of the current URL when a test fails and
  • Generate a screenshot of the current page when a test fails

Listed below you will find some code snippets showing how we implemented this. The Java class containing all the snippets is attached to this post.

Log statements

The following example shows how to write log statements, this could be encapsulated in a utility method.

Reporter.log(new Date() + ">> Your log statement", true);

Output current URL and screenshot

In case of a test failure the current URL is written to the log file and also a screenshot is taken. In the example below you have to adapt the path of this image file to your environment. The link to the screenshot is also written to the log file, so that the screenshot can easily be accessed from the test results view. We implemented a method in our base test class with the TestNG annotation AfterMethod which is called after every test.

...
private static final String HREF = "Last URL: <a href="%s">%s</a>";
private static final String HREF_IMG = "&lt;a href="%s"&gt;&lt;img src="%s" alt="" width="100" height="100" /&gt;&lt;/a&gt;";
...
@AfterMethod (alwaysRun = true)
public void logResult(final ITestContext context, final ITestResult result)
 throws IOException {
  Reporter.setCurrentTestResult(result);
  if (!result.isSuccess()) {
    Reporter.log(String.format(HREF, driver.getCurrentUrl(),
      driver.getCurrentUrl()));
    File scrFile =
     (TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
    String imagePath = result.getName() + ".jpg";
    File file = new File(imagePath);
    FileUtils.copyFile(scrFile, file);
    Reporter.log(String.format(
      HREF_IMG, file.getName(), file.getName()));
  }
}

Linking test report with Javadoc

We include in our test logging also a hyperlink to the Javadoc at the beginning of the log output of each test. We find this helpful because it saves time to look up the Javadoc. In order to get this link for every test case we implement a method with the TestNG annotation BeforeMethod in our base test class. This method is called before the test is executed.

import java.lang.reflect.Method;// Adapt the javadoc path to your environment
private static final String JAVADOC_LINK = "<a href="build/docs/javadoc/%s/%s.html#%s">%s</a>";
 
@BeforeMethod (alwaysRun = true)
public void logTestStart(final ITestResult result, final Method method) {
  Reporter.setCurrentTestResult(result);
  Class&lt;?&gt; c = this.getClass();
  String javadoc = String.format(JAVADOC_LINK,
    c.getPackage().getName().replace(".", "/"),
    c.getSimpleName(), method.getName(), 
    c.getSimpleName() + ":" + method.getName());
  Reporter.log("&gt;&gt; Start test method: " + javadoc);
}