Acceptance testing with SpecFlow and Selenium WebDriver
Today I write about my experiences with the introduction of a solution for developing and performing acceptance testing for a web application using SpecFlow.
The test scenarios should be developed textually in the language of the stakeholders and not in a programming language. Another requirement of the solution was that the tests could be created with a minimum of support from the developers.
The testers, also the developers and other project participants must support the solution.
Tool Selection & Set-Up
The web application is developed in .NET. For the selection of the tools it was important for me to rely on tools that can be used with .NET too. The reason for this is that the developers of the project could support better, because they can use their familiar development environment and the existing Continuous Integration environment could be used.
As tool SpecFlow (website) was chosen because it allows to separate the development of the tests of the technical implementation. The format SpecFlow uses to describe test scenarios is the Gherkin language. With this description language it is possible to implement test scenarios textually in the Given / When / Then format.
Selenium WebDriver implements the access to the browser and NUnit runs the tests. SpecFlow also provides a commercial test runner (SpecFlow+ Runner).
As IDE Visual Studio was used with the extensions SpecFlow and Nunit.
First, I created a project of the type NUnit 3 Unit Test Project in Visual Studio and then added the packages as listed in the screenshot below to the project using the NuGet Package Manager to start implementing my acceptance testing:
Please excuse if the source code (you will find it attached to this post) does not correspond to the .NET conventions, I come from the Java world. If existing solutions are available for what I mentioned here, I also ask for apologies.
The aim was that to be as little as possible implemented in a programming language in the creation of the different test scenarios.
For the web application I had to test I needed mainly steps to click buttons or links, to enter values into input fields and steps to verify the value of a field. I created generic steps for this which can be reused multiple times for different test scenarios.
BaseSteps.cs class (please see attachment) implements these steps. However in developing these generic steps it must kept in mind that the implementation keeps readable and maintainable.
BaseSteps.cs also contains methods to launch the browser automatically and to stop it and to automatically take a screen shot in case a test scenario fails.
I could image further generic steps in this BaseSteps.cs class, e.g. selecting an element from a Select Box, verify whether a field contains the required value, …
Most of the methods in BaseSteps.cs contain declarative tags (called attribute) to associate run time information.
BeforeTestRun – This method is called before starting the first test scenario, e.g. to start the browser. AfterTestRun stops the browser.
AfterScenario – This method is called after execution of a test scenario. In my example I take in the method attached with this attribute a screenshot in case the test scenario has run into an error.
Given / When / Then – These attributes bind the step definition to the implementation.
I had encapsulated the calls to access the browser (open, close, …) in the class Browser.cs.
In order to find HTML elements with Selenium WebDriver you need to specify a element locator.
When setting up the generic steps you could use the locators directly in the test scenarios. I did not prosecute this idea further, because from my point of view the test scenarios would become less readable and in case a locator changes, you have to adapt all the test scenarios using this locator.
In order to keep the test scenarios readable and maintainable I put them into a config file. This had the advantage, that a readable name could be used in the test scenario (I would suggest to use the same name as shown on the page) and the locator is maintained only at one place.
In the example below the key Search … maps to the css locator input.search-field.
<appSettings> <add key="Search ..." value="cssselector:input.search-field"/> <add key="Search" value="cssselector:button.search-submit"/> </appSettings>
Below you will find two very simple test scenarios executing tests for this website, but this website had not been the web application I had to test ;-). The first one selects a link and checks the page content and the second one executes a search.
Feature: TestAndWin Scenario: Click link and check page content Given I am on the page "http://testandwin.net" When I click the link "Compare Screenshots with Selenium WebDriver" Then the text "Compare screenshots implementation" is displayed Scenario: Search Given I am on the page "http://testandwin.net" When I enter the value"screenshots" in "Search ..." And I click the button "Search" Then the text "useful to compare screenshots" is displayed
The screenshot below show the test run of the second test scenario.
With this solution it was easily possible and with little effort to provide a good solution to do acceptance testing. Using the generic steps many test scenarios could be developed without repeatedly implementing source code. When developing generic steps it must be ensured that the test remains comprehensible what I would regard in this case as given.
With this procedure many tests have been developed for the web application. In some places, it was sensible to implement specific steps for the web application. In particular for the Given Steps to jump directly to a page, without having to call various steps beforehand.
The local Visual Studio installation executes the tests. In the next post I will describe the integration of the acceptance testing in the Microsoft Team Foundation Server.