Automation

Visual Regression Tools: BackstopJS

Visual Regression

In recent days, I came across mind blowing topic called ‘Visual Regression‘. There is still no proper definition of the visual regression but idea is fairly simple. In the Visual Regression world, we take screenshots of the live applications as a reference of base screenshots, and while we develop additional features we compare new screenshots with the reference or base screenshots to make sure existing functionality is not broken. If screenshots differ then visual regression testing tools warn you about the changes. There will be more in depth discussion in the #LDNSE 12 London Selenium Meetup at Huddle. Keep an eye on the meetup ‘Visual Regression : PhantomCSS and Selenium“. Details here

LDNSE #12 Visual Regression : Selenium and PhantomCSS @Huddle

Wednesday, Mar 18, 2015, 7:00 PM

Huddle
1 Braham Street, Maersk House London, E1 8EP London, GB

100 Selenium users Went

Hello Selenium Users,Hope you are keeping well. I am glad to announce our first meetup in 2015 on 18th March.  Huddle is sponsoring our first meetup in 2015. Thanks team ‘Huddle’.We will have three interesting talk lined up at the moment. Two from Huddle itself and one speaker is travelling all the way from Israel. Details below.Follow event on …

Check out this Meetup →

Visual regression testing sometimes called ‘CSS Regression Testing. There are many tools available now to perform visual regression and it’s challenging to pick the correct tool based on your project need.  Visual regression tools works very well to test web and responsive layouts. The visual regression testing tools mainly uses following libraries to capture screenshots

Visual Regression Options

Currently there are lots of options to perform CSS regression. Let’s have brief look at some of the popular visual regression tools

  • PhantomCSS

PhantomCSS is a very popular CSS regression testing framework developed at @Huddle by James Cryer. PhantomCSS can be scripted with CasperJS.

  • BackstopJS

BackstopJS is visual regression tool developed by Garris. You can find step by step tutorial of BackstopJS on CSS Tricks site but since them there are many updates in the BackstopJS so we will cover latest setup in this post.

  • Wraith

Wraith is another screenshot comparison tool developed by the developers at BBC News. The detailed documentation of the Wraith found here

  • Success

Success is another nodeJS package can be used for the CSS regression testing. Now we will discuss and compare each tools but in this post we will discuss how to setup BackstopJS from Scratch.

BackstopJS

Although detailed step by step tutorial on BackstopJS on CSS-Tricks website, there are few changes since then as Bower dependencies are deprecated.

Installation

Now that, we assume that we have NodeJS installed. Now create  new directory ‘visual-regression-backstopjs’

Now, let’s create a package.json file with all our dependencies for the BackstopJS

Let’s install all these packages

This will create ‘node_modules/backstopjs’ directory. Now cd into that directory and install node dependencies

 Generate Config file

Now we can generate backstopjs.json config file.

This will create ‘backstop.json’ file in the project We can modify this accordingly. It will look like this

Generate Baseline Screenshots

Now let’s create baseline images by running

This will generate base images to compare against. The images will be in the ‘node_modules/backstopjs/bitmaps_reference’ directory

reference

Generate Test Screenshots

Now that, we have generated reference screenshots to compare against, let’s generate new screenshot and compare both

This will test latest screenshots against the refence screnshots.The newly created images will be in the ‘node_modules/backstopjs/bitmaps_test’ directory

test

Once it finishes test. Google chrome will pop up with result.

pass_backstopjs

When it fails, it shows the screenshot diff.

GitHub Repo  : visual-regresion-backstopjs

 

This way we can use backstopJS as a visual regression testing tool.

  • Yuriy Gerasimov

    Check our SaaS platform for visual regression testing http://backtrac.io. It allows to start testing your site without any need to set up your own system.