BDD

Faster BDD: Parallelising Cucumber and re-running failed scenarios

Parallelising Cucumber

The ‘parallel_test‘ is a awesome rubygem to speedup your Rspec, Cucumber and Test Unit scenarios. Parallel _test also allows us to run Cucumber into different process and speedup the entire execution. In my previous blog post ‘speed up BDD with parallel cucumber‘ we have seen the basic setup of parallel-test, scenario execution and reports aggregation.

In this post, we will see how to handle flacky scenarios by re-running them with parallel cucumber.

Source Code

Source Code of this demo is available on GitHub ‘ parallel-cucumber-failed-rerun

 

Setup

Inside cucumber.yml

if you got existing project add this to your cucumber.yml report logger as a new profile

 

This will created ‘cucumber_failures.log’ with all failed scenario which we can use it for rerunning.

You can also add another profile to generate html reports of each cucumber process.

Run features in parallel

Now, you can run your features in parallel using that profile

 

This will run your entire features into 10 different processes and logs failed scenarios in the ‘cucumber_failures.log’ file.

Re-run failed features

Now that, we got ‘cucumber_failures.log’ file with all failed scenarios. We can use that file to re-run failed cucumber scenarios. like this

 

Rakefile

You can put all things into one Rakefile i.e deleting reports, running in a parallel and re-running failed scenarios.

rakefile

 

Rakefile has 4 different tasks to perform different cucumber operations

cleanup

This task delete old html reports and cucumber failures log file and create new one for fresh execution.

parallel_cucumber

This task runs features in parallel and create ‘cucumber_failures.log’ file with failed scenarios.

rerun

This task will rerurn only failed scenarios.

test

This task will combine everything in one.

 Using Demo Project

You can start from simple demo on Github

 

You can watch video demo on Youtube here

 

 

  • Sam Eeles

    Can you re run the fails in parallel?

    • @sameeles:disqus Yes. I don’t think so Parallel test has rerun formatter but I think it is terribly idea to run failed tests in parallel as you probably wouldn’t expect many failures in your suite if thats the case then its time to refactor suite 🙂 Thanks for pointing this I would try to add re-run formatter of parallel cucumber if time permits 🙂

  • Ian Hamilton

    Hi Shashikant, parralel_test is great, I’ve used it in the past. The only issue I faced is when using cucumber reports jenkins plugin – my manager wanted regular reports. I found it incredibly difficult to merge the reports into one (from the 10 browser sessions). Did you manage to get around this?

    • Nagaraj

      Hi Ian, Even I have the same requirement, please let me know if you find any solution

      • Alejandro

        The best solution I was able to come up on my previous job, was to create a separate results repository (using MySQL DB with phpMyAdmin) and generate my own reports. It is a bit time consuming to set up but the advantage is that you can also keep historical data for other type of reports

    • patricio