Testing Vim iOS Mobile App with Cucumber & Calabash

Mobile Test Automation

Mobile testing is a real challenge when it comes to automation because of variety of the platforms, versions and interfaces. Let’s say, iOS Apps, We got iPhones, iPads and different versions of them. Android devices are even more complex. Mobile test automation is a emerging topic in recent days and selecting automation libraries takes lot of research and investigation of the variety of test automation tools. In this tutorial, I will explain how I set up Calabash, Cucumber test automation framework on Mac OSX to test Vim iOS application.

Calabash: Mobile Test Automation Library

Calabash is a mobile test automation technology which currently supports iOS and Android Apps. Calabash is maintained by company called ‘LessPainful‘. Github repositories are maintained by Karl Krukow [@karlkrukow].

I got introduced with Calabash while attending Tech Talk by Moncef Belyamani at AOL. He has worked on various AOL mobile apps automation by using combination of  Calabash and Cucumber. Watch his talk at Mobile Testing Summit last year on ‘Calabash: Tips and Tricks‘. By reading his blog post ‘iOS Automated Testing with Calabash, Cucumber and Ruby’, I have decided to dive directly in to Calabash without spending time on evaluating other tools. Calabash seems like awesome option for mobile test automation.

Calabash currently support Cucumber which is BDD tool for the Ruby application. We look forward Calabash support for Behat as well in the near future.

Vim iOS Application

There are number of open-source iOS Apps available here but I have selected Vim iOS application. It’s because Vim is different than other application because of it’s interface and usage. It doesn’t have Buttons, Fields, Placeholders or links. I thought, it would be challenging to test text editor like Vim on mobile devices.

You can find source code on GitHub here. You need to setup Vim app as a Xcode project. It’s simple, from your Xcode, Goto File->Open->Vim[your_project].

Get Started with Calabash

I asume that, you have already setup development environment on your Mac by using Moncef’s blog. Basically, you need to install following tools and services.

  • Xcode with Command line Tools 
  • Various versions iOS Simulators. [ It comes with Xcode]
  • Homebrew [Package Management ]
  • Git
  • RVM and Ruby gems
  • Cucumber

I have already setup all these things on my Mac OSX. Again I would strongly recommend Moncef’s blog to do this.

Download Vim iOS Source-Code

Once you done with all the necessary setup, you need source code for your application. You can clone Github repository directly into your Mac

$ git clone https://github.com/applidium/Vim

Now, it would be an awesome idea to separate RVM for Calabash project from other Ruby application.

$ cd Vim/
$ rvm --rvmrc --create use 1.9.3@calabash

Install Calabash-Cucumber

Now, It’s time to install Calabash-Cucumber gems into our Vim directory & setup Calabash framework. You need to run following commands in sequence

$ gem install calabash-cucumber
$ calabash-ios setup
$ calabash-ios gen

These commands will setup Calabash Framework and create sample Cucumber feature file into your project. Here is screenshot.

calabash-cucumber

We need to hit return after this step to continue installation.

Setup Xcode

You can setup Xcode project manually by following step by step instructions on Moncef’s blog but I think, you can still use ‘Fast Track‘ which will do most of the setup thing automatically and then you can perform checks if setup is right or wrong.

In order to do that, you need to run ‘cucumber’ command which will fail complaining about Xcode setup but it will perform most of the task mentioned in the ‘Manual Setup’

$cucumber

Now, here are some important checks, you need to perform on Xcode in order to confirm status of your Xcode setup

Ensure Scheme [-cal]

First thing, you need to ensure that, new scheme is automatically generated. This duplicated project is called ‘Vim-cal’. You will see screen something like this:

XcodeVim-Cal

 

Check Build Setting: Product Name

In your Xcode, Go to Build Settings and Search for ” Product name”. Make sure result id ‘Vim-cal’

productName

 

 Check Calabash Framework in the App

You can simply ‘ls’ into your project and see if ‘Calabash-Framework’ directory is available.

Shashi-MacBook-Pro:Vim user$ ls
Contents               README_bindos.txt      Users                  runtime
Contents.info          README_dos.txt         Vim.info               runtime.info
Filelist               README_extra.txt       Vim.xcodeproj          
Makefile               README_mac.txt         Xxd.info      
README.txt             README_ole.txt         calabash.framework      src
README.txt.info        README_os2.txt         configure              src.info
README_ami.txt         README_os390.txt       csdpmi4b.zip           uninstal.txt
README_ami.txt.info    README_src.txt         farsi                  vimdir.info
README_amibin.txt      README_srcdos.txt      features               vimtutor.bat
README_amibin.txt.info README_unix.txt        libs                   vimtutor.com
README_amisrc.txt      README_vms.txt         nsis
README_amisrc.txt.info README_w32s.txt        pixmaps

 

Ensure Link Binary with CFNetwork.framework.

Go to ‘Build Phase’ and expand ‘Link Binary with Libraries’ You have to make sure you got ‘ CFNetwork.framework’ library.

LinkBinary

 

 Ensure Other Linker Flag

Go to ‘Build Seting’ and Search for ‘ Other Linker’. You should see flag is pointing to ‘Calabash framework’

OtherLinkerFlag

 

Ensure Accessibility Turned ON.

In order to perform some functions on iOS app you need to turn, Accessibility ON feature ON. In order to do that, you need to RUN Vim-Cal App. Once emulator is launched, Check the ‘Output’ by clicking on the ‘View’ in the top right corner.

Vim App output

 

Go to ‘Settings->General->Accessibility” and toggle ‘ Accessibility Inspector ‘ON’. You will see something like this.

Accessibility Inspector

 

Calabash-Cucumber

Now that, we have done with all the setup. It’s time to execute Cucumber to see the auto generated feature is passed.

$cucumber

Now, you will see auto generated feature is passed and screenshot is saved in the project directory.

cucumberPassed

 

Add Another Feature

Now , we will add another feature into out Vim project.

$ sudo vim visual_mode.feature
Feature: Test visual mode of the Vim editor
  In order to use Vim 
  As a Vim user
  I want to enter on the visual mode 

Scenario: Test visual mode 
    Given I am on the Welcome Screen
    When I swipe left
    And I wait for 2.3 seconds
    Then I don't see the "VIM - Vi IMproved"

This feature is using predefined Calabash Steps so that we don’t need to write implementation code. Now we will run Cucumber to see all steps passed. Watch the video demo below.

What Next?

In this post, we have seen testing look and feel of Vim using predefined Calabash Steps. In the future post, we will see

  • Using Calabash Custom Steps & Ruby Api’s to test Vim iOS app 
  • Testing all features of the Vim like insert mode, copy, syntax etc etc.

I know, it will be challenge, you can join me to contribute to this project. Please feel free to fork Github and Contribute.

GitHub : Calabash-Cucumber-Vim

Conclusion

Calabash is a awesome automation framework for testing iOS and Android apps by using Cucumber. Calabash helps QA, Developers to write robust automated test suits to test mobile App. Again, we look forward, Calabash support for the Behat.

Happy Mobile Testing !!

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>