Mobile Test Automation

Mobile testing is a real challenge when it comes to automation because of variety of the platforms, versions and interfaces. Let’s say, iOS Apps, We got iPhones, iPads and different versions of them. Android devices are even more complex. Mobile test automation is a emerging topic in recent days and selecting automation libraries takes lot of research and investigation of the variety of test automation tools. In this tutorial, I will explain how I set up Calabash, Cucumber test automation framework on Mac OSX to test Vim iOS application.

Calabash: Mobile Test Automation Library

Calabash is a mobile test automation technology which currently supports iOS and Android Apps. Calabash is maintained by company called ‘LessPainful‘. Github repositories are maintained by Karl Krukow [@karlkrukow].

I got introduced with Calabash while attending Tech Talk by Moncef Belyamani at AOL. He has worked on various AOL mobile apps automation by using combination of  Calabash and Cucumber. Watch his talk at Mobile Testing Summit last year on ‘Calabash: Tips and Tricks‘. By reading his blog post ‘iOS Automated Testing with Calabash, Cucumber and Ruby’, I have decided to dive directly in to Calabash without spending time on evaluating other tools. Calabash seems like awesome option for mobile test automation.

Calabash currently support Cucumber which is BDD tool for the Ruby application. We look forward Calabash support for Behat as well in the near future.

Vim iOS Application

There are number of open-source iOS Apps available here but I have selected Vim iOS application. It’s because Vim is different than other application because of it’s interface and usage. It doesn’t have Buttons, Fields, Placeholders or links. I thought, it would be challenging to test text editor like Vim on mobile devices.

You can find source code on GitHub here. You need to setup Vim app as a Xcode project. It’s simple, from your Xcode, Goto File->Open->Vim[your_project].

Get Started with Calabash

I asume that, you have already setup development environment on your Mac by using Moncef’s blog. Basically, you need to install following tools and services.

• Xcode with Command line Tools 
• Various versions iOS Simulators. [ It comes with Xcode]
• Homebrew [Package Management ]
• Git
• RVM and Ruby gems
• Cucumber

I have already setup all these things on my Mac OSX. Again I would strongly recommend Moncef’s blog to do this.

Download Vim iOS Source-Code

Once you done with all the necessary setup, you need source code for your application. You can clone Github repository directly into your Mac

Now, it would be an awesome idea to separate RVM for Calabash project from other Ruby application.

Install Calabash-Cucumber

Now, It’s time to install Calabash-Cucumber gems into our Vim directory & setup Calabash framework. You need to run following commands in sequence

These commands will setup Calabash Framework and create sample Cucumber feature file into your project. Here is screenshot.

We need to hit return after this step to continue installation.

Setup Xcode

You can setup Xcode project manually by following step by step instructions on Moncef’s blog but I think, you can still use ‘Fast Track‘ which will do most of the setup thing automatically and then you can perform checks if setup is right or wrong.

In order to do that, you need to run ‘cucumber’ command which will fail complaining about Xcode setup but it will perform most of the task mentioned in the ‘Manual Setup’

Now, here are some important checks, you need to perform on Xcode in order to confirm status of your Xcode setup

Ensure Scheme [-cal]

First thing, you need to ensure that, new scheme is automatically generated. This duplicated project is called ‘Vim-cal’. You will see screen something like this:

 

Check Build Setting: Product Name

In your Xcode, Go to Build Settings and Search for ” Product name”. Make sure result id ‘Vim-cal’

 

 Check Calabash Framework in the App

You can simply ‘ls’ into your project and see if ‘Calabash-Framework’ directory is available.

 

Ensure Link Binary with CFNetwork.framework.

Go to ‘Build Phase’ and expand ‘Link Binary with Libraries’ You have to make sure you got ‘ CFNetwork.framework’ library.

 

 Ensure Other Linker Flag

Go to ‘Build Seting’ and Search for ‘ Other Linker’. You should see flag is pointing to ‘Calabash framework’

 

Ensure Accessibility Turned ON.

In order to perform some functions on iOS app you need to turn, Accessibility ON feature ON. In order to do that, you need to RUN Vim-Cal App. Once emulator is launched, Check the ‘Output’ by clicking on the ‘View’ in the top right corner.

 

Go to ‘Settings->General->Accessibility” and toggle ‘ Accessibility Inspector ‘ON’. You will see something like this.

 

Calabash-Cucumber

Now that, we have done with all the setup. It’s time to execute Cucumber to see the auto generated feature is passed.

Now, you will see auto generated feature is passed and screenshot is saved in the project directory.

 

Add Another Feature

Now , we will add another feature into out Vim project.

This feature is using predefined Calabash Steps so that we don’t need to write implementation code. Now we will run Cucumber to see all steps passed. Watch the video demo below.

What Next?

In this post, we have seen testing look and feel of Vim using predefined Calabash Steps. In the future post, we will see

• Using Calabash Custom Steps & Ruby Api’s to test Vim iOS app 
• Testing all features of the Vim like insert mode, copy, syntax etc etc.

I know, it will be challenge, you can join me to contribute to this project. Please feel free to fork Github and Contribute.

GitHub : Calabash-Cucumber-Vim

Conclusion

Calabash is a awesome automation framework for testing iOS and Android apps by using Cucumber. Calabash helps QA, Developers to write robust automated test suits to test mobile App. Again, we look forward, Calabash support for the Behat.

Happy Mobile Testing !!