iOS CI Olympic-Player Four: Nevercode

Shashikant Jagtap

Before diving into this article, make sure you take a look at the intro article on iOS CI Olympics game here.

In the introductory post on iOS CI Olympics, we have selected the few player and few games for this competition. We will be evaluating the player one by one on the basis of their performance in the each game. In this article, the player is Nevercode, another mobile continuous integration service.

Nevercode

In the iOS Continuous Integration Server Olympics competition, the fourth candidate selected is Nevercode. It is designed specifically for the mobile continuous integration which supports iOS and Android platforms. The clients of Nevercode include some big brands like Sainsbury’s, Toyota, Eurostar etc. The little bit of the background of Nevercode organisation

Company: Nevercode
Founded: 2015
Headquarters: London, England
Description: Continuous Integration for Mobile Pros
Founders: , Categories: Android, Enterprise Applications, iOS
Website: http://nevercode.io

In this post, I will share my first impressions of getting started with  Nevercode for my favourite pet project XCFit using the iOS CI Olympics games. So Let’s get started with our games

SignUp & Project Setup

Nevercode signup process was very straightforward. I have chosen to use my Github account to sign up which took me straight into the portal. The next step to select the GitHub repository to be built with Nevercode. The setting up project requires the access to Github repositories that we want to build on Nevercode. There will be a big plus button to add an app to build. In the project settings, we need to select an app to build

 

 

As you can see above, there are few tabs we need go through to set up our project. We can select environmental variables, drag and drop provisioning profiles and distribution certificates, select the build configurations like the branch, project schemes. We need to give Nevercode a permission to scan the project.

Once, the project is scanned we will be able to select and test scheme or project, perform some tasks and publish the reports. All the screens are very easy to go through.

Medal: Nevercode has earned the Gold medal for the project setup game

Default Build Configuration

The default build criteria for the Nevercode has to be set up by the human by selecting the project and scanning the project. Nevercode the start building the default scheme which seems to suitable for execution. The default criteria don’t always what we want to set up but we can change that from the settings tab.

Medal: Nevercode has earned Silver medal for the default build configuration game.

Build Execution Process

Nevercode starts the build as soon as the commit is pushed to the Github repositories or branches. There wasn’t any delay in starting the build. The first build I have started with selecting Master branch and Xcode 9. I have selected the XCFit-Example scheme to test for UI and unit testing. However, the build is failing with 32-bit support error, It was passing on other CI servers like Travis, BuddyBuild and Bitrise. I still don’t know why it’s failing only for Nevercode

I have to switch to test with old Xcode version so I have switched to Swift31 branch and tested XCFit on this branch.

The build execution has two tabs or links one for activity and other for logs. The activity shows an overall build status and logs displays the execution of the commands. Nevercode uses both xcpretty and xcodebuild output format.

The UI/UX design is not the greatest but we can clearly see the build execution process is very smooth.

Medal: Nevercode has again earned the Gold medal for the build execution game

Build Artefacts and Test Reporting

Once the build is executed then the build artefacts are available to download in the activity tab of the build execution process. There is  Download Artefacts button. The build artefacts for every build are also available to download from the Dashboard as well.

I couldn’t manage to execute the tests as part of the Nevercode build so can’t check test reporting feature.

Medal: Nevercode has earned the Silver medal for Build Artefacts and Test reporting game

Real Device Testing

Nevercode has lots of the simulators to run tests on but there is the inbuilt capacity to configure our XCUITests on AWS device farm. It would be much handy to have inbuilt support for third-party services support inside the CI server.

Medal: Nevercode has earned the Gold medal for Real Device Testing game.

Build Customisation

As mentioned earlier, Nevercode has few tabs to customise the build configuration. There are various tabs to do different things. As shown below

As we can see there are options to setup environment variables, code signs an app with profiles and distribution certificate, build, test, code analysis, caching of directories etc. The important thing is Nevercode can publish the build to various third-party integrations at the same time. It also has notifications and hooks as well. Nevercode is simply amazing in build customisation.

Medal: Nevercode has earned the Gold medal for Build Customisation game.

Parallelisation & Concurrent Builds

Nevercode has a lot of iOS simulators available for testing so parallelising the tests is just a matter of selecting multiple simulators. The starter plan of Nevercode has only 1 concurrent build as you spend more money,  you will get more concurrent builds. The Unicorn plan can give you up to 11 concurrent builds with parallel test execution.

Medal: Nevercode has earned the Gold medal for Parallelisation and Concurrent Builds game.

Build Pipelining

Build pipeline allows us to define stages of the build that we can run parallelly or sequentially. Nevercode doesn’t seem to have build pipeline concept but we can define the workflows to achieve build pipelining.

Medal: Nevercode has earned Silver medal for Build Pipelining game.

Infrastructure As a Code

Nevercode has all the configurations to be done using UI so there isn’t any mechanism that I am aware of which can be used to code the build configuration.

Medal: Nevercode has earned the Bronze medal for Infrastructure as Code game.

Directory Caching: CocoaPods & Carthage

CocoaPods and  Carthage are the two most used dependency manager for iOS projects. Both dependency managers check out the dependency source code and build the frameworks in Pods and Carthage directory. This is a very time-consuming process to do for every build.  We need to cache those directories to speed up the build execution. Nevercode has the great interface to cache the directories. We can cache any directory with the build source code.

We just need to enable the dependency caching and specify the directories to cache. It’s super simple with Nevercode

Medal: Nevercode has earned Gold medal forDirectory Caching game.

Code Signing & Distribution

Nevercode has a separate tab to handle the code signing for the project. The tab just asks for the provisioning profiles and distribution certificates, the rest of the stuff is handled by the Nevercode.  We have an option to select a manual or automated code signing.

We just need to drop the certificates and profiles in this tabs.

Medal: The areas that I haven’t fully tested with XCFit projects, I will be giving the Silver medals by default untested but available features.

Third Party Integrations

Nevercode has the wide variety of the third-party integrations. Just now we have seen the integration with AWS device farm to run the tests on the real devices. There are various integrations available in the publishing tab as well to publish the build artefacts to the different channels.

As you can see that, there are various platforms available to publish an app. Nevercode can publish an app to TestFlight, HockyApp, Crashlytics and Testfailry, that means every possible testing services. An engineer doesn’t need to write custom scripts to do this. On top of this, Nevercode has workflows that allow users to write custom scripts to perform a specific task before and after publishing the artefacts. Nevercode has nicely managed most of the third-party services needed to deploy your iOS app.

Medal: Nevercode has earned the Gold medal for the third party integrations game.

Speed

Nevercode was quick to start my build. Whenever the commit is pushed to Github, the build started instantly. The source code checkout and build execution process was fast too!

Medal: Nevercode has earned the Gold medal for speed.

Secret Store

Nevercode has an ability to add environmental variables to the build configuration means we can store our secrets in the form of environmental variables. There is a possibility to add/upload a file that isn’t in the source repository. This file will be used during the build execution.

We can store our secret in either way.

Medal: Nevercode has earned the Gold medal for Secret Store game.

Xcode & Apple Developer Tool Upgrades

Nevercode wasn’t quick enough to update Xcode versions. When I was using Xcode 9 GM for other CI servers like BuddyBuild, Bitrise and Travis, Nevercode was still showing Xcode 9 beta 6 for a long time. The beta Xcode images weren’t working as expected as well. Nevercode should really try hard to to get the latest updates as soon as possible.

Medal: Nevercode has earned the Bronze medal for fast Upgrades of Apple developer tools.

Open-Source Project Support

As per my understanding, Nevercode neither support open-source projects not has any free plan . Nevercode has 14 days free trial to explore the features. Nevercode should learn from other competitors and should be able to provide support for the open-source community so that users will have the opportunity to try out before they buy the services.

Update: Nevercode gets in touch with information that they have hobby plan which is completely free but it lacks some features.

Medal: Nevercode has earned the Silver medal for Open-Source Project Support.

Learning Curve

Nevercode interface is easy to learn and has inbuilt support for the code signing, build, test and deployments. This reduces the burden on engineers to write custom scripts to get the build process working. It is still possible to add custom scripts using workflow feature of Nevercode.

Medal: Nevercode has earned the Gold medal for Learning Curve.

Documentation & Support

Nevercode has detailed documentation on the developer hub covering almost all the topics. Nevercode also has a forum to discuss any issues that users are facing. On top of this, Nevercode is very active on Twitter and aware of every tweet sent to them.

Medal: Nevercode has earned the Gold medal for Documentation and Support.

Clean UI/UX

Nevercode has some great features but Nevercode UI/UX is so poor, seriously, very poor. It reminded my university project I have done in the past which was just functional but UI was poor. I am not saying without any base, here are a couple of examples. The build execution page still looks like this in my safari on macOS High Sierra.

When I was selecting the branch, Nevercode detected the Podfile and got confused and shown me warning like this in the Project area.

Nevercode needs to focus a lot on the UI/UX. I heard that new version Nevercode 2 is coming with loads of improvements. I hope some of these issues will go away.

Medal: Nevercode has earned the Bronze medal for Clean UI/UX.

Price

Nevercode has four plan at the time of writing this post. The plans are called freelance, startup, superstar and unicorn. They are different in price according to with different features

Update: Nevercode gets in touch with information that they have hobby plan which is completely free but it lacks some features. I also realised that Nevercode is the bit cheaper than other CI services. Nevercode is being upgraded to Gold now

Medal: After updates. Nevercode earned the Gold medal for the price game.

Nevercode: Pros and Cons

As of now, we have seen the features of the Nevercode in brief, now the time is to decide what’s good and not good with Nevercode. There are certainly some great features that Nevercode offers but there are some limitations as well.

Pros

The benefits of using Nevercode are as follows

  • Getting started with Nevercode is very easy and painless
  • Nevercode has easy for code signing
  • Nevercode has an ability to connect with real device testing
  • Nevercode has integration with all the major third-party services.

Cons

There are some pitfalls using Nevercode which are as follows:

  • UX is poor
  • Support for lasted SDK might not be fast.
  • GUI based manual build configuration which is hard to repeat and recreate for other projects

XCFit: Build Details

The build logs executed for the XCFit project are available here but you need to be logged into Nevercode portal.

Nevercode Medal Summary

[table id=10 /]

Nevercode medal counts looks impressive. In the rest of the Olympics competition, we will see how other players will perform. We will publish the final medal table at the end of the competition. In the next post, we will see performance of the CircleCI. Continue Reading …….