Pros and Cons of using Xcode Server for iOS Continuous Integration

Shashikant Jagtap

Continuous Integration a.k.a. CI is an integral part of iOS development process which gives early feedback when something breaks during application development. In an agile application development, producing working app is not only writing good code but also setting up infrastructure to deliver it continuously. The process of Continuous Integration and automating the build can help to achieve continuous delivery.  There are various continuous integration tools available in the market which can be used in the iOS team. The CI server solution can be self hosted or cloud. You can get whole list of CI servers here with recommendations. In the iOS development we have few challenging options while selecting self hosted CI server

If you are looking for cloud solutions then there are some promising services like

There are multiple tools so challenge is which one to choose and evaluate why one one is better than other. Cloud CI Servers are easy to setup, managed by other companies and they just works if configured correctly. The other things is when using cloud CI solutions, we don’t need someone to manage servers. However, you have to be careful of various things like privacy, cost, ease of use, availability and platform support while using cloud based solutions. Choosing Self hosted vs Cloud CI has it’s own pros and cons, you should consider what is best for your team. Xcode Server seems to be easy options for the self hosted CI as it is managed by Apple and deeply integrated with Xcode. In this post, let’s see what are the pros and cons of using Xcode Server for the iOS Continuous integration.

Brief Introduction to Xcode Server

Xcode Server is a continuous integration platform developed by Apple for analysing, building and testing and archiving the iOS and macOS apps. One of the painful part in the iOS development is Certificates and Provisioning profiles. It became more hectic when we want to setup Certificates and profiles on CI server. The latest version of Xcode Server solved lots of issues with certificated and profiles. Xcode Server is very tightly coupled with Xcode so it become so painless to setup and use Xcode Sever to perform Continuous Integration. Apple has very comprehensive documetation on Xcode Server setup for CI however I wrote short tutorial to setup latest macOS Server to run Xcode bots which is also published on DZone. You can refer those guides to get started with Xcode Server. Xcode Server has following features

  • Free or Cheap Solution
  • Self hosted and Deeply Integrated with Xcode
  • Device testing
  • Over the Air (OTA) installation
  • Managed and Maintained by Apple

02server_starting

Xcode Server has all these features but some people struggled a lot to get Xcode Server working properly. Let’s see what are the pros and cons of using Xcode Server

Xcode Server – Pros

Xcode Server is only designed to work with  Apple operating systems and stand out from other self-hosted CI solutions like Jenkins, TeamCity because of it’s managed by Apple. The other options like Jenkins, TeamCity are managed by the third party companies and mostly written in Java or some other programming language. Xcode Server has improved a lot since it’s released first time with Xcode 5. Latest version of macOS Server and Xcode Service has all the features needed to perform efficient continuous integration.

  • Easy to setup

As mentioned in the Apple’s guide to Continuous Integration, Xcode Server can be setup within few clicks if you have a Mac/Mac mini and macOS Server. in my previous post, on setting up CI with Xcode Server, I wrote down some basic steps to get up and running with Xcode Server and Xcode bot creation. This article has been published on DZone as well. Any iOS/macOS engineer can set that up within few hours.

  • Cheap

It’s almost free if you have paid Apple Developer account. You can get the macOS Server by redeem your code. It will  cost just £14.99 if you are not a Apple developer. This cost if comparatively low as well as it’s just one time investment.

  • Deep Integration with Xcode

The Xcode Bots are directly integrated into the real Xcode, not any Xcode plug-in and tied to the source code and schemes.  Xcode Server uses Xcode server from the macOS Server. It’s real Xcode running on server so it’s deeply integrated CI Server with Xcode. Developers can see all the test reports, code coverage reports straight into Xcode running on development Mac. The developers can create, edit and manage Xcode bot from the local machine using Xcode. There is no need to go to Web interface to manage CI jobs however in the Jenkins or TeamCity, we nee to visit web interface to manage CI jobs.

  • Painless Certificates and Provisioning Profiles Setup

One of the painful thing in iOS Development is  setting up Certificates and Provisioning Profiles.  Since, Apple introduced new way of code signing in WWDC 2016 in the talk ‘What’s new code signing‘, the process become so easy and painless with Xcode 8. While using it Xcode Server, we need to copy all the Certificates and Profiles to the ‘/Library/Developer/XcodeServer/Certificates’ and ‘/Library/Developer/XcodeServer/ProvisioningProfiles’ directories and we are done.

 

  • In-Built Code Coverage

Apple introduced this nice feature of  generating code coverage in WWDC 2015 in the talk ‘Continuous Integration and Code Coverage in Xcode‘ . The process of generating code coverage is easy, only things we need to do is t’tick’ the box while creating Xcode bot. The developer don’t need to use any other third party tools like ‘Fastlane‘ or ‘Xcov‘ or something else to configure code coverage.

08buildconfig

Now that, code coverage will be generated for the for the each integration. The Cool things is it shows code coverage file by file so that we know that where to add unit tests.

  • In-built Real Device Testing

Xcode Server supports testing in the simulators as well as in the real devices. It has built in ‘Test” service as shown in the above image so it reduces burden of configuring ‘Fastlane Scan‘ or scripting ‘xcodebuild‘ to run our unit and UI tests with XCTest. Once devices like iPhone, iPad are attached to the server then they start appearing in the ‘Devices” tab so that we can select then for our app testing.

  • Distributed Test Execution without building

Previously, there was ned to build an app before testing but Xcode 8 introduced nice feature called ‘ Build for Testing’ in the talk ‘ Advance Testing and Continuous Integration‘ which generate ‘xctestrun’ file in the Derived Data which can be used to run tests without building using latest ‘xcodebuild ‘ so that we can share this file with multiple Xcode Servers to enable parallel and distributed testing.

xcodebuild_build_for_testing

  • Build Hosting & OTA Installation

Xcode Server can ‘Archive’ and host a build which on Xcode Server machine. The hosted build can be distributed to QA and Product People to have a look. The build can be downloaded using Server address in an browser like Safari from the devices. There will an option to ‘Install’ build on the devices. This reduces burden os using a build hosting service like “Crashlytics” or ‘HockyApp” as test users can download build straight from Xcode Server.

  • Rest API & Environmental Variables

There is an option to use Xcode Server API‘s and Environmental variables so that we can script Xcode Server according to our project need. We can use Xcode Server environmental variables in the pre-integration or post-integration scripts in the ‘Trigger” sections. We have an ability to define our own environmental variables.

  • Customisable Triggers

Xcode Server gives us ability to run bash scripts i.e triggers before integration and after integration. Triggers can be extremely useful for setting up dependencies e.g CocoaPods, Carthage, or Swift Packages, before running build as part of the ‘pre-integration’ triggers. Xcode Server also has ‘post integration triggers’ which can be used torun other scripts to do something or send email notifications.

  • Less Fastlane

Xcode Server eliminated need to use lot of Fastlane tools as most go the things are pre-configured. Fastlane is a nice tool build on top of Apple’s command line tools for easy configuration of complex scripts. It automated iOS Development tasks using multiple tools to enable continuous delivery. Xcode Server eliminated need of following fast lane tools

  • Scan – Testing in inbuilt with Xcode Server
  • Xcov – Code Coverage is inbuilt
  • Gym – Xcode Server can build app
  • Sigh/match – Code signing become much easier in Xcode 8

However, some of the fast lane tools like pilot, deliver, snapshot are still useful but main tasks are automated by Xcode Server.

  • Physical Access to macOS Server

Xcode Server being an self-hosted CI solution, we have physical access to machine and macOS Server installed on the machine. We can tweak Xcode Service as per our need if required. We can have fun control on the CI server with customisation option.

Xcode Server – Cons

As we have seen that, Xcode Server has lot of useful things but there are some disadvantages using Xcode Server. The most of the previous old pain of code signing and certificates has been removed in the  latest Xcode. There are still some issues whiles using Xcode Server, let’s cover them

  • No Pull Request Support

Assuming most of us are using Github as code hosting service, we need to create pull request before we merge code in the main branch. Xcode Server support integration on different Github branches but it still doesn’t support testing of pull requests, which is most painful part of app development. In order to enable pull requests testing, we have write our own Github web hook and generate Xcode Bots using Xcode Sever API Or we have too open-source tools like ‘Buildasaur‘ which require additional setup.

  • Build Never finished

Some developers complained that Xcode Bot kept running forever if something goes wrong and debugging is extremely difficult. This problems occurs when there is issue with code signing.

  • Extra management of Servers and Bots

Xcode Server being an Self hosted CI solution, it might not run on its own eve we configured it correctly. Someone need to take care of machines and Servers to make sure everything this running smoothly. It gives rise to hiring an additional DevOps of TechOps resource to manage those servers or use developers time to manage servers which is costly.

  • OTA installation fails on public Domains

Some Xcode Server users complains about configuring Xcode Server with public domain. Over The Air installation doesn’t work all the time on public domain. It throws ‘Couldn’t Connect Error’.  The issue was once clicked on ‘Install” button nothing happens or “Install” button never appear. It needs additional Encryption, SSL Cert and port setup to get that properly working. This post explained pain of using Xcode Server

  • No Cross Platform Support

Xcode Server is designed to work only with Apple operating system so there isn’t any possibility to use those server machine for any other platform like Android.

  • Not Suitable for the distributed Teams

Xcode Server can be used effectively with small iOS teams but if you have big organisation and want to share Xcode Server all over the distributed teams then it probably hectic to manage. It requires additional setup for domains and networking.

Conclusion

Xcode Server is a valid choice for Continuos Integration solution for iOS app as it’s maintained by Apple which has inbuilt static analysis, testing, building, archiving and reporting mechanism to reduce burden on developers however we need to consider cost of managing CI Server machines.

What are your experiences with Xcode Server and do you feel it will be the number one choice for app developers? Weigh in with a comment below!