Cache Carthage to speed-up iOS Continuous Integration

There is hardly any iOS project that doesn’t have any dependency. Almost every project has to manage some sort of dependencies either internal or third party. Every major programming language has their own dependency management system. Swift has various options to manage the dependencies. Apple is working on its own Swift Package Manager since ages. Seriously, ages, iOS Developers are keen to get the official dependency manager that has been supported by Apple but it seems like the things progressing with tortoise speed. The Swift Package Manager still not supports iOS project. CocoaPods is the most commonly and traditionally used dependency management for iOS projects as developers didn’t have any other option. CocoaPods is a magic, black magic, superpower, incantation that f*cks up your Xcode project to get things working. It builds dependencies, creates a workspace, executes some scripts and my other things.  CocoaPods works well but very few developers have understood that what has been happened under the hood when they run pod install command. Everyone is happy if everything works, but when pod install fails then everybody gets panic. Nobody knows how to fix that, it takes hours, days or weeks to sort out that mess created by CocoaPods. Love it or Hate it, either way, you have to use it.

Birth Of Carthage

Somone understood the pain that iOS Developers are having with CoacoPods and produced a baby called Carthage to be an alternative for the CocoaPods. Carthage only works for the dynamic frameworks and gives developers an ability to take full control of what they are doing with Xcode project. It simply builds the frameworks and developers have to manually link those frameworks to respective build phases of the target. Carthage works great but it has few drawbacks

  • Carthage only works with dynamic frameworks
  • Carthage has small community of developer contributing to the project
  • There are few manual steps involved while using Carthage
  • The major drawback was, it is so slow especially on CI server. Carthage Checkout and Build each framework for every build. This is not definitely great approach for CI as builds will be incredibly slow and lengthy. There was no clean solution to cache the Carthage/  directory if nothing changed. In this post, we will see how to cache the Carthage directory on CI servers and speed up the builds.

 Carthage On CI

The most common use of Carthage is to run   carthage update  or script as part of the CI script which modifies all the dependencies to use the latest version changes Cartfile.lock  file which is not correct and causing the checkout and building of all the dependencies. This will make the build the CI builds utterly slow. So ideal way to use Carthage on CI would be

  • Check that Cartfile.lock  exists or not. If this file doesn’t exist then we have to bootstrap all the dependencies using Carthage bootstrap.
  • If Cartfile.lock  exists but not changed, we don’t need to perform bootstrap or update operation and we can use contents inside Carthage directory from the previous build.
  • If Cartfile.lock  exists and changes some dependencies then we have to only bootstrap those outdated dependencies rather than all.

Now that, we know the mechanism of how to use Cartfile.lock  status to perform bootstrap or skip bootstrap. The next step would be to write a script to do that. There are lots of ways we can achieve this, depending on the environmental variables provided by your CI server. The script to use with Jenkins can look like this

The Jenkins GIT Plugin sets up that variables to compare the status of the Cartfile.lock . The full explanation can be found in this blog post here. There is an open-source tool to compare the status of Cartfile.lock called CartfileDiff which can be used to determine the changes in the Cartfile.lock if your CI server doesn’t have specific environment variables. We will see how to use CartfileDiff later in the post.

Carthage Setup

Caching the Carthage/  directory is easy on the self-hosted CI servers however the cloud-based CI servers launch fresh virtual machine for every builds. This makes caching of Carthage directory difficult. The most of the cloud based CI providers for iOS has recognised this problem and provided the mechanism to caching e.g Travis has Cache for most of the services. There might be other CI services like CircleCI might be doing the same things. Let’s see how to use TravisCI cache service with the demo project.

Let’s create a single view project in Xcode called Carthage-CI including the UI Test target. Now, we will get XCFit framework which is a small library that I wrote for BDD in the iOS project using Carthage. Create Cartfile  and add

Now run,  carthage update --platform iOS to download and build all the dependency. Add the built framework to the UI Test target manually frag and drop and run Carthage script as part of build phase as per the Carthage documentation.  Now that, we have framework inside the app, it’s time to write the script to use Carthage intelligently on CI.  Make couple of executable file in the project

Now that, we have created two files, in script/bootstrap the script we will run carthage bootstrap  with dependencies passed to that script. The  script/intelligent-bootstrap script we will do the magic of determining the Cartfile.lock  status. In the script/bootstrap file insert the following code

You may need to install CartfileDiff  tool at this stage by following the CartfileDiff README. This will bootstrap the number of dependencies passed to this script. Now, in the script/intelligent-bootstrap  file insert this code.

This script will use CartfileDiff tool to determine if Cartfile.lock  has been changed or not and execute the bootstrap accordingly. If there are no changes in the Cartfile.lock then, the bootstrap step will be skipped.

Caching Carthage on TravisCI

Now that, we have all the setup to determine the status of the Cartfile.lock file. The next thing is to configure the TravisCI to run those steps with cached Carthage directory. Create a .travis.yml  file at the root of the project and insert the following code.

Note that, we are using cache from TravisCI of Carthage/  directory. We also have to download and install  CartfileDiff tool and Fastlane to run tests for the scheme Carthage-CI. Now that, we need to enable the project on Travis to trigger the build.

Source Code and TravisCI Job

I have created the repository on Github to demonstrate what we said earlier. The repository is Carthage-CI and you can see the TravisCI job here. Look at the job history below, the latest job saved almost 3 minutes because of caching. As XCFit is a very small library, if you have bigger libraries, this approach would be a time saver.

Conclusion

Using the Caching features of the CI servers and smart strategy to perform the Carthage bootstrap,  we can reduce the build time drastically on the CI server. How are you using Carthage on CI, what are your experiences, feel free to comment below.