At WWDC 2016, there was an awesome talk on ‘Advance Testing and Continuous Integration‘ which mentioned lot of new features in the XCTest Framework, Xcode-Sever and xcodebuild command line tool. We can use those features to speed up the iOS Continuous Integration process.

Current iOS CI Limitation

Before Xcode 8, we have to build and compile and application before running Unit Tests and UI tests on CI which is duplication of running task. The distributed testing is time consuming as well as we have build and compile the source code on every machine. Actually building and Compiling takes lot of build time on CI Server.

Xcode 8 and xcodebuild Features

Xcode 8 release bought couple of new features in ‘xcodebuild’ which can add lot of value to iOS development and testing process.

‘xcodebuild‘ is the command line tool to build, run and execute our application from command line. This is used in the Xcode server. Xcode 8 now has some improvements in the xcodebuild command line tools.

Build For Testing

The xcodebuild now has ‘build-for-testing’ option, it takes workspace scheme and destination as usual but on top of that it will create ‘XCTESTRUN’ file. We just need to execute ‘build-for-testing’ action

This should build our app for the testing and create xctestrun  file in the DerivedData.

Test without Building

The xcodebuid also has another option called ‘test-without-building’ where we don’t need to provide workspace instead we specify the XCTESTRUN file which will inject that file and runs all the tests.

This feature can be highly useful for the distributed testing as we can created XCTESTRUN file at one machine and distributed to other test specific machine. In order to use it we can specify this option to ru tests without building

This time it won’t build and application.

Only-Testing/Skip-Testing

Suppose, you don’t want to run your unit test. Xcodebuild now has two new testing options

• –only-testing : Include Test suites
• –skip-testing : Exclude Test suits

Speeding up the XCTest on CI

Using the features mentioned above, we can speed up the test execution as well as save the build time. Let’s jump into the Xcode and see how to do that

Setup Xcode Schemes

Create a sample iOS app in the Xcode and name it XCTestRun  with unit and UI tests included. You may also want to save this project as ‘workspace’ as most of the real projects are saved as ‘.workspace’. You can do that using File->Save as Workspace  Xcode option. You can also create separate schemes to run Unit Tests and UI Tests. Thats it

Build For Testing

Let’s now try to build this project using new features of xcodebuild . Let’s first use  build-for-testing  feature to prepare our application for testing.

This will build an application for testing and creates XCTestRun_iphonesimulator10.2-x86_64.xctestrun file inside the  build/Build/Products  directory. We can now use this file to run tests without building.

Test Without Building using xctestrun

We can now use this file to run tests without building using test-without-building  options, we can also use -only-testing  and - skip-testing

We can execute those script as part of CI or we can use tools like Fastlane mentioned below.

Using Fastlane

Fastlane has set of tools to use for iOS automation as well as loads of actions to perform tasks. In order to setup Fastlane we need to create  Fastlane directory and  Fastfile inside that directory.

Fastlane has xcodebuild action as well scan tool but unfortunately none of support build-for-testing  and test-without-building  natively without additional effort at the time writing this post. We need to write custom action to perform those steps. I have written custom Fastlane Action called ‘xctestrun‘ which takes few options to run the build and test without building.

Now that, we have custom action, let’s us that in the  Fastfile by adding the following code.

This fast file has separate lanes for ‘build-for-testing’ , running Unit tests and UI tests also note that we are using custom action ‘xctestrun’

We can now run fastlane lane ‘build-for testing’ to build and app for testing.

This will build an app for testing

Now, our app has been built for testing, let’s execute Unit tests using ‘unit test’ lane

This will execute only ‘unittest’ scheme.

Similarly, we can execute UI tests

This will execute only UI Tests.

We don’t need to build or compile an app to run unit and UI tests.

Source Code

The Source code for this demo is available on Github ‘XCTestRun‘ You can try it yourself.

Conclusion

Using new features of ‘xcodebuild’ we can save build time of iOS Continuous Integration process and achieve distributed testing by passing xctestrun file to multiple test machines. This will definitely speed up iOS Continuous Integration Process.