One of the goals of having Continuous Integration setup is to find the issues or bugs early in the application development lifecycle. Sooner we detect bugs, cheaper to fix them. The earliest stage that developer can find the issues in the apps is while writing the code or running the code inside the Xcode. In WWDC 2017, Apple has announced Xcode 9 with loads of enhancements in the debugging and detecting issues at runtimes. Xcode debugger already highlights most of the issues like compiler errors, compiler warnings, test failures while running the application or tests. However, Apple has introduced another category of the tools called Xcode Runtime tools to find the issues while running the code which can be enabled from local Xcode to help engineers to fix issues quickly. However, there are some lazy engineers still exists in the world who might check-in the source code ignoring these issues. In order to catch those issues, we can enable Xcode runtime tools on CI servers. In this post, we will find out what those tools are and how to activate them on iOS Continuous server.
Xcode Runtime Tools
Xcode has four different types of runtime tools also known as code diagnostic tools to find the issues. These tools are
- Address Sanitizer
- Thread Sanitizer
- Main Thread Checker
- Undefined Behaviour Sanitizer
Apple has brief documentation of all these tools on the official documentation page of code diagnostic. For this post, I have created a demo single view iOS app called Code-Diagnostics which has default unit and UI tests. Now, let’s have a quick look at the each tool and how to enable then locally as well on CI server.
Address Sanitizer
Address Sanitizer a.k.a ASan reports the memory related issues like memory corruption and other memory-related security vulnerability issues. You can read more about the Address Sanitizer here. We can enable address sanitizer from local Xcode by editing the scheme–> Select Action –> Diagnostics Tab and tick Address Sanitizer.
Enable Asan on CI
In order to, enable Asan on Continuous Integration server, we have the parameter that can be passed to xcodebuild command. The parameter is -enableAddressSanitizer YES that can be used for the build, test actions. We can run our tests using the following command on CI server.
|
1 |
$ xcodebuild test -scheme Code-Diagnostics -destination 'platform=iOS Simulator,name=iPhone 6,OS=11.0' -enableAddressSanitizer YES |
Thread Sanitizer
Thread Sanitizer a.k.a TSan detects data races problems. It also detects other thread related issues like thread leaks. There is detailed documentation about Thread Sanitizer on Apple’s official documentation here. We can enable the thread sanitizer same way as of address sanitizer but the important things to note is
Address Sanitizer and Thread Sanitizer cannot be turned ON at the same time
Enabling TSan on CI Server
In order to, enable TSan on Continuous Integration server, we have the parameter that can be passed to xcodebuild command. The parameter is -enableThreadSanitizer YES that can be used for the build, test actions. We can run our tests using the following command on CI server.
|
1 |
$ xcodebuild test -scheme Code-Diagnostics -destination 'platform=iOS Simulator,name=iPhone 6,OS=11.0' -enableThreadSanitizer YES |
Undefined Behaviour Sanitizer
Undefines Behaviour Sanitizer a.k.a UBSan detects undefined behaviours in the code at runtime. The behaviours like dividing by zero, loading memory from a misaligned pointer, or dereferencing a null pointer. You can read more about UBSan here. We can enable UBSan in Xcode same way as we have done ASan but select the ‘Undefined Behaviour Sanitizer` checkbox.
Enabling UBSan on CI Server
In order to, enable UBSan on Continuous Integration server, we have the parameter that can be passed to xcodebuild command. The parameter is -enableUndefinedBehaviorSanitizer YES that can be used for the build, test actions. We can run our tests using the following command on CI server.
|
1 |
$ xcodebuild test -scheme Code-Diagnostics -destination 'platform=iOS Simulator,name=iPhone 6,OS=11.0' -enableUndefinedBehaviorSanitizer YES |
Main Thread Checker
The Main Thread Checker is new tool launched with Xcode 9 which detects the invalid use of Apple’s frameworks like UIKit, AppKit etc that supposed to be used from main thread but accidentally used in the background thread. The effect of the invalid usage can result in missed UI updates, visual defects, data corruption, and crashes. You can read more about the Main Thread Checker here.
Enabling Main Thread Checker on CI Server
The Main thread checker is automatically enabled with Xcode debugger so we don’t need to explicitly enable Main Thread Checker. However, disabling it on CI server is unknown at the moment. If anyone knows how to do this please, let me know.
Setup Continuous Integration
Now that, we have seen how to enable to Xcode runtime tools for the CI server. We will setup CI with Travis for our demo app, Code-Diagnostics. We will add configuration file .travis.yml with the following content.
|
1 2 3 4 5 6 7 8 9 |
language: objective-c osx_image: xcode9 install: true script: - echo "Enabling Address Sanitizer to detect" - xcodebuild test -scheme Code-Diagnostics -destination 'platform=iOS Simulator,name=iPhone 6,OS=11.0' -enableAddressSanitizer YES - echo "Enabling Thread Sanitizer, Main Thread Checker and undefined behavior sanitizer" - xcodebuild test -scheme Code-Diagnostics -destination 'platform=iOS Simulator,name=iPhone 6,OS=11.0' -enableThreadSanitizer YES -enableUndefinedBehaviorSanitizer YES |
On TravisCI, I have enabled the project for building here
https://travis-ci.org/Shashikant86/Code-Diagnostics
We can see in the Travis log that project is building with Xcode runtime tools.
Further Information
You can get more information about them on
- WWDC 2017 Talk – Finding Bugs Using Xcode Runtime Tools
- Apple’s Official Documentation – Code Diagnostics
Conclusion
Xcode runtime tools are very useful to detect the bugs early in the development lifecycle. It would be great practice to use them all either locally or on CI server to find the bugs.