The Apple Worldwide Developers Conference ( WWDC 2016 ) brings us lots of improvements in softwares although there are no hardware release this time. There are many software release including iOS10, macOS Seirra, Siri on Mac, Xcode 8 etc etc there is brief summary of the what has been announced at WWDC 2016 is here on The Next Web Blog. I would recommend watching the session “Platform State of the Union” which is kind of keynote for the Developers where you can find lots of the announcement about development tools including
- Xcode 8
- Apple File System
- Swift 3 : Open-Source Status
- Swift Playground app
- Code Signing Improvements
In this post we will cover whats new in terms of the Continuous Integration and Testing.
Xcode Server and XCTest
Apple has some related sessions at previous WWDC conference. Here is list of those related sessions including
- Testing with Xcode 6 – WWDC 2014
- Continuous Integration with Xcode 6 = WWDC 2014
- UI Testing in Xcode – WWDC 2015
- CI and Code Coverage in Xcode – WWDC 2015
Unfortunately there is only one session about the the Testing and Continuous Integration this year of WWDC. You can watch full session on WWDC video on “Advanced Testing and Continuous Integration” Or click on the image below
Let’s cover what has covered in this talk as part of this post including enhancements in Xcode, Xcode Server and xcodebuild command line tools.
Whats new in XCTest
Xcode 8 now has feature of gathering crashes in the test report itself.
Testing as four characters
As mentioned in the talk, there are four stages of the testing at this stage while testing out app.
- XCTest – Test Frameworks
Used to write Unit and UI tests
- Xcode – Test harness:
used to author test, execute tests and review reports
- Xcode Server Continuous Integration:
used to schedule bot runs, generate reports and send notifications
- xcodebuild- Command Line Tool :
Used by Xcode server to build and execute tests and for custom CI system.
Zoltan then explained fundamentals of ‘testing timelines’ with two concepts
- Test hosting
Depending on the test it’s been hosed on app, In unit test test hosting is done inside the app while in the UI testing hosting is done at the Test Runner. Unit tests can have direct access to data and APIs and all test running in the same launch while UI tests have not direct access to data and APIs as it uses accessibility to interact with element so needs every test launch app for every test case.
- Test Observation
The test observation is details process of observing behaviour of the details. It can be done using XCTestObservationProtocol. This protocol has some call backs including
Bundle Will Start, Suit Will Start, Case Will Start, Case Did Finish, Case Did Fail, Bundle Will Finish
Test Crash reporting
This is brand new feature in the Xcode which will report all the crashes as part of the test report. This has following
- Improved test reporting with crash logs
- It works for both unit and UI tests
- Logs includes in the test reports
- View crashes in the Xcode Debug Navigator
What’s new in Xcode Server
Xcode Server is a continuous integration sever can be used to run Unit and UI tests. Xcode Sever has lots of improvements in the latest release which includes
- Custom environment variable
- Advance trigger editing
- Issue tracking and blame
- Upgrade integrations
- Configurable integration user
Let’s explore in the each of the feature in brief.
Custom Environment Variable
This isn’t new but Xcode 8 will now allow us set environmental variable without creation of extra schemes.
Xcode 8 will will now have two types of triggers, script triggers and email triggers. We will have now control over how to set email contains.
Issue Tracking and Blame
Xcode Server will send email to person who broke build. We cab now configure bots instead of changing code.
Xcode Server now re-integrate the your project if it fails using same revision. It also prevent blaming for the broken build.
Configurable Integration User
Xcode Server will have now it’s own use’ called ‘Xcode Server” which give you full control with password as any normal macOS user. Setting new user seems fairly easy in the Xcode Server
Some of the best practices using the Xcode Server user
- Dedicate a new user
- Avoid using administratiove accounts
- Stay logged in with Fast User Switching
- Disable Screen Lock
- Customise need such as Simulators, Networks and advanced provisioning
Whats new in xcodebuild
xcodebuild is the command line tool to build, run and execute our application from command line. This is used in the Xcode server. Xcode 8 now has some improvements in the xcodebuild command line tools.
Xcodebuild now has two new testing options
- –only-testing : Include Test suites
- –skip-testing : Exclude Test suits
Build For Testing
The xcodebuild now has ‘build-for-testing’ option, it takes workspace scheme and destination as usual but on top of that it will create ‘XCTESTRUN’ file.
Test without Building
The xcodebuid also has another option called ‘test-without-building’ where we don’t need to provide workspace instead we specify the ‘XCTESTRUN file which will inject that file and runs all the tests.
This feature can be highly useful for the distributed testing as we can created XCTESTRUN file at one machine and distributed to other test specific machine.
Those are the only updated from XCTest and Xcode Server from WWDC 2016. I will post more about performance, accessibility updates soon
Hope you enjoyed WWDC 2016 this year !!