Apple recently announced Swift 3.1 development snapshot and XCode 8.3 for the developers. There are couple of handy classes added to the XCTest framework to enable Asynchronous Testing for iOS and macOS applications. In this post, we will see how we can perform asynchronous testing using XCWaiter.
Swift 3.1-Dev
Newly added classes are available in Xcode 8.3 which is currently available for the download if you have Apple Developer Account. You can get it from the Downloads section of the developer account. Xcode 8.3 needs macOS version 10.12 and above. You can download compressed XIP file which is around 4.52 GB. If you already have previous version of the Xcode then remove it or you can keep it but you have to switch between Xcode DEVLOPER_DIR. Once downloaded you can extract the file to install Xcode 8.3 beta and wait for installation of Xcode and command line tools. Once Xcode 8.3 is fully installed with all the command line tools, we can drag it into /Applications path. Now, we have to switch to the new Xcode version by running following command
|
1 |
$ sudo xcode-select --switch /Applications/Xcode-beta.app/ |
This will set new DEVELOPER_DIR and we are ready to use Xcode 8.3. Make sure you are using correct toolchain using xcrun --find swift command which will shows current tool chain you are using.
|
1 2 3 |
$ xcrun --find swift /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift |
Now, make sure you export toolchain and using correct version of Swift which is Apple Swift version 3.1-dev at the moment. You can easily do that by running following commands.
|
1 2 3 4 5 |
$ export TOOLCHAINS=swift $ swift --version Apple Swift version 3.1-dev (LLVM 40fb70e1b6, Clang 658ce8b57d, Swift d6c7fe1067) Target: x86_64-apple-macosx10.9 |
This will ensure that you are using Swift 3.1. Now we are good to try new features of XCTest Framework.
Current Waiting in XCUI Test
The XCTest framework allows developers to write unit and UI tests for iOS, macOS apps. Apple introduced Xcode UI testing in the WWDC 2015 which allow us to write UI tests within the Xcode. As part of the Xcode 8.3 release, Apple has aded couple of new class to the XCTest framework to support asynchronous testing. It means there are no handlers involved while waiting for the XCUIElement to appear. Previously, we had waitForExpectations(timeout: handler:) method combined with XCTestExpectation to test asynchronous code which looks like this:
|
1 2 3 4 |
let predicate = NSPredicate(format: "exists == 1") let query = XCUIApplication().buttons["Button"] expectationForPredicate(predicate, evaluatedWithObject: query, handler: nil) waitForExpectationsWithTimeout(3, handler: nil) |
This piece of code will wait for 3 seconds to find the button and it will fail after 3 second if it doesn’t find element. We have passed nil to handler which invokes error once timeout is reached. This cause test to fail as well error thrown is very generic and not often useful while debugging. Luckily, we now have better control over error and handlers with XCWaiter.
XCTestWaiter
The XCTest Framework now has XCTWaiter class to wait for the element. The list of the newly added classes and sub-classes are as follows:
- XCTWaiter : Takes control over to the waiting strategy.
- XCTNSPredicateExpectation: Readable expectation while writing automate tests
- XCTKVOExpectation: Readable expectation while writing automate tests
- XCTDarwinNotificationExpectation: Another readable expectation for notifications.
XCWaiter class returns results of the expectations in the form of boolean. It returns enum of four possible situations those are .completed , .timedOut , .incorrectOrder or .invertedFulFilment . We can simply add extension to XCUIElement or add simple function which returns result of waiter function which returns one of the result from XCWaiter.
|
1 2 3 4 5 6 7 8 |
func waiterResultWithExpextation(_ element: XCUIElement) -> XCTWaiterResult { let myPredicate = NSPredicate(format: "exists == true") let myExpectation = expectation(for: myPredicate, evaluatedWith: element, handler: nil) let result = XCTWaiter().wait(for: [myExpectation], timeout: 5) return result } |
There is no callback block or completion handler. The helper method simply returns a boolean indicating if the element appeared or not. Let’s briefly go through each of the boolean results returned by XCTWaiterResult to understand what it is doing
Completed
This returns true when defined expectations are fulfilled or completed. It returns false when expectations are not met within specified timeout. Tests will likely to fail when expectations not met.
TimedOut
This will return true when expectations are not met within timeout. We can defined how to fail our test case by providing useful error message for debugging.
Incorrect Order
This tells us the expectations which are succeeded and which are still waiting to succeed. We know that what is taking time to diagnose the problem.
InvertedFulfilment
Not entirely sure how it works but as per docs If an expectation is set to have inverted behavior, then fulfilling it will have a similar effect that failing to fulfill a conventional expectation.
Readable Expectations
There are some new expectations added to the XCTest Framework which are XCTNSPredicateExpectation, XCTKVOExpectation and XCTDarwinNotificationExpectation. I think, idea behind that is to make expectations more readable and customisable. Previously we have to wtite expectations with handlers. Now we can write expectation like this :
|
1 2 |
let myExpectation = XCTKVOExpectation(keyPath: "exists", object: element, expectedValue: true) |
We can then pass this expectation to the XCWaiter to get boolean result. You can find some examples in the demo repo here.
What’s Benefit of XCWaiter
Better Waiting Strategy
XCWaiter gives us boolean results from each expectations which allows us to define our XCTest with better control. This might help to deal with flakiness of the tests. XCWaiter results simply return whether element appeared or not.
Ability to define Multiple Expectation
We can defined more than one expectation and wait for the XCWaiter to return result for each of them. As mentioned above expectations can be written in more readable way.
Handling Timeout & Error
Previously XCtest used to fail if expectations are not fulfilled within the specified timeout and error was very generic. Now we have .timedOut result from XCWaiter so that we can control test failure with proper error message.
Reduce Flakiness
XCWaiter gives us more control how we want to define test failure. This will reduce amount of the flaky tests. The XCWaiter results can also give us ability to fail or pass tests with proper error message
Highlight Performance Issues
XCWaiter gives us ability to define multiple expectations and each expectation tried to be fulfilled within timeout period. There is XCWaiter result .incorrectOrder tells us how many expectations are fulfilled and how many still waiting to fulfilled. This gives us indication why those expectations are slow and might have some performance issues. We can diagnose the slowness to make test and application faster
GitHub Repo XCWaiter Demo
I have created GitHub repo Xcode83_Demo with sample iOS application to give the XCWaiter try and experiment XCWaiter features. Feel free to clone and give it a go .
Let me know if you have other comments on XCWaiter.