Fearless iOS Continuous Delivery with TestFlight and Phased Release

Shashikant Jagtap

The major obstacles to continuous delivery are the lack confidence in the release candidate and fear of critical bugs in the production. While releasing an iOS app, there is always fear and risk of bugs in the production. No matter how much time spent on testing in the pre-production environment. It’s because we are unsure that our beta testing environment is mimic of the production environment. There are loads of third party iOS beta testing services available in the market but Apple has it’s own beta testing service platform which is called TestFlight to give the confidence in our testing.

In previous posts, we have seen how to setup end to end continuous delivery pipeline for iOS app with Fastlane. Using this setup we can deliver any commit or merge to GitHub branch to TestFlight causing multiple builds uploaded to iTunes Connect. It would be a good idea to maintain those builds in TestFlight and distribute to relevant people to test.

Why TestFlight?

There are various beta testing platforms available to test ad-hoc builds like Fabric, HockyApp etc. There are 11 of them as mentioned in this great blog post  TestFLight Alternatives which compares all of them. TestFlight has major enhancement as mentioned in What’s New in iTunes Connect at WWDC 2017 session 302.  There are multiple reasons why we shouldn’t use other beta testing platforms, some of them are as follows.

  • TestFlight is Apple’s official beta testing  platform so there is no possibility of breaking it if API is changed
  • It’s easy to distribute the builds to anyone by adding testers to internal or external testers
  • No Need to maintain devices and provisioning profiles for ad-hoc distribution.
  • No need for separate configuration to distribute .ipa  file other beta testing platforms.

This is a small list but there might be various other reasons that you should think of getting rid of these third party beta testing services.  The top reason would be the recent enhancement by Apple in the TestFlight which we will explore in this post.

Multiple Builds and Group

As per the latest in the TestFlight, we can create multiple groups in The TestFlight and distribute specific builds to a specific group.  In an Agile world, we can create a group for QA Engineers, Product Owners and Stakeholders. We can continuously deliver the builds to TestFlight and assign the build to a specific group to get the early feedback. As suggested in the WWDC talk, we can create, Alpha, Beta and Release candidate group. As build go mature, we can promote it to final Release Candidate group. We can also create a group for the experimentation to check specific feature separately.

 

Switch Between the Builds

There might be a tester added to multiple groups, they can now see the previous builds in the TestFlight for which they have access to and select anther builds for testing. This feature will be ver helpful for the QA Engineers to test multiple features easily and without hassle. The previous builds can hold the test information so that testers can identify the information to test. Now TestFlight builds will also show the session count and the crash count. Apple is also planning to extend the testers limit to 10,000 testers at some point this year.

Phased Release

The practice of Continuous Delivery enables us to deploys new builds to TestFlight but the production is a unique environment.  No matter how much testing we have done on pre-production, we have to watch the behaviour of the app on production once released. If we are about to release the new feature which involves risk or we are less confident about but we want to test how it performs on production without taking too much risk then phased releases can solve the problem.

When the feature is enabled for the phased release then it will be distributed to certain users for seven days. The limited user will receive the updates and try the build. on the production to verify new feature. The percentage of users selected are shown in the figure below

There is the difference between the phased release and feature flagging, you can read the main difference on the blog here but it has some similarity.

Enable Phased Release

There is new radio button in the iTunes Connect called “Use phased release” which can be selected to enable the phased release for the build. We can select for any build which we are less confident about and we don’t want to distribute to 100% of our users.

Roll out Or Rollback

There are two possibilities after we released the build using phased release, either our feature is working fine in the production or our feature has bugs.  We have the ability to deal to deal with both of the situations.

  • When phased release is OK then we can deliver to 100% user by pressing “Release to All Users” button. This will stop the phased release and everybody will get the updated version.
  • When phased release has minor issues then we can pause the release for a maximum of 30 days and we can submit another build with the new fix.
  • When phased release has serious issues then we can remove the current version from the sale.

Advocate Continuos Delivery

The newly announced features like multi-build, multi-group TestFlight and Phased release are the catalyst for the continuous delivery. These new features give the confidence to manage risky releases by taking a calculated risk. There will be following big changes likely to happen in iOS projects all over the world.

  • iOS developers won’t be scared about making multiple TestFlight builds provided they are automated.
  • Everybody will understand the value of automated release/deployment process.
  • With help of groups in TestFlight, we can get early feedback from QA, Product Owner and Stakeholders. Once everyone happy then releasing is a click away.
  • Product owners won’t be scared to make frequent releases using phased release as there won’t be major risk even if application is broken in the production

Conclusion

Delivering the product (iOS apps) to end users quickly and whenever needed is the key to success. It can be easily achieved with setting up the Continuous Delivery pipeline and using the newly added feature like Multi Build TestFlight and Phased release.