Accomplishing Continuous Feature Deployment for Mobile Apps
The world of software development is currently going through a fundamental change in how developers and product owners are thinking about software versions vs. feature delivery.
A Short History of App Development A couple of decades ago we were still shipping software on floppies, CDs or DVDs. That meant very long development cycles measured in quarters or years and huge QA efforts. It also meant developers had basically zero control over their software once it was shipped. Then came along the Internet which eventually brought with it the SaaS based software movement which gave developers full control over the production environment and the ability to release new features continuously. It also meant you could have a single “live” version of your software - all users needed to do is reload their browser. For the most part, having a single environment for all of your live users greatly simplifies the development process. You have one “pipeline” where code goes from dev to staging to production. That’s an oversimplified version, but you get the point.
The downside of having a single live version is that it can slow down the speed in which you’re able to innovate.
Where App Development is Going In an ideal world, you could easily run multiple live versions of your software in parallel while measuring the business and technical KPIs of each version separately. This would allow you to move even faster than having a single live version as now you can control, test and compare anything you want in your software. The problem is that from a technical perspective, having multiple live “versions” of your software can get very messy and complicated without a proper solution to manage, control and measure your each version. One of the first examples of being able to run multiple live versions came in the form of A/B testing. Also called split testing, A/B testing is usually used to compare different versions of UI elements or text with the goal of improving conversion rates. While split testing is a step in the right direction, what we’re now seeing is a movement to support multiple versions of anything in your software, not just multiple versions of specific elements. Instead of thinking in terms of discrete versions which describe your software as a whole, ie v1.3, v1.4, v1.5 and so forth, you should be thinking of discrete features which can be on or off for any group of users.
It’s time you detached your code release process from your feature delivery process.
Every Change is a Feature. Every Feature is an Experiment: Continuous Feature Deployment for Mobile Apps https://t.co/pShoJW6W0c
— Rollout.io (@RolloutIO) February 20, 2017
All of the top sites and apps such as Amazon, Ebay or Airbnb are already doing this. They consider every change to be a “feature” that should be tested. They test a new features on a small percentage of their users before releasing it to everyone. This enables them to test the impact of any change they make to ensure new features do not have any negative business or technical impact before releasing it to all users. Every change, whether it’s new functionality, a technical optimization, a bug fix or a UI modification, can and ideally should be tested as an experiment. This is accomplished by measuring and comparing the business and technical performance of your app, with and without the change.
Every change is a feature and every feature is an experiment.
Being able to selectively turn features on or off not only accelerates development and business results, but also greatly reduces risk, as any negative impact can be immediately stopped. A Closer Look at the Mobile Development World When mobile took off, especially mobile apps (not just mobile web), the release process unfortunately took a step backward in some aspects. We’re back to “shipping” software to users, though at least now you just need to download it from the App Store or Google Play instead of opening a box with a CD in it. As a mobile app developer, you also no longer have direct access to your live code. As a result, it’s harder for you, a mobile developer, to release new features at the speed your business wants and make sure you’re not sacrificing any technical or business concerns. The limitations imposed by not being able to modify live mobile apps make separating releases from features even more crucial than in the Saas world. The ability to have control over your live app has become a “must-have” capability for popular mobile apps. I’ve come across a wide range of solutions to address this need in the form of open source projects, commercial offerings and also in-house tools. As a matter of fact, most of the large companies with “top 10” mobile apps have already created an in-house tool that gives some some control over individual app features. For mobile developers, it’s now a given that you should integrate some form of “remote control” into your app, both at the code level and the development process level, but it’s usually not clear how best to go about it. At the end of the day, when you make a change in your app, whether that be adding a new feature, improving an existing feature or changing the UI, all of your efforts are done towards a goal. For example, get more users, increase revenue, reduce churn, etc. Ideally, you should measure the impact of every change you make in your mobile app, regardless of the specifics of what type of change it was. Want to add a new feature? Compare the app health and business metrics of users with and without the new feature. Want to change the UI? Use a different internal datastore? Even fix a bug? Measure it, and make sure you’re measuring both business and technical aspects (such as crash rate).
Treat every change you make as if it were an experiment - Measure it and Control it.
Of course you need to be able to easily turn experiments on and off as needed and also target the experiment to the right users. In my next article I’ll go into more detail and address how to look at “experiments” in terms of duration, targetting, environments and variability. I’ll also cover how experiments for mobile apps are different than for the web and how to address these differences.