Erez Rusovsky is a director of product management at CloudBees and one of the former co-founders of Rollout.io, a ground-breaking feature flag management platform acquired by CloudBees that evolved into CloudBees Feature Management.
This is the second part of a series of blog posts by Erez. Keep reading to learn how feature flags can benefit every point in the CI/CD pipeline.
Why Are Feature Flags So Important to CI/CD?
It goes back to why you need CI/CD. It goes back to velocity and speed. Everybody wants to have high velocity. The DevOps Research and Assessment (DORA) group is getting a lot of momentum these days. They do quantitative research into what makes a good software delivery team.
Essentially, DORA measures four things—frequency of deployment, which is how many times you deploy to production; lead time for change, or how long it takes for code to get to production after you've merged it; mean time to repair, or how long it takes to fix production bugs and then change failure rate, meaning, what's the ratio between changes I make to the failures I'm introducing to the system?
The elite performers, companies like Facebook, Uber, Google and Netflix, deploy many times a day. The lead time for change is less than a day. From the moment an engineer commits code, it reaches production in less than a day. They have the lowest numbers in terms of MTTR. It takes them less than one hour to fix things in production, and the ratio at which they introduce new bugs to production compared to the amount of changes that they're making is the lowest.
The interesting thing is that you move fast, but you have fewer safety issues and fewer bugs in production. I think that's the core of the entire DevOps movement. Mark Zuckerberg at one point said, “Move fast and break things.” But the whole notion of feature flags is, “Move fast and break nothing.” CI/CD covers a lot of that space—being able to automate a pipeline and be able to run all these checks and go through these processes to make sure everything is legit.
The Power of Feature Flags Across CI/CD
Feature flags amplify a lot of dynamic capabilities into production. The real magic comes once you integrate feature flags in CI and in CD. Again, when these tools are separate, that's great. That's progress. Once you want to move into an automated phase, though, there is this notion of progressive delivery. This is a new term that is getting a lot of momentum these days. With progressive delivery, you control the blast radius of your software delivery in production. You always release to a subset of the user base and then gradually, when you see that everything has a positive impact or no impact as opposed to a negative impact, you continue with the release.
Yes, you can do that on the CD side, the deployment or delivery side where you do kinetic deployments or blue-green deployments, but doing this under CD is very complicated and requires a lot of orchestration, especially if you're in a microservice environment. In general, it's very hard to do. Doing rollbacks for a version is complicated. Not to say, don't do it. I say, do it, but it's complicated.
If you think about the fastest way you could move to do progressing delivery, it's with feature flags integrated into your CI/CD pipeline. It boils down to being able to test new capabilities in production on real users on a repeatable basis, not just as a one-off thing. If you want to do that, you want to integrate feature flags in your CI/CD pipeline. If you integrate feature flags into your CD pipeline, you get visibility into which feature flags are going to each release, the configuration, where it was deployed to. You can actually control the feature flag from your CD pipeline.
You can ingest metrics from other parts of the organization into your CD pipeline. It could be performance-related like APM data, remote file management data, or even user analytics data. And then in the CD pipeline, we've released this feature to this amount of users, and this is the impact that we're getting from these other sources. We could automate if we want to move forward or roll back or stay as it is. We can automate the approval flow—who are the stakeholders that need to sign off in order to release this?
If you integrate feature flags and CD, that's what you get. If you integrate that with CI, it's also super beneficial because it helps you manage the lifecycle of feature flags. I can see that when I'm building the code; I can debug my builds better and see the feature flags that are in my build, and what are the values. I can actually rewind my builds in the CI phase when I'm changing the values.
All of it together—CI/CD and feature flags—should help you become more like these elite performers. You're pushing stuff very quickly to production, and then you have a very low failure rate and a very quick time to recover from a bug in production.
To learn more about how and why to add feature flags to your CD processes, check out the CloudBees whitepaper, “Grow Continuous Delivery Maturity Using Feature Flags.”
Stay up to date
We'll never share your email address and you can opt out at any time, we promise.