This is the first part of a series of blog posts by Erez Rusovsky, a director of product management at CloudBees and one of the former co-founders of Rollout.io. Keep reading to learn about the motivation behind starting Rollout.io, and how people initially responded to the product.
Why A Feature Flag Management Platform?
We didn’t start with feature flags. We actually wanted to solve a problem in the same space, around mobile engineering, and how the release cycle for mobile engineers is very, very slow. As a developer, I need to send the app to the App Store, have it reviewed and approved—and only then, the users can download that to their phones. Even if you have a bug in production on a mobile phone, as a developer, the only way for you to fix this is release a new version, and that would take ages because, again, you have to pass all these steps. Sometimes it would take as long as 10 days.
We created a very innovative solution that allowed mobile engineers to make changes to their application without redeploying their software on a code-level basis, so they could actually change lines of codes remotely. We had a lot of success with that. As anybody who hears that story might guess, though, we were circumventing Apple’s approval flow. At one point, Apple decided to shut it down, and so that was that.
We started by helping engineers, specifically mobile, be able to ship faster. But then, the more we talked to companies and the more we understood how they work and the discovery process and selling to customers, the more we heard about feature flags being used—as a more structured way of having post-deployment, production-level control over their software. It's not on the code level—it's on the feature level, and it allows a lot of dynamic capabilities.
Basically, with the flip of a switch, we wanted teams to be able to turn off a feature that is not behaving as it should or a piece of code that's not behaving as it should. It could be performance-related, but it could also be tied to error rates, crash rates, load on the systems, or it could also be business KPIs.
The more we talked to customers, the more we saw validation for the need. We've seen so many companies that have invested in a homegrown solution for feature flags. They started off asking, “How do we mitigate risks? How are we able to shut down something if it's not behaving as it should be?” But then we saw them move to other use cases. We saw that, over time, they often built an entire mini-system themselves because they really wanted the functionality. We also saw that investing in that system is a big effort, and you're never going to do all of it on your own because it becomes complicated. It becomes a specialty. That's when we decided to build a system that does that for our customers.
Feature Flag Use Cases
If you think about it, feature flags have a bunch of use cases, but they boil down to a few main ones. One is risk mitigation, and we've just talked about this. Risk mitigation could be reactive, meaning I have a bug in production and I want to turn that feature off. Or it could be proactive, meaning I'm concerned about this next release and I'm going to gladly open this to a subset of the user base for testing. Maybe start with just internal employees, then alpha users, beta users, 5% of the user base. Once I see everything is okay, I'll open this to everybody. Again, this could be on the system performance side, but also on the user case, business KPI side.
Another use case is the enhancement to the velocity of the engineering team. We were surprised that a lot of times the main reason people were adopting feature flags was the need to move faster and the need to have better engineering productivity.
Companies that want to move to trunk-based development will merge code to the main branch all the time. That's a known technique for teams that want to be high performers. Sometimes as an engineer, if I haven't completed my work, I can't merge into the main branch. If our team uses feature flags, I can put that thing behind a feature flag and merge incomplete, untested code to the main branch—and we're able to move forward.
The last piece I'll mention is common knowledge on feature flags—the fact that you get release predictability. If you have a late-stage QA bug, instead of postponing the release cycle, you could just turn that feature off and continue with the release cycle.
How Feature Flags Empower Engineers
We saw a lot of value in helping engineers move faster and be safer. Then there is also the more advanced use case of feature experimentation, which essentially means I'm going to measure the impact of a feature on my system, on the performance side or the business side. With data, I'll be able to say “this feature caused that”—whether that’s more crash rates, more or fewer error rates or improvements to the business KPIs.
So we started building the tool. It was easy to sell. We got a lot into conversations with companies that built something internally with very limited capabilities. Database-driven or files-driven, no UI, no user management, with a limited ability to choose to whom you deploy.
That was how we got in. We saw a lot of the value being generated on the enterprise side. With feature flags, there's an easy start—one team, one feature flag at a time. You don't have to go all-in, but what we've seen over time is that many of the companies that adopted this very quickly have moved to a point where everything is gated behind feature flags.
That was always super interesting. We always kept looking at how many flags a team has at any given point as a way to see their satisfaction and their adoption level.
To learn more about how you can use feature flags at your organization, check out the CloudBees ebook, “The Ultimate Feature Flag Guide.”