This month we hosted Gene Kim, co-author of The Phoenix Project and other great DevOps books, and our own Anders Wallgren, CTO of CloudBees, for a webinar discussion focused on key technical practices for DevOps. But why focus on technical practices? DevOps isn’t about tools - it’s about the people! So true. No one will argue that the most important ingredient to a successful DevOps recipe is your people. But, transforming people and culture is a long term and often murky endeavor, regardless of what size your businesses is. The good news? DevOps technical practices are things you can start doing now to kick your transformation into high gear. Of course, they’re not trivial but they are concrete. And, getting good technical practices in place can have an immediate impact on your ability to scale CI/CD, modernize traditional releases and applications, and deliver faster. Plus, they’ll also reduce the friction that comes with trying to get broader culture and people transformation to take root across your organization. To better understand the importance of DevOps technical practices, we had Gene and Anders break down their “10 Dos of DevOps.” These are made up of the key technical practices that have a proven track record of driving DevOps success in large enterprises. Through insights and stories, Gene and Anders walked through each key technical practice, shared recommendations, practical steps, and how businesses are using these technical practices to become elite DevOps performers. They started by reminding us that the numbers are pretty compelling. Organizations that have mastered key DevOps technical practices have 46x more software deployments, greater than 2600x faster recovery times, and over 2500x shorter lead times.
DevOps Transformation Can be Like Camping
Framing the discussion - Anders shared a great analogy to highlight the relationship between culture change and technology. Imagine you want to your urbanite friend who hates the outdoors to become a camping enthusiast like you. Doing that will require a cultural transformation that will be challenging and take time. You can be sure that if that process involves a poorly built tent, old sleeping bags, and food cooked over a fire that you have to manually start each morning with pine needles - you probably won’t get very far in your “camping transformation” initiative. Having good DevOps technical practices in place is like having a good tent and a using a stove rather than pine needles! As Anders put it,
“DevOps and culture change can be like a first time camping experience: the tent, sleeping bag, and technique you use for building a fire will affect whether your nature-challenged friend will want to go again.” - Anders Wallgren
Take a Non-Oppressive Approach to Tooling
Speaking of how technical practices can impact culture change, one of the points Anders and Gene addressed was how to take a healthy approach to tooling. Anders highlighted that, “Organizations are using the same tools but there are always better ways to do things. Think in terms of abstractions and integrations so people can use what they’re productive with, nurture their curiosity, and get better at their craft.” In this context, he talked about how the “guild” style approach to promoting tools, practices, and architecture in DevOps differs from a more top down “centralized tool team”. He noted that there’s nothing inherently wrong with the central tool team (and can often have important positive benefits with regard to compliance and security), but that the “guild” culture really aims to foster experimentation, continual improvement, and feedback. This promotes a more “opt in” and collaborative team community and ultimately satisfaction and joy. (Ensuring that the tools and technology you use is complaint and secure is just one of the ways a guild culture will apply continual improvement to tool selection). Anders also made a very liberating suggestion when he encouraged teams to start creating abstractions and integrations with existing tools and automation without worrying about “beauty”. In this sense, exposing the automation and tools you’re currently using to others in a consistent and consumable way will have way more value than making it beautiful. By the way, this will often be a two way street between Dev and Ops. Ops benefit from leveraging what Devs are doing (configuring applications, test tooling, etc), and Devs benefit from leveraging what Ops are doing (provisioning infra, environments, etc.)
Decoupling Deploy Process from the Release Pipeline
Another insight that came out of the key technical practices discussion was the importance of decoupling deployments from the release pipeline. Anders shared a great story about a recent travel delay - and his amusement at how the updates and information on the airline mobile app, text messages, emails, and airport status screens were so incredibly out of sync and inconsistent. He pointed out that this is a release problem - different systems, with different technologies, being developed and released at different cadences.
“If the first time you’re using your production deploy code is when you do the “big release” then you’re in big trouble.” - Anders
Feature toggles, blue green, and canary deployment patterns are different approaches to deploying code into production without fully “releasing” to production. This is one type of decoupling. From the perspective of the the pipeline and environments - decoupling the automation, modeling, and configurations is important as well. This allows you to develop and template release pipelines that can be used with different deployment processes and tools, allowing you to promote pipeline architectures across teams that will be deploying different software. As much as you can - use the same tools and environment configurations in pre-production activities that you do in production. Taking even the smallest steps in this regard will yield huge gains in de-risking the release.
“Decoupling deploy from the release is one of the biggest “a-ha” moments for me. This is one of those powerful patterns because it allows us to hold the technical teams and the business and marketing teams accountable for what they should be accountable for.” - Gene Kim
CI and CD Must Be About Fast Feedback
Continuous Integration (CI) and Continuous Delivery (CD) aren’t new concepts to most of our listeners, but Gene and Anders reminded us that CI/CD need to be all about fast feedback. The software lifecycle needs to be based on short code point lead times, fast build times, fast test times, etc. As Gene commented,
Your business is best served when developers most strenuous efforts are spent focusing on solving business problems, not messing around with infrastructure.
If a company is not doing CI, they have large batch sizes which are prone to more potential problems down the road and slow ways of fixing and breakages. Small batch sizes create the confidence in submitting something that has a higher success rate out of the gate with the understanding that if something does break, it will get caught sooner before it ever gets to production. This element of “fast feedback” is instigated by speed amongst the team members with the follow-on completeness of testing and automation to help people see and realize that the outcome of the work they’ve done.
“It’s almost impossible to overstate how important that is to give a team confidence that they’re not breaking things when they submit changes.” - Anders Wallgren
Gene also succinctly connected how having good technical practices in place can deeply influence general moral and culture,
“When you have lead times of months, you never see the outcome of the work you do. All the joy vanishes.” - Gene Kim
These are just some of the great insights that came out of this rich discussion with Gene and Anders. If you want to hear all the key DevOps technical practices they discussed take a listen now. You’ll be that much closer to better scaling your CI/CD efforts, modernizing traditional releases, and improving agility across your entire organization regardless of platforms, tools, or maturity.