Why You Should Standardize Workflows, Not Tools

The following is a guest blog post by Mazedur Rahman, lead software engineer at Airbus UTM.

Software engineering teams often dream of standardizing everything. In the name of efficiency or simplicity, they decide to adopt a single standard programming language. A standard MVC or REST API framework, a standard database and cache or a standard set of continuous integration and deployment tools – and so forth.

But in my experience, engineering organizations soon regret this strategy. The problem: Many of these decisions to standardize end up leaving you unprepared to meet future needs.

For example, teams might find themselves in a situation where they’ve written the entire monolithic backend of an application using a single programming language – but it’s one they can’t easily extend or swap out, leaving them in a bind. The same thing can happen for MVC or REST API frameworks, where teams might get stuck using an outmoded database or cache.

You also see teams build highly specialized CI/CD environments, only to realize over time that they’ve become so “special” that nothing can be iterated or debugged outside of that unique environment. Other teams will tie themselves to a specific type of deployment tooling that worked a few years ago, but barely suffices today. 

The inconvenient truth is that shifts in business and technology paradigms happen continuously, and your toolset needs to be able to keep up with the changing times.  

In the last decade or two, we’ve seen paradigm shifts impacting the entire technology spectrum. We’ve seen version control tools trend from CVS to SVN to Git. Architectural patterns have evolved from monolithic to SOA, and from SOA to distributed or microservice architectures (and for some of us, back to monolithic again!). The typical infrastructure setup has moved from bare metal to cloud. And these days multicloud, hybrid cloud and edge are all the rage. 

Of course, CI/CD tools have gone through their own evolution, with CruiseControl giving way to Jenkins, which is still thriving today. Likewise, various provisioning and configuration management tools have jockeyed for market dominance in recent years, with no clear winner. We must also accept the fact that many of these tools may one day fade into the sunset. 

The lesson is clear: No matter what tools and technologies you pick today, some of them will turn out to be inadequate in the years to come.

I’m convinced the best way to survive in this constantly changing world is to embrace it. That means not locking in permanently on a single technology, toolset or architectural pattern. Be open to change and accept a diversity of solutions. 

Here is the other key: Even as your tech landscape and toolset evolve, find a set of stable workflows that will remain unchanged over time and standardize on them.

For example, let’s say we need to build a tech stack that involves a load balancer, a web server, a database, a caching solution and so on. Well, we can easily find a lot of common tasks and workflows among them:

• We need to build and test how everything works

• We need to deploy to the target environment

• Once deployed, we need to keep it up and running

So, why not standardize these simple but powerful workflows that won’t likely change over time? Thus, our set of workflows would simply consist of build, test, deploy and manage.

The key to survival, however, is to keep these “stack flows” as composable (decouplable), swappable and testable as possible. To do that, the “deploy flow,” for example, could be broken down into a series of “subflows,” such as deploying a cache, database, ingress, pods and TLS resources. And – here’s the key – each one of these subflows needs to be testable in isolation. For example, if we needed to swap the cache, we would just change the cache-related logic and test that component alone without changing the entire workflow. 

We also need to keep these subflows swappable. For example, while we might decide to standardize on Kubernetes to deploy the ingress, pods and TLS resources, we might use tooling from our cloud provider to deploy managed services such as caching and database services. Or we could even use other tooling such as Terraform or Pulumi. 

The beauty of this stack workflow pattern is that it allows us to innovate as well as decommission, but it does not break our known stable workflow. That’s the key to staying ahead in a changing world.

Check out the "Why You Should Standardize Workflows, Not Tools" presentation from DevOps World.