CloudBees DevOptics Releases Gate Performance Metrics

Are you aware where you lose most of the time within your CD process? With advancements of DevOps best practices we are seeing an increased amount of automation, which speeds up software delivery and removes much of the burden from individuals and teams. But, it also introduces new forms of “waste”; activities, automated or not, that are inefficient.

Key areas of waste in the software delivery system

• Processing Time. Time spent in automation processing work. Inefficient automation or unnecessary steps can cause long running pipelines that result in slow feedback cycles.

• Waiting Time. Time spent in between activities. This could be due to hand-off, inefficient information flow or not optimized automation in between stages of your CD pipeline. This is much harder to identify and make visible. Value stream management surfaces this wait time in between activities.

• Unreliable Processing. The more we automate, the more we need to treat the automation system as production-critical. Flakey automation and flakey test suite execution cause waste. It is often hard to measure how much time is actually lost because of this to justify an extended amount of time improving test suite or automation reliability.

• Defects. It is important to capture defects, compliance and security violations as part of your delivery process. The quicker these feedback cycles the more quickly these can be resolved and the less context switching the development team has to do. The later these defects or violations are surfaced as part of the CD process the longer the feedback cycles and the more time spent moving non deployable work through the system, which results in waste and inefficiencies.

Announcing gate performance metrics

DevOptics already surfaces the lead time and the change failure rate of each gate. We are excited to announce that now we also provide insights into DevOps waste as part of lead time by splitting it up into queue time, processing time and idle time on a gate level:

• Mean queue time. Shows the amount of time commits spent in the queue before the job started running.

• Mean processing time. Shows the amount of time it takes from when a job starts to when it completes successfully.

• Mean idle time. Shows the amount of time a successful commit spends in a gate after it is processed and before it is picked up by a downstream gate.

Together with Change Failure Rate on the gate, the new performance metrics enable you to analyze your lead time in detail and better understand where inefficiencies may be slowing the velocity of the value flow in the software delivery process.

DevOptics case study

Here is some context. The “API build” gate on the DevOptics value stream is failing. This draws attention to the gate and triggers further analysis of the performance of this gate:

• Comparing processing and idle time. The time it takes to successfully process a change is 3x higher than the time it takes for the change to be picked up by the next step in the value stream.

• Analyze waste due to unreliable processing. Additionally, the Change Failure Rate (CFR) of 30% is contributing additional waste to successful processing and causes almost ⅓ of all runs to fail, blocking work from moving on. What is interesting is that Change Failure Rate seemed stable, but then started to spike and increase.

• Impact of gate change failure rate on processing time. These spikes in Change Failure Rate also caused the processing time to spike and become more unpredictable.

As mentioned above the "API build" gate is failing right now, completely blocking work from moving forward. That means we need to improve the resilience of this process to stabilize and optimize processing time and then zoom back out to look at the overall value stream to see if there are bigger inefficiencies.

At CloudBees, we are continuously improving our DevOptics value stream analysis and aim to provide actionable insights into your software delivery system.

If you are interested in value stream management and key DevOps metrics, get in touch and learn how CloudBees DevOptics can help you improve your software delivery process.

Or start using CloudBees DevOptics Free to get the visibility and insights to measure, manage and improve your software delivery platform.

Additional resources

• Download the whitepaper about measuring DevOps metrics and performance

• Read more about exporting DevOptics metrics to CSV files

• Learn about creating value streams

Alex Tacho is director of product management for CloudBees DevOptics. Prior to working at CloudBees, he was director of product management at CodeShip. At CodeShip, he was involved in building a continuous integration and delivery service to help product and engineering teams automate and improve their software delivery workflows.