As an engineering leader, you certainly care a lot about improving the efficiency and productivity of the developers on your team. But improving your engineering team starts with measuring it. After all, as the saying goes, you can’t really improve what you don’t measure.
That’s easier said than done, though. There are plenty of metrics out there, and not all of them are equally valuable. Today’s post helps you by offering a shortcut: we’ll show the three most impactful metrics you should start tracking to measure and improve engineering team efficiency in your organization.
We’ll open the post by briefly covering in more detail why it’s important to track engineering efficiency. Then, we’ll talk about some of the mistakes that teams and organizations often make when trying to measure productivity. After that, we get to the meat of the post, where we’ll walk you through our list of the most important metrics.
Engineering Efficiency: Why It’s Important to Measure It
You’ve probably heard of cargo cult programming. That is the practice of adopting programming techniques or procedures without understanding why that’s important. Well, we’d say that there are also organizations that do cargo cult monitoring: they monitor and track a plethora of metrics “just because,” without really understanding the value they get—or maybe don’t get—from all of that.
It might sound counterintuitive, but the engineer efficiency metrics you track have more to teach about your team and organization than about individual developers. They can help you find bottlenecks in your development process, including inadequate automation. They might help you discover that there’s room for improvement when it comes to communication between the dev team and the business. Perhaps more importantly, they can help you uncover deeply ingrained cultural problems, such as a lack of clarity when prioritizing work and deciding how to allocate resources. In the end you want an engineering team that is efficient and focused on contributing to business success and customer satisfaction.
That’s why you track engineering efficiency: because it helps you assess in a very objective way the health of your project and team so you can take the necessary steps to improve them.
How Not to Measure Engineering Efficiency: The Mistakes Teams Make Along the Way
Before we get to the list of metrics you should be aware of, let’s briefly mention some of the main mistakes teams and organizations make when trying to track and improve engineer efficiency. As we’ve mentioned earlier, there are many metrics out there, and not all of them deserve your attention. Caring about metrics you shouldn’t care about is certainly a mistake, but you don’t need to worry that much about it since we’ll tell you which metrics you should care about.
So what are the mistakes you should avoid?
Focusing too much on individual metrics. As you’ll see, the most valuable metrics are the ones at the team level. There’s a reason for that: your main goal here is to diagnose and fix problems to improve the development process itself.
Attaching metrics to rewards or outcomes. A metric that becomes a goal is no longer a helpful metric. That creates perverse incentives that encourage people to game it. So tying metrics to any type of reward—especially financial ones—is a sure way to undermine the value of said metric.
Using good metrics for bad reasons. There are metrics that, by themselves, aren’t bad. But when they’re used for goals they weren’t originally designed for, things can go south quickly. A good example is agile velocity. When used to estimate the capacity of the team for the sprint, it’s a fine metric. When used for measuring team performance or making comparisons between teams, it can lead to perverse incentives for the developers—e.g., point inflation.
The Three Most Effective Engineering Efficiency Metrics
Without further ado, let’s cover the metrics we’ve been promising you. For each metric, we’ll briefly define it and then explain why it’s important that you track it.
1. Average Cycle Time
Defining Cycle Time
Cycle time means the amount of time spent actually doing the work. In more concrete terms, we can say that the cycle time refers to the number of hours or days from the moment the developer starts working on an issue until it gets delivered to production. Between these two milestones, a lot can happen, and that’s what determines the importance of this metric.
Cycle Time Has a Lot to Say About the Efficiency of Your Development Process
The shorter the average cycle time in your teams, the better. Not only because that means that you’re delivering value to your customers sooner. That’s important as well, of course, but cycle time is a crucial metric for another reason. It speaks volumes about how efficient and streamlined your software development life cycle (SDLC) is.
Do pull requests (PRs) in your team take a long time to be reviewed? Bad sign. That might indicate that your developers are so overwhelmed they don’t have the bandwidth to review and merge PRs. Or maybe the PRs are so huge the reviewers get discouraged. This can be either a planning problem or a culture problem.
Do your automated tests routinely fail due to flakiness? Automation inefficiency. Do engineers struggle to code the features they need due to poor requirements? Possible communication issues.
No matter the cause, long cycle times are a sign there’s room for improvement in your development process.
2. Time Spent by Area
What Do We Mean by Focus Areas?
In an ideal world, engineers would spend 100% of their time—or close to this—working on delivering value to the customers through new features. However, we don’t live in such an idyllic place, and engineers have to divide their time among multiple areas of focus in the application. Those include fixing bugs, doing security-related tasks, refactoring, doing test suite maintenance, and more.
Work Isn’t Homogeneously Divided: Learn Where Your Dev Hours Are Going
Why is it so important to know how your engineers are spending their development hours? Well, all else being equal, you’d rather have them working on generating value through innovation and the creation of new features, right? That’s what we thought.
Of course, bugs need fixing. But the fact that the team is spending a disproportionate amount of time working on bug fixing is what should give you pause. Tracking the time spent on each type of activity is the first step to ensuring your team has the right conditions to work on high-value tasks without too much blocking, interruptions, or constant context switching.
3. Time Spent on Planned vs. Unplanned Work
Defining the Metric
Our final item is a spiritual successor of the previous one since it also has to do with finding where your development hours are going.
More specifically, this metric refers to the ratio between planned vs. unplanned work. For instance, what percentage of the work didn’t have an associated epic?
Lots of Unplanned Work? Probably a Red Flag
Why is tracking planned vs. unplanned work important? It all boils down to focus. Having a disproportionate amount of time devoted to unplanned work is a bad sign. It indicates the team is struggling with barriers, distractions, and overall lack of focus.
That might also be a sign that the team is struggling with planning, misestimating their capacity for work. Lack of clarity when defining priorities for work might result in “urgent” tasks needing to be handled during the sprint.
Ensure Your Team Builds What Matters the Most, Quickly and Efficiently
Measuring engineering efficiency isn’t about single developers but rather the whole team. By tracking and improving the right metrics, you’ll be able to ensure that your team
spends most of their time on areas that generate the most value for customers;
focuses their time on the right tasks (i.e., planned work); and
does everything as quickly and efficiently as possible, without giving up quality.
In other words, you want your team to build the right stuff, the right way, in a timely manner. We invite you to take a look at CloudBees’ Engineering Efficiency to see how it can help you in achieving those goals.