DevOps World | Jenkins World in San Francisco has come and gone for another year but I’d like to dive into a conversation I had on the last morning with a young DevOps engineer named Glenn who works at a bank. We talked about a number of things, including his frustration getting the security and app development groups to work with the DevOps group in a timely manner. In other words, the bank needed to break down silos in order for its digital initiatives to succeed.
Sounds like a common scenario, right? “Those app dev guys or security guys would be so much happier if they just adopted our new process/tool/magic.” Glenn was really frustrated with the app dev guys because they all waited until the last possible minute to adopt a new mandated process, even though senior management had directed everyone to do so. He also bemoaned the Department of No (aka security) for stopping releases or new process adoption after Glenn’s group had done all the hard work on them.
I felt like I was rereading a chapter of The Phoenix Project.
Glenn was particularly frustrated because the bank had recently restructured the DevOps team. The team had been their own group but management felt that was not efficient so they got rolled into engineering to be more integrated with the software release process. Since the restructuring, however, Glenn hadn’t seen any change in the level of communication across the teams nor any more willingness to collaborate.
All the teams agreed that going faster was in the best interest of the company but nothing was changing. Meetings were still frustrating and unproductive. Teams always seemed to change their priorities - even after promising to deliver certain things right away. The silos SHOULD be starting to break down but there was no evidence that was happening, nor would it in the near future. On the surface, teams were making the right noises but when it came to execution, it was the same old story.
Maybe I was pre-reading an unwritten chapter of The Unicorn Project, Gene Kim’s unpublished follow on to The Phoenix Project.
As much as I enjoy both books (the second one is due to be released in October but I got a pre-release version at DevOps Enterprise Summit in London), I think they both underplay a key component of organizational behavior: Compensation plans. An individual’s comp plan is very clear in telling that person what, exactly, they are expected to do. Individuals and teams are measured on doing certain things and the comp plan monetizes and prioritizes those activities. If it isn’t in the comp plan, a person generally isn’t going to do it because they have no incentive to do so, even if it makes perfect sense or is key to the long-term success of the company.
We at CloudBees have seen some painful examples of this mismatch at customer sites and the deleterious effect it has on behavior, collaboration and quality.
Security mistrusts dev because engineers are shipping buggy code so they can meet release dates.
QA writes incomplete test suites just to clear that gate. “Hot-fixes belong to security.”
Real security concerns get ignored because they are buried in a pile of insignificant ones and no one can tell the difference.
Dev is paid to deliver features or applications with no connection to quality, cost of production or user satisfaction.
I suspect a compensation plan mismatch is what is in play at Glenn’s bank. While the reporting structure and goals of the organization may have been “aligned” for digital transformation, if the compensation plans of those teams haven’t been restructured to dictate collaboration or prioritize common goals, then no amount of cajoling, executive decree or two-pizza lunches will make any difference.
When I was in sales, I always asked prospects how they were measured: what metrics their company used to know they are doing a good job. You’d be surprised how many salespeople never do that. The answers were always fascinating but told me exactly how to structure the conversation and value proposition from that point on. Instead of wasting their time talking about stuff they don’t care about, I focused my sales pitch and value proposition on exactly how my stuff helped them make their bonus.
Side Note: At CloudBees, we’re currently structuring around what we call Product Business Teams (PBTs) to ensure we have maximum collaboration and can efficiently translate business objectives into release strategy. The various PBTs have a few clearly defined metrics they are tracking and all initiatives, projects, and decisions are tracked back to those metrics. Anders Wallgren, our vice president of technology strategy, has a favorite saying that applies here: “Decide on the outcome you want first, then figure out how to measure it.” I would add once you start measuring it, make sure everyone is compensated on those metrics and ONLY on those metrics. For my PBT, since we’re all measured and paid on the same two things, engineering, sales, marketing and operations now have a common language, a common sense of accomplishment and the incentives to work together.
Like the hapless sales rep trying to sell something that doesn’t address the prospects’ corporate and personal needs, a team like Glenn’s that doesn’t understand the needs of the other departments will never attain the level of cooperation and improvement that is the goal of DevOps. While it’s not Glenn’s job to change comp plans, I did suggest a few things he could do to better understand how the other teams were motivated and use that knowledge when he had to work with them. He should ask each group the same two questions:
How they were measured, and,
How they were doing against those metrics.
Their answers would tell him exactly why the other groups weren’t cooperative or responsive. More importantly, armed with this new information Glenn would finally be able to communicate with his colleagues in a language that would encourage collaboration: “Your priority is X? Great! I have a process that will help you attain that by Z, Y and Q.” He might not be able to change compensation plans but he can change the conversation.
Read more about what’s needed in DevOps
Download the free whitepaper on Software Delivery Management
Join the CloudBees SDM solution preview program to see what it means to be on the same page