Diego Lo Guidice of Forrester: The "Age of DevOps" is Here

Andre Pino: We're coming to you today from DevOps World/Jenkins World in Nice, France and today I'm joined by Diego Lo Guidice, vice president and principal analyst with Forrester. Welcome Diego.

Diego Lo Guidice: Thank you very much. Good morning Andre.

Andre: Diego what's – maybe you can talk a bit about what your area of coverage is and what your specialty is?

Diego: Sure, so I'm part of the global analyst team in Application Development and Delivery in Forrester and the areas that I cover, I've been covering for quite some time, has been helping clients build better, softer, faster, that's my mission and the related research is around agile. I've been writing a lot of research around agile from the very beginning I would say, agile and DevOps, but I also focus a lot of my research on quality and testing and how that relates back to these two very important topics because testing is one of those areas that people tend to forget about.

Andre: Yeah, absolutely. So in our research what are some of the more interesting trends you're seeing these days in DevOps?

Diego: So I can definitely confirm that it is the "Age of DevOps." It's actually – you can see it here the vibe and the number of people that you guys brought down to Nice in Europe. I think being one of the first, the first DevOps World that you've organized. I mean this seems to be a great success in my eyes.

Andre: Yeah, we love all the energy we're seeing here.

Diego: And it reflects you know the general trends numbers that we see that everybody is doing some form of DevOps in organizations. You know more than 50 percent of the organizations claim they're doing DevOps – planning or doing it. Why are they doing it? Mostly one of the important metrics that we that seems to be the trend is how fast can I deliver releases into production. That's kind of been improving slowly in the past few years but...now I think it needs the next big thing to really make it increase. We see that more or less 30 percent of the organizations deliver monthly or sooner, that's around 28, 30 percent, and the rest are still stuck into the quarterly, six months, yearly release. So yeah, I think that's kind of the important – some of the highlights on the trends that I see.

Andre: Do you think at this point in time organizations generally have a good understanding of what DevOps actually is?

Diego: So that's a very good question actually and the answer is no. I go to large organizations, I was at a very large bank about a month ago, and I was there to talk about their DevOps transformation and I ended up only talking with the head of infrastructure and operations because he was thinking it's all about automation. And buying a tool, and installing it, and now we can build – we can do DevOps, right?

So I think there's two things that were missing in that conversation, which I obviously recommended to that client, which was well if you have some transformation going on in the bank you have to link it to that transformation more broadly, business transformation, digital transformation, but also you must have or if you haven't you should be having a transformation on the development side as well. You know in these years, everybody has been trying to adopt agile and so these things should be connected. You can't just have these isolated efforts going on. And so I think that leads me to think that they don't really understand that DevOps is basically the other side of agile right? They're two sides of the same coin, that it's a cultural thing, it's a movement, it's an ideology, and it's not just automation. Automation is key.

Andre: Yeah, of course.

Diego: In technology...if there's no automation, you can't really go fast but these other aspects are really important, leading fundamentals.

Andre: Yeah, so you've mentioned agile. You've covered agile for a number of years now and now we have DevOps. So how do you see agile and DevOps working together?

Diego: So I kind of hate when I hear some kind of talk about post-agile or talk about agile 3.0 or 2.0 or that now DevOps is replacing agile and you know I think that's wrong; that's the wrong view. In my view, agile and DevOps are two sides of the same coin where agile gives you, like a nice Ferrari car does, that great experience, the emotion of using something that you like, at speed with quality, but you can't enjoy driving a Ferrari or any other wonderful car if you don't have a proper infrastructure. Think about driving a Ferrari on a bumpy road or on a winding – well I would enjoy a winding road – but if you don't have a safe infrastructure with grit on the road, that's dangerous. If you don't understand the signs on the road that's dangerous too because you're driving 200 miles an hour and there's a sign saying, "Hey, bump ahead at 100 miles," you'd better slow down, right?

So what's important is the communication language, the language between development and operations and that's what DevOps brings. So agile tears down the wall with business, DevOps tears down the wall with operations, and now you can have really the end-to-end agility. So you can't do one without the other if you really want to leverage successfully delivering better software fast.

Andre: And are you seeing data that supports this?

Diego: I was waiting for that question because I don't want people to take just my word for it. It's a nice analogy and I know that people love the analogy of infrastructure, infrastructure that scales and roads and stuff, but yeah, no, I do have some data to back that up because I do this survey every two years on agile, the state of agile adoption in the market and ask all sorts of questions around you know what -- not so much if they're doing agile but how they're introducing agile, how they're being agile, the practices that they use, the impediments, et cetera. And then I ask a question do you – of course we talk about DevOps, and I have a specific question asking about is your DevOps initiative somehow a joint initiative with agile? Are they linked together? Are they separate?

I'm trying to find out strategically...really they're going after this. And so I get answers on that and now if you cross the data, you get some real facts, interesting facts that show that the teams that are doing just agile on the question faster business value that you're achieving 41 percent of the agile teams would say, "Yes, we're getting faster business value." But the teams doing agile plus DevOps you get 69 percent of the teams saying that. Quality, you get 29 percent if they're just doing agile; you get 72 percent if they're doing agile plus DevOps, improved technical quality. Even functional quality goes from 47 percent to 72 percent if you're doing both agile and DevOps.

Andre: That's pretty significant.

Diego: Right? Yeah.

Andre: So in there can you infer that that also implies that the dev team and the ops team are working together better?

Diego: Yes, absolutely because in a value stream as we like to think about it there's a theme that is common to both agile and DevOps and it's lean, lean thinking. Focus on value, eliminate waste. And so the waste that gets into that pipeline when developers and operations are not collaborating is causing things not to be deployed fast, right? You can go as fast as you want with agile with business but if you haven't fixed your stuff on operations, if the operations folks don't know what you're doing, when you're going to be deploying, what tools you're using, common procedures, automating, testing if testing is not happening between the two as well you can't get that improvement of faster business value delivered and you can't get that better quality, right?

Teams are integrating code continuously that we know as a computer scientist that the more you integrate, the more you fix things early on the more things go better. So it's definitely about collaborating more, having the same language, having the ops learn the development skills so that they can automate, but also the developers understanding what the requirements are in production and having and demanding from operations, you know, I want a safe infrastructure, an easy infrastructure to use. As a developer I'm an application developer. I want to do a simple put. Someone else has to take care of what that means going into production, not me.

Andre: Right. So you mentioned value streams, which are the process side of DevOps, right, it's the end-to-end process and I know a lot of organizations are using value stream mapping to define what that process is but what are you seeing with respect to value streams?

Diego: So that's a good...

Andre: Does it go beyond just the mapping?

Diego: It absolutely goes beyond the mapping and I have to give credit also to Kurt Bittner when he was in Forrester with me. He was my counterpart in the Modern Application Delivery Playbook that we were writing and we had this – we wanted to write about value stream, we called it value stream management for lack of a better definition because what we were seeing is that first of all it's a lean concept so it deploys nicely and the visualization of that end-to-end. So what a value stream is, it's really the people, process and technology that help someone from a change request or an idea supporting that implementation and execution and deployment into production; that's the value stream and you can think of it in simple terms as a process.

Andre: Right.

Diego: And so you know visualizing that journey like let's say from a change request, doing what...people would call a Gemba walk when fabrics were around because value stream was used is a lean concept in the industry manufacturing industry, going and having a Gemba walk to see how long things take and where they get stuck and what exactly happened, you can read the process in a book but what happens is different, right? So you go there and you see exactly what happens. So in value stream management if you had a tool that could allow you to say that from change it took two days to come up with the initial business idea, it took another two days to write the business idea, it took another four days to implement the business idea to write the code or design it, and then a week to test or whatever, and then you go on until it deploys and that's process time; that's the real time it would take. You sum those up, let's say hypothetically it takes 39 days end-to-end. Well unfortunately if you look what really happens it might take 140 days and what those – that delta of 90 minus/plus days are is waste. So in between each one of those execution processes, process time, there's lead time and that lead time is in between all those steps and usually it comes in the form of waste, of meetings, of creating artifacts that are not really necessary and not working code, so it's prioritizing on the wrong things. So if you show a nice scheme of that process with the lead time and the process time it's really eye-opening for executives and for them to say, "Well we definitely need to do something about this. We can shorten this time down."

Andre: It really exposes the inefficiencies.

Diego: The inefficiencies, yes, the waste right?

Andre: And a lot of that can be just communications between teams right?

Diego: Just communication, just how we're meeting, long meetings, someone is on holiday and he has to approve. Well that's governance, right, and guess what? We can automate governance. One of the important objectives of DevOps is automating governance. It's not getting rid of governance because that scares organizations. It's automating governance so things we can move faster but you start from analyzing where your failures are, where the big problems are, and that's typically where – which what I recommend clients do when they're starting an agile/DevOps journey is take a value stream and and go for that.

Now there's also the next step of that is visualizing what's happening in the value stream and I think when I earlier said there's something that has to happen to bring really that speed time to delivery down or up in terms of positive, faster, it's going into the value stream management direction, which means okay, so if you create these pipelines, if you create multiple pipelines, you have a release that is formed of 10, 15 applications and products that you're working on, you have to orchestrate all that, you have to deploy it into production, but you want to know what's going on in that pipeline. You want to have visibility. All the stakeholders want to have visibility. Business wants to know why are we putting all this money in agile and DevOps? What's the value that I'm getting? That brings us to the next level, which we call value stream management, and I think you guys called it yesterday, Sacha on stage called it CRM for continuous delivery. I liked it. I like that definition.

But that's what value stream management is about so I think that's going to be...you need tools, of course, to support what we're just talking about, how to automate governance, how to visualize what's going into the pipelines from different stakeholders.

Andre: Yeah and so to do that not only do you need process but you need data, you've got to have a model around that data.

Diego: Absolutely, so it means now you have a chain of tools, right? I mean if you look at an end-to-end delivery pipeline, I just published a Tech Tide, which was an update of a TechRadar of two years ago on continuous software delivery tools. There's about 18 different pieces of technology that in a complete, comprehensive way would contribute to automating…

Andre: It's a lot, right?

Diego: It's a lot yes. And so you start with four or five and then you start adding and becoming better at it and just in testing there's about eight tools that you could use, unit testing, functional testing, performance testing, security testing, and automating all that. So you know in order to pull these tools together, manage them, so manage the pipeline itself, the efficiency of the pipeline itself, having data about that and knowing where to intervene, but also managing what's flowing through the pipeline, right, and so that's what value stream management is about. And so these tools all have different data models, what a work item for one might not be a work item for the other, so you kind of need to..

Andre: And so if they have different data models it also means they have different meanings of that data, right?

Diego: Exactly. Understanding and interpreting so you have to kind of have something that federates or interprets and…

Andre: Transforms…

Diego: Transforms that data so that it can now be visualized. These days we talk mostly about data-driven, insights-driven. I guess that brings us to what might be coming next, which is an insights and data-driven development approach in DevOps.

Andre: Yeah. So you mentioned one of the things that you've watched over the years is testing.

Diego: Yes.

Andre: And it seems to me that testing, automated testing is core and fundamental to the continuous delivery process. What are you seeing there in terms of how that's going for organizations?

Diego: So it's again a people, process, and technology conversation to have. So the first thing that's happened since agile started, I would say the journey and DevOps is completing it or at least making it, energizing it let's say, the concept of the big, large testing center of excellence organizationally is dead. We're seeing lots of organizations moving testers down into the teams.

Andre: So is that because they move too slow in that model?

Diego: Well because that basically keeps that separation of quality is our job, right, and you developer just write your code. I'll look for the bugs and then the finger pointing starts, right, and that doesn't work. We know that. And also it's like it considers testing as an afterthought.

When I mentioned the analogy of the Ferrari there was another reason, which is if you look in Formula One they eliminated pit stops. Sorry, they didn't eliminate pit stops. They turned pit stops into something that keeps the car quality high to win the race. But 10 seconds to change four wheels could be basically, you lose the race. They're talking milliseconds over there or even less than that. So what they did is they shortened from 10 seconds, they changed four wheels in 1.8 seconds. You know you don't even count one, two, four wheels are changed.

Quality has become part of the strategy for winning the race and that's the same that has to happen in agile and DevOps. It is speed but it's really quality at speed. And so to make quality at speed happen is you have to have testing be a problem for everyone. Quality has to be an objective of all the teams, developers and testers, testers work together, developers get on more testing because automation is hard, and we need developers to kind of work on automation, so there is a people change, skills change.

There's a tooling also change. We've gone from tools that just do UI-led types of automation to tools that go beyond the user interface and do API testing, can support API testing. We need to see, you know revert that 80 percent done UI-led to 20 percent API and see that go the other way around, 80 API and 20 UI for automation. Getting developers involved with the right tools, with versioning all the assets for testing so that we can easily deploy things and automate the process. It's shifting testing to the left, it's about not just functional testing, of course unit testing, but also performance testing. I'm writing a research right now on shift left performance testing, which can developers do something about performance early on and can we get rid of some of the really easy, stupid bugs that sometimes are created and that become really hard to fix when they go into production early on in the process? And so that's the big trend that's happening in testing. I think it's being quite revolutionized.

Andre: So what are you seeing from a testing technology standpoint? Is the technology there for people to automate their testing fully or is there a way to go yet?

Diego: So there's a lot of technology. The problem is I think more how the technology is used on one side but on the other the tools have been improving. As I said they've been adding tools that used to do only UI-led automation also are offering now API automation testing. Tools are – they've been scaling more, they've been adapting to agile, they've been coming more lightweight, test management tools have completely changed if not even the old, traditional ones kind of disappearing because they don't support the process. There's security tools that have been around for quite some time that are becoming also more integrated into the CI/CD loop. So I think the next revolution of the tools is what I'm writing about right now, which is the use of more data, using machine learning-driven data to drive the automation and also the augmentation of testers.

Andre: Right. So related to testing is benchmarking and so what are you seeing – how are you seeing DevOps impacting benchmarking?

Diego: That's interesting. So I wrote a bit of provocative research on benchmarking. In the Modern Application Delivery Playbook there's one document we have to write on benchmarking and I don't remember the exact title but it was saying something – benchmark against – don't benchmark against the average industry but benchmark yourself against your own internal competitor, the business, which is not a competitor of course but it was kind of saying, "You know what, you need to – your benchmark of how fast you're going should be aligned with what your business wants, the value that it needs, when it needs it, more than averaging, comparing to the average industry."

That's what I see high-performance teams do. They don't care about being compared to others in terms of we are – first of all benchmarking velocity is kind of wrong – because you can't really benchmark velocity across different teams. We all know that but we still try to do it. So I think there's a new concept around benchmarking, which is you do have to set your own baseline. Yesterday I had a panel here with metrics and there was, I think his name was Dave from Indeed.

Andre: Mm-hmm.

Diego: Yep. He was explaining, and I totally agree with him, that you have to – you just create – generate data, set your baseline – and improve against that baseline, improve the baseline based on what the problems are, what business wants you to achieve. Traditional benchmarking is a bit of a political thing, right? It's just like I want to go to my manager and say, "I need more budget because our competitor X, Y, Z is going at that speed and we can't go at that speed."

Andre: Right.

Diego: It's my view.

Andre: Well great and we really appreciate you being at DevOps World here in Nice, France and we're hoping that you'll be with us in Lisbon, Portugal next year December 2nd through 5th.

Diego: I will. I've signed my 'Save the Date.' I'm enjoying it, I've enjoyed it, and thanks for having me.

Andre: Well great. Thank you very much.

Diego: You too.

Andre: Thanks, Diego.

Diego: Thank you. Bye-bye.

Announcer: Like what you've heard today? Don't miss out on our next episode. Subscribe to DevOps Radio on iTunes or visit our website at CloudBees.com. For more updates on DevOps Radio and industry buzz, follow CloudBees on Twitter, Facebook, and LinkedIn.