Harris Kirk of Wolters Kluwer Leads a DevOps Transformation by "Eating Your Own Dog Food"

Brian Dawson: Okay, hello. I'm Brian Dawson. I'm guest-hosting DevOps Radio live fromDevOps World/Jenkins World 2018. I've had the privilege of being able to get a number of the impressive attendees that we have here at the show in here for interviews with DevOps radio and Harris Kirk a DevOps engineer with Wolters Kluwer is another one of those impressive attendees. Hello Harris.

Harris Kirk: Hi Brian, nice to be here.

Brian: How you doing?

Harris: I'm doing very well, thank you.

Brian: Good. Can you start by giving our listeners a brief overview of your background, introducing yourself to our listeners?

Harris: Sure. Briefly I started out life as an analytical chemist. Brian: That's interesting. Harris: I found out I hated chemistry and really loved programming, so I became a Java programmer for pharmaceutical companies and healthcare companies and then found I liked to do DevOps automation work, so I kind of transitioned into DevOps that way.

Brian: Okay, awesome. Thanks for sharing that. I actually have a question that I want to ask but I cannot resist asking about how prevalent DevOps is in software development and best practices are in the pharmaceutical industry?

Harris: They are quite prevalent as in any other industry. I'm not in pharmaceuticals right now, but I am in healthcare and that's what I'm actively doing and I was here presenting earlier, so we're quite active in that space just as most other companies are.

Brian: Just as like we say every business is a software business that applies to pharmaceuticals and healthcare?

Harris: You got it.

Brian: Awesome. Okay, so you're actually one of the presenters here at DevOps World/Jenkins World, correct?

Harris: Yes.

Brian: Can you tell us a bit about your talk?

Harris: So it was a talk about how although we promote the principles of continuous delivery to our audiences, to our customers, to our development teams. I would also like to think that we develop our own software, the DevOps software and tooling that we put in place using those same practices, continuous delivery and continual testing and fail fast. And so the talk is about an application that creates Jenkins jobs automatically, but it itself is a continuously built, tested, and released piece of software.

Brian: Ah, so there's actually a correlation. Did you catch this morning's keynotes?

Harris: Yes.

Brian: What did you think of the configuration or the Jenkins configuration code Kohsuke talked about?

Harris: Oh, I was salivating because that's really what I'd like to get to, where I don't need to worry about setting these things up, where I can immediately jump into code and creating things on the fly dynamically. So I'm eagerly looking forward to progress in that area.

Brian: Awesome. And then I know it sounds like you've been busy, keynote, talk,now you're here doing a DevOps Radio interview. Don't know how much of the show you've gotten to experience, but I'll still ask what do you think of the show so far?

Harris: I'm enjoying it a lot.

Brian: Have you been to other Jenkins Worlds?

Harris: Only a couple actually. I just got out of the Director of Engineering at CloudBees,the woman presenting about Docker containers, which was fascinating.

Brian: Oh okay.

Harris: So yeah, I'm enjoying that a lot.

Brian: Awesome, awesome. So this year's theme at DevOps World/Jenkins World is transform. What was the impetus for your DevOps transformation or have you guys – first I guess I should check have you guys gone through a DevOps transformation?

Harris: We are absolutely. We had a large application affectionately known as 'The Blob' and it was very difficult to maintain and deploy. We wanted the ability to be nimble and take parts of the functionality and deploy them more quickly so we needed to get into microservices, we needed to get into containerization. We are well along that path so the application I described earlier is one part of that. It allows us to create jobs that will deploy things more quickly.

Brian: So I'm curious, let's go back a little bit as we talk about the need to create microservices and use containers for that. Now, sometimes we run into a chicken or egg versus the chicken versus egg thing, right? We see some people that adopt continuous delivery or they decide they're going to implement continuous delivery and in order to deliver faster they have to decompose apps into microservices and leverage containers, or I've also seen the reverse, right? We've decided that we need to decompose our application to make it more manageable. Hey, now we can apply some of these CI/CD best practices. Did one come before the other with you guys?

Harris: No, the decomposition clearly came first. We are not continuously delivering these microservices yet. We are still in the process of decomposing and taking our time to deploy them as we have time to do that, but that'll be the next step. Ultimately yes,we'd like to get to continuous delivery just as we are with the application I spoke of earlier for our Jenkins jobs, but we're not there yet.

Brian: You're not there yet, but microservices and containers are providing you the ability, the opportunity to go faster.

Harris: Yes, right.I mean it's a multi-step process as I'm sure you know, but it starts with containerization and we are well along that path for our microservices.

Brian: Awesome, awesome. So what tools or processes would you say are necessary for an organization to have or consider if they are going to undergo a DevOps transformation?

Harris: Well, I know people like to talk about tools and processes and they are important, but let me offer,the first step is kind of a cultural mindset that has to change. The ability to be able to fail early, quickly, and have that an accepted practice is key. It's important to be able to not have to wait two weeks or two months to get something into a production-like environment. So regardless of what tools you use, the culture and the mindset has to change where things aren't thrown over the wall.

Brian: Now do you have any tips for our listeners as how to go about...well first let me say I also agree. We often say DevOps is really more about a culture and there are technical principles and practices that help you achieve or recognize that culture. But I also think that that cultural component of DevOps is the soft science. It's one of the hardest things for us to overcome.

Harris: It is.

Brian: In your experience are there any accelerators, tips, or tricks?

Harris: Yes. I would say definitely getting things into version control and typically into getting having many things run through pull requests, be able to review that and I've heard other people talk about that and it's absolutely true. It's the ability to visualize the change from one human being to another rather than some email that somebody was going to do, but actually collaboratively see the change together and say, "Yes, this is what we're going to do."

Brian: Yeah, that's actually an awesome point that I probably under-consider. I almost, I don't mention enough, we probably don't talk about it in the industry enough. GitHub and pull requests starting collaboration around the code kind of fundamentally starts a cultural shift.

Harris: Absolutely and I can't tell you how many times somebody has walked into my office and they said, "You know I sent you an email about this change and blah, blah, blah," and I will stop them right there and say, "Let's look at the pull request together right here while you're in the office."

Brian: Yeah. So interesting, at CloudBees we have product marketing managers, community managers; there's virtually everybody within the organization within CloudBees knows how to go into GitHub and issue a pull request or review a pull request, so I actually have to thank you for bringing that up and reminding me of the importance of that. Any other tips in terms of building a collaborative and experimental culture as a foundation?

Harris: Right, incremental changes, small changes. It's amazing how in my whole lifetime I found even back in my days as a chemist, we want to overanalyze and overdesign things and smart people do, they want to build complex things. But the ability to slice it finely and do a small change and get that into a production environment. Incremental changes would be the other.

Brian: Yeah, I can appreciate that and it's probably again, yeah, for people with a master's in computer science, it's about careful sort of risk-adverse engineering. It's very hard. I also do think though that a segment of that population is also very data-driven, right? And if you work in tight, iterative loops where you're getting feedback you actually get to operate better informed, you get to operate based on real data so your architecture and design can be based...

Harris: That's also a very good point. So in addition to the socialization and collaborative aspect, there's the feedback loop so I totally agree.

Brian: Awesome. So what are some common obstacles that organizations encounter during a DevOps transformation? In particular, if you have anecdotal example it would be great to hear and how do people overcome those obstacles.

Harris: So, I'm not sure if I can give you the best examples, but clearly back to what we were talking earlier is building large, monolithic applications is an anti-pattern and we all tend to do that. So, kind of the flipside of what I was mentioning earlier is the small changes, everything in version control, collaborate through pull requests; those are the practices to follow. So an anti-pattern would be using a version control system that doesn't support that at all. I know in my previous life we used another tool, large, monolithic tool that didn't allow that, so that made things more difficult.

Brian: So let me ask, so is part of your role with sort of continuously delivering your continuous delivery system that puts you in a position where you're supporting other teams within...

Harris: Yes. Brian: Within Wolters Kluwer?

Harris: Yep, that's right.

Brian: And have you found any common themes across those teams with either things that they are, I'd hate to say it, doing wrong or not doing optimally or places you find resistance to this modernization or change? What learnings have you got from the teams that you work with?

Harris: Well what we are finding, and the company has acquired several small companies...

Brian: Ah, that's always great.

Harris: And what I do find is a lot of people, they are happy to have someone else handle the DevOps for them. So if they have systems in place they are very motivated to bend their processes to support something that's already in place like our Jenkins jobs, our automated creation of those jobs, do certain things and they are motivated to bend them.

Brian: Oh, so that's interesting. So people will arguably give up a little bit of control to get the benefit of offloading all the stuff that they don't want to do so they can focus on what they want to do.

Harris: Right, correct. Brian: Awesome, awesome. I'm going to go completely on a tangent because it just popped in my head. I'm really, really curious; this one may be a hard one to answer. Okay, so you come from an analytical chemistry background, right? Harris: A long time ago, yes.

Brian: You had a stint as a chemist, you found that you were more attracted to programming or software engineering. I am curious, and this is a stretch, is there any correlation that you can draw from your experience as a chemist to the transformation that the software development culture and environment is going through now?

Harris: Well the one that comes to mind is what I alluded to earlier is the overdesign of equipment. I mean I remember automated titration equipment as a chemist that was all becoming very automated.

Brian: What was that, titration? See now you've gone right over my head.

Harris: Yeah, there's a process called titration in chemistry, yes, and there's automation for that, but the early ones were incredibly complex and over designed and so that was a lesson I remember taking away thinking, "Does software really have to be this complicated? Do we have to make it this hard for the end-user to use when 80 percent of the time they only want 20 percent of the features?”

Brian: So that's an interesting point. So is that a lesson that you've incorporated in designing CD systems?

Harris: Oh absolutely. This is absolutely part of the 'you ain't gonna need it' principle, waiting until you need something to deliver it. Small, incremental improvements don't add more value than you need at the moment so it's part of Agile. Brian: Right, it's more in line with avoid premature optimization . Harris: Yeah, yeah, absolutely.

Brian: Take user experience into account even when you're building CI/CD systems because I assume if it's not a pleasant experience even these companies you've acquired that are happy to hand over systems if the system you deliver to them is overly complex and a pain to work with, they're probably not going to adopt it.

Harris: Right, whereas if it's easy to use and easy to set up as we like to think it is, they will be motivated to use it, yes. Brian: Awesome, great insight. Thanks for sharing that. So what are some indicators that a team or an organization is succeeding at DevOps?

Harris: So the single biggest metric would be the cycle time, which would be the time from a developer making a commit into source control to that software being in a production-like environment. So that's probably the most important metric in software.So an indicator that we're doing well is if we see that cycle time shrinking.

Brian: Okay, okay. Now this is a tough one but I'll ask, is there a standard number that you want to see for that cycle time?

Harris: No, ideally immediate would be an ideal, but no. I mean I think every organization has their own numbers. I mean we have released four times a year so we're trying to get that number down.

Brian: Okay. It's all relative and I'd say some organizations,you know we talk about Amazon and Netflix committing thousands of times, delivering thousands of times. Not every organization...

Harris: Exactly, measured in seconds or minutes right? I mean that's the ideal.

Brian: Right, that would be awesome, but not every organization needs it.

Harris: No, no, and there may be good business reasons why they don't. There could be compelling reasons why they have the ability to deliver sooner, but they choose not to.

Brian: Right, awesome. So you had a panel or lightning talk right here at DevOps World focused on delivering enterprise architectures, which you just talked about. Was there,and actually we're going to roll this back because that would be redundant. I'll take that back.

Harris: Okay, sure.

Brian: So you've mentioned the phrase in the past, “eat your own dog food.”Can you define this for our listeners and kind of elaborate the importance of this or what's meant by it?

Harris: Well okay. So I have found that it's often best to lead by example, so I'm happy to eat my own dog food. I mean it's one thing to be in an organization and tell other developers that you know you really ought to be testing more often and releasing more often. It's one thing to do that, it's quite another to do it myself. So I try to use my own principles with the software I'm designing so that's what I mean when I say, “eat my own dog food.”

Brian: Does that give you,I could see, I'm curious that one, that gives you new and additional insights, right, as to what your user experience is? Does it also give you credibility with your users?

Harris: I like to think so. It makes my job more enjoyable so I do it for my own reasons, but I also do it as a sense of what challenges are there? Maybe I'm advocating something that's actually a lot harder than I think, or it could be that I'm actually finding ways to do software development that would help them as well.

Brian: Right, right. Now going back to your talk from earlier ,I had you and I and the listeners may not know, we met earlier. I'm a track chair for your talk and I was curious to see how your talk went, you had a short amount of time. I think I asked you earlier if you got time to get Q&A in and you said yes.

Harris: Yep, we did.

Brian: Are there any questions that the audience had after your talk that stood out that you could share with us? Harris : Well one of them was about whether we allocated a specific Jenkins test instance just for the testing and I said, "Yeah, we do." So we had a separate Jenkins test instance that not only all the jobs are laid down on, but many of them are actually run as a kind of a smoke test.

Brian: Oh, to make sure I'm getting it because I did not get the privilege of being in your talk, this is your pre-prod environment for your Jenkins deployment itself.

Harris: Correct. Brian: So you guys have a copy of your jobs that run in your production Jenkins environment? Harris: They are actually created from scratch, the jobs are created, the application creates the jobs on the Jenkins test instance so that process of laying them down is tested and the jobs themselves, not all of them, but representative ones are tested all before creating them on the production Jenkins.

Brian: Ah, okay. So going back to your major point, it's providing sort of software development and delivery best practices to your development and deployment of your Jenkins instance.

Harris: Right and that happens all automatically and a commit to master lays them on test, tests them, and if they're successful, the code is tagged – and rolled into production all automatically.

Brian: Awesome so I know I'm going to have to go back and watch the recording of your talk and I think the slides will be posted for the listeners. What's the name of your talk again so they know where to go?

Harris: Pipeline of Pipelines, Eating Our Own Dog Food.

Brian: Okay, Pipeline of Pipelines, Eating Our Own Dog Food. Well we're starting to get towards time. Before I wrap up, I just want to ask do you have any final thoughts for our listeners and/or in particular with our theme being transform, do you have any final thoughts for people that are in the midst of or embarking on a DevOps transformation?

Harris: Definitely. The points I made in the talk were to start small, test frequently and refactor frequently. Socialize the code. So start small, socialize the code, refactor continuously and test continuously so those are the starting points to not be afraid to start.

Brian: Awesome. Thank you Harris. So that concludes our time here with Harris at DevOps World/Jenkins World 2018. We appreciate your time.

Harris: Thank you Brian very much. Bye-bye.

Brian: Thank you.

Announcer: Like what you've heard today? Don't miss out on our next episode. Subscribe to DevOps Radio on iTunes or visit our website at CloudBees.com. For more updates on DevOps Radio and Industry Pause follow CloudBees on Twitter, Facebook, and LinkedIn.