Michael Neale, Rob Davies, James Rawlings, James Strachan - The Brains Behind Jenkins X
In Episode 39 of DevOps Radio, host Brian Dawson talks with the CloudBees' developers and brains behind Jenkins X including co-founder Michael Neale, Rob Davies, James Rawlings and James Strachan. The team discuss why Jenkins X is important for developers who want to create cloud native applications for Kubernetes.
Brian Dawson: This is Brian Dawson. I'm guest opening DevOps Radio for Andre Pino. We are broadcasting live from DevOps World/Jenkins World 2018. I have an awesome group of guys with me, here. I'm not gonna blow it by introducing you myself, so what I'm gonna ask is, can you guys go ahead and introduce yourselves? We'll go ahead and start with Michael.
Michael Neale: Yeah, hi. Michael Neale. I was one of the co-founders of CloudBees, and I am too lazy to learn Kubernetes, which is why I'm interested and got involved with Jenkins X. And we've got another person named James, joining us now, so just ignore him.
Michael: And this is another James, here.
James Rawlings: Okay, so, James Rawlings. The good James out of the two Jameses. I was involved in creating Jenkins X with these fine folks, here.
Rob: Hi, I'm Rob Davies. I'm VP of Engineering for CloudBees. I've been here eight months, and I've been working on Jenkins X.
Brian: Awesome. Thank you, Rob. So, I'm just gonna ask you guys as a group—there’s been a lot of talk about Jenkins X. What the heck is it, and why should we care? Who wants to take a stab at that one?
Rob: Excellent. Jenkins X is something that we've been thinking about for a long time. We worked on the fabric8 project.
Rob: And that was to—
Michael: That was it when you were at Red Hat, right?
Rob: Yeah, that’s when we were at Red Hat. But the basis behind fabric8 is to try to make it really easy for developers to pick up and use Kubernetes.
Rob: So, we had the opportunity when we came to CloudBees to actually rethink that and rewrite it and actually be more opinionated about how we were gonna do delivery. So, there’s things we've built into Jenkins X which we adopted from a DevOps report about best practices.
Rob: So, it’s an opinionated approach, but there’s two facets to Jenkins X. One is actually making it extremely easy for developers to take their applications, and then deploy them onto Kubernetes, and then build their applications and provide all the plumbing around that. So, they don’t have to know about Kubernetes.
Michael: So, you don’t have to be an expert in Kubernetes.
Brian: So, it works for you.
Michael: I keep asking that, and I'm glad, because I'm just worried one day they're gonna say, “Actually, you do need to know it now.”
James R.: I lied all along.
Michael: Yeah, so that’s comforting, because it’s pretty overwhelming.
James R.: Yes, absolutely.
Rob: Well, one of the things is really about the on ramp to Kubernetes. There is a lot of steep learning curves, there.
Rob: So, I think one of the things we wanna do is talk about terminology that developers are familiar with, like applications and environments.
Rob: These are not things that are native to Kubernetes, but they can—they do map to things in Kubernetes terms.
Rob: So, if we can get people on there—developers on there, teams on there, working on there, right? Get their applications built, developed, and running on Kubernetes, and then, as people become more mature, they can learn more. And then, they can actually peel behind the curtain and use more Kubernetes-native concepts as well.
Brian: Okay, so you may need to eventually be an expert.
Michael: I might have to eventually learn it. Not for any time soon.
Brian: But not initially.
Michael: Yeah. So, someone asked me at the booth yesterday, I think.
Michael: They said, “I know Kubernetes, I have it running, and I can get Jenkins running really easily on Kubernetes. It’s a very flexible platform.”
Michael: And he said, “Well, what does Jenkins X give me?” And I kind of know a lot of the answers, but what’s the short answer? Especially for the type of crowd here that might be seeing it for the first time? Because it’s not really about running Jenkins, is it?
Michael: It’s about apps and environments, microservices.
Brian: Well, let me add in, before you answer, let me just layer on top of that. But it’s not also necessarily just about running Kubernetes, either, right?
Michael: No, right.
Brian: It’s not running Jenkins, it’s not necessarily just about running Kubernetes.
Michael: No, they're both means to an end, so what is it really about? What’s the end?
James R.: It’s actually about having this cohesive development environment. So, end-to-end, so taking an application, how you actually code, and then being able to provide a complete continuous integration, continuous delivery. So, we've got the stages in there for continuous delivery. And it’s more than that. We provide the infrastructure so you can do these builds. We provide an access so you can store what you're building without...
Michael: And I don’t have to install that, that’s installed for me, yeah.
James R.: That comes from Jenkins X. So, out of the box, we've got a very nice, concise command line interface. So, actually, if you installed Jenkins X on our cluster, all you have to do is do a couple of blurbs in a CLI and it’s installed for you.
James R.: For example, on GKE, if you want to create a cluster on GKE, it’s just jx create cluster GKE. I mean, we will do everything for you, install all the infrastructure you require, including setting up the actual cluster for you.
Rob: And one of the interesting things about that is, we are constantly delivering. We're using Jenkins X to build Jenkins X. So, continuously delivering. And one of the cool things is, is the amazing technology and innovation that’s happening around the Kubernetes ecosystem. So, when new projects come up, we think they will benefit users of Jenkins X, people want to develop software on Kubernetes. We can actually add that in, maybe replace certain things as well...
Michael: And the benefit I get if I put on my user’s hat is, I don’t have to be an expert or fear that I'm missing out on these new things.
Michael: Jenkins X has kind of picked those, and I can do a jx upgrade..
Rob: Yeah, exactly.
Michael: And I'll get whatever version of Helm that has the thing without Tiller, I don’t have to know all the details.
James R.: I think another thing that’s interesting is, as a user of the cloud, they're all very different, right? Amazon looks very different to Azure, that looks very different to GKE.
Michael: Right, yeah.
James R: They all use different tooling.
Michael: He’s cuddling up to get closer to the microphone. It’s very...
James R.: They all use different tooling, they all use different approaches. We give a simple CLI that works with all the different clouds. So, if ever you need to move from Azure to GKE or GKE to EKS...
James R.: The same commands work. So, rather than having to learn all these different ways of working with Kubernetes on all the different clouds, we give a nice simple, common way of doing it.
Brian: So, you guys have mentioned a couple of key words, right? It’s lowering the on-ramp, right, reducing the learning curve, an opinionated solution. You've talked about portability or fungibility. Can someone tell me why—how is all of this important to an overall broader DevOps strategy or the overall state of DevOps?
Michael: I can tell you that.
James R.: Next question? There was a reason you didn't get a mic.
Rob: So, the State of DevOps Report and the Accelerate book has detailed what are the best practices of high performing teams.
Rob: And Nicole’s speaking tomorrow about the Accelerate book and all of the findings.
Brian: I'm sorry—Nicole who?
James R.: Nicole Forsgren.
Rob: That’s it. Dr. Nicole Forsgren.
Brian: Dr. Nicole Forsgren, okay.
Rob: Yeah, and she’s speaking at 11 tomorrow, and there’s a whole bunch of best practices which high performing teams have been proven to adopt, which has a dramatic difference in velocity of value. Like, 2,500 times faster in high performing teams versus low performing teams. And those are things like automating CI/CD, using strength-based development, using source control.
James R.: Things like experimentation as well. So, if things are a little bit more woolly, then how do you help developers experiment? One of the things we've actually, these occur in Jenkins X is about Preview environments.
James R.: So, we've moved—it’s one of the capabilities as well. It’s one of the features, I guess, that’s quite exciting around Jenkins X is being able to dynamically provision an environment.
James R.: We deploy a change in there from a pull request and actually see that change being deployed and running. You run automated tests on it, then you can also collaborate as a team. You can experiment on a pull request before that change gets moved into market.
Rob: You get fast feedback.
Brian: Ah, interesting.
Rob: Without preview moments, you’d have to look at the pull request and go, “Oh, my God, what’s that Git command to clone a fork in a pull request?” Spend 20 minutes on Googling for that and then figure out the CLI, then building everything locally, then figuring out how to run it locally, to, “Oh, yeah, that looks...”
Michael: Yeah. That’s been my life for years, and I know I've assembled in the past ways of doing preview apps, I've customized pipelines, but what I like from the user point of view of Jenkins X is, I didn't have to write my pipelines. I just ran jx import, and I picked the right pipeline, I got preview apps for free. If I knew enough about Kubernetes, I could probably do it myself.
James R.: I mean, you can do everything yourself, right? But high performing teams don’t do everything themselves. They automate everything. So, automating everything is really important, but also using the cloud well is another high performing trick.
Brian: So, another sort of correlation—a couple of things, right? We've used the word opinionated.
James R.: Yes.
Brian: Why is an opinionated approach to CI/CD important?
Michael: Well, I think it’s...I think it’s, there was a book written in, published in 2000 called Don’t Make Me Think. And that introduced the concept of satisfied—I think that’s the word, satisfied....Where people would just, they would try out enough different things until they found something that was good enough, it crossed a threshold and they go, “That’s what I'll do.” And that might be an inferior solution, or in the case of DevOps and CD, there’s so many tools out there, and every single day, there’s something new. There’s something new on Kubernetes, there’s something new in Golang, and it’s exhausting. And you never finally get that threshold, so you never get anything done.
Michael: Or if you do, you feel like you've done something inferior for this crappy, handcrafted pipeline. So, to me, the opinionated means that you've got a whole bunch of people, a whole community, actually, now hundreds of people contributing from years of learned experience—this is the right way to do things for you. There’s buildpacks that will choose the right pipeline automatically for you. So, it’s just—to me, it’s just a way to cope. I don't know what to do otherwise.
James R.: One of the driving things for Jenkins X is that we can, there are so many people who just want to actually deploy, after they've built their applications onto Kubernetes, and they didn't want to know all the hard stuff.
Brian: All the other steps to do it?
James R.: Yeah, we just wanna make that really easy for you.
James S.: For an example...
James R.: Another quick point, though, is, it’s opinionated.
Michael: Sorry, James. You'll get a turn.
Rob: You're next, you're next.
James R.: It’s opinionated, but...
Brian: Well, I have a question.
James R.: It’s not just, “We have this opinion.” The opinion is the State of DevOps Report proven, scientific facts.
Brian: That’s what I was gonna say, it’s not..
James R.: This is what high performers do. It’s not our random opinion.
Brian: It’s not an opinion, it isn’t a guy from up above saying it.
James R.: These are proven, high performing teams.
Michael: There’s a lot of data backing these decisions, and you can get on with building your app.
Brian: And focus on what you need to do. So, before we go to Rawlings in a minute, and I'll get to you for a comment—for people, we are on DevOps Radio. I just want people to know that Rob Davies and James Strachan.
Rob: We're hugging.
Brian: Are hugging and this is the collaboration of the Jenkins X team.
James S.: Yeah.
Rob: We're always hugging.
Michael: Yeah. It’s a James Sandwich.
Brian: So, I’d like to ask Mr. Rawlings your comments on opinionated. But then after you comment on that, can you also tell me a bit about GitOps and how GitOps plays a role in this overall picture of Jenkins X?
James R.: I think that’s actually—it was kind of one of these light bulb moments at kinda the start of the year. Because Rob was talking about fabric8, we actually managed to, when we joined CloudBees, we managed to re-look at how we were doing everything, all the lessons learned. GitOps was one of those light bulb moments. Actually, you pinged me this horrendous e-mail that was, I tried to digest it.
James R.: Don’t change that. I was like, “Whoa!” And then right at the end, I thought, “Oh, that’s brilliant!” The idea is that, in Kubernetes, most people actually go into an environment and they do kubectl apply. That’s how you deploy an application.
James R.: Which is okay for fun, but when you're talking about running real production workloads, real changes, you don’t want to be going into that environment and people making changes. You don’t want to be changing the environment variables or tweaking memory limits, for example. Because it’s not repeatable. You don’t have a traceability for that. So, using Git or GitOps is an approach of actually, every change to an environment that actually has to be accepted into a Git repository. So, a change is proposed and a new application or a new version or a memory tweak, for example—that configuration change and sequence as well will actually get proposed most into the Git environment and then massively deployed out. And of course, we have, Jenkins X has automation to actually create pull requests, do auto merges and finish other tasks.
Michael: So, the state of your system is always in a Git repository.
James R.: Exactly.
Michael: There’s always an audit log, there’s access controls. It’s kind of—
James R.: It makes sense.
Michael: Advanced shops and advanced operators have been doing this for years. And this is kind of now—even I can do it. Even I can do it.
James R.: But then, what’s also kind of the really great thing is, we can have—let’s say you have pull requests on an environment. You can now have CI for your environment. So, you can start having—okay, you're running a certain number of checks, quality checks, governance, for example. That’s where CloudBees is certainly looking into having more assertion that those changes to an environment, it actually fits the purpose going through.
James S.: As we roll out new changes of Jenkins X, we could do CI on your CI/CD system. We could do a Preview environment that spins off a Jenkins X install in the Preview environment and then runs your pipelines to check the new version of Jenkins X works as well as the previous one. So, we could CI your CI.
Brian: We could CI your CI.
Michael: And what’s better than—yeah...
Brian: So, this actually brings up, this is a good segue for me. I'm gonna put some things in. So, we had, you guys were all at the keynote, many of you participated in today’s keynote, right? We heard Kohsuke talk about configuration as code.
Brian: We also heard about this thing Kube CD.
Brian: And that’s gonna lead—and I'm gonna start by directing this towards you, Rob. How has what you guys are doing with Jenkins X and the concept of Kube CD and configuration as code—how is that being represented in CloudBees’ overall portfolio of products? It seems like containers, microservices, cloud native are becoming much more important. Can you comment from a CloudBees perspective on where that fits into the product portfolio?
Rob: Sure. I think containers and the cloud and Kubernetes are really important for the industry. We're just following that and providing our products to actually represent that in the best way. So, if you've got today this Core which is for the main subscription, Kube CD, which is supported Jenkins X with a UI which gives you the power to control your Jenkins X.
Michael: Someone was asking me that today. So, Kube CD is the UI that’s a feature of Core that works with Jenkins X. So, that’s the difference with Jenkins X, right?
Rob: A supported Jenkins X.
Michael: Supported Jenkins X.
Rob: With UI.
Michael: User interface.
Rob: And it’s a feature of Core.
Rob: So, if you've got a Core subscription, then you're entitled to get Kube CD as part of it.
Brian: Okay, awesome.
James S.: And another small detail is, Kube CD is an add-on. So, we don’t fork Jenkins X, there only is one Jenkins X.
Brian: Okay, right.
James S.: Kube CD is an add-on commercial product.
Michael: So, you don’t have to uninstall a version of Jenkins X and install...
James S.: Exactly, and then switch to something else.
Michael: You know, like...
James S.: It’s the same Jenkins X. You can add on Kube CD or you can take it away again for whatever reason.
Brian: Okay. I'm gonna challenge this though, and actually, I'm gonna go to James Rawlings for this. Okay, so, if I tend to work in the command line, I'm issuing pull request by command line with Jenkins X, if Kube CD is just a UI on top of Jenkins X, why do I need it? Why do I care if I'm comfortable with the command line?
James R.: I think—well, from a, as a developer, as a single developer worker on the project, then you're working in your ID or you might be working in your terminal—which is fine, which is great, which is where you wanna be.
Brian: Or Visual Studio.
James R.: Or if you have VS codes.
Brian: The right plug-in, then?
James R.: Yeah, we have...well set up, yeah.
Rob: All the intelligent—intelligent as well. We have intelligent...
Brian: We're plugging the plug-ins.
James R.: We're plugging some plug-ins. So, you can actually have pipeline status updates actually in your ID tooling. Which is fine if you're working locally. When you're talking about this microservices world where you're writing 10 microservices, maybe 100, maybe 1,000, maybe 10,000 microservices.
James S.: Maybe 500 developers.
James R.: Five hundred developers, also fixes or integrations with understanding where your changes are? Then, that’s really, really hard to actually visualize. So, you need some kind of visualization to get to do that.
Brian: Right, okay.
James R.: Also, your environments and how things are actually operating and promotions, maybe, you might want to be....
Brian: Well, now—because I do have to let in, I know a little bit about it. So, you're actually, you also—you need a UI to visualize it, to manage it at that scale, but you also manage it through Kube CD, right?
James R.: Yes.
Brian: It’s more than just visualization?
Rob: You can do promotions and rollbacks and stuff.
James R.: Yes.
Rob: Like, what’s—Jenkins X, you can do everything from the command line. But promoting to production on the command line is a slightly scary—
Rob: Promote version 100—oh, I missed the dots!
Rob: Oh, crap, I've just gone back—I meant 10.0.
Rob: Like, it’s easy to do a typo on the command line.
Rob: Kube CD has a nice, simple button where you click promote and it promotes, and you see exactly what’s being promoted, what issues are being fixed and whatever. So, it gives you that sense of security and ease of use.
Michael: Or you could have your product manager breathing over your shoulder. Like, “When is this really important thing that makes the curve just that bit smoother?” And I'm talking from real, lived experience, here. They got really antsy if the curves are not right and they wanna—you're not gonna be allowed to leave, and they can see that that version with that issue ticket in there—
James S.: Is done.
James R.: Is done, yeah.
Michael: Yeah, and then they can believe it’s done—right, yeah.
Brian: Alright, so, we're starting to get towards time. I actually would love—I mean, I could spend hours with you guys, but you guys are celebrities and you have to go sign books, I believe. Let me ask, what is—I heard something about your clothes being torn, Mr. Strachan? That’s another conversation—NSF....NSFW. What’s next for Jenkins X, guys? Who wants to take that?
James R.: Serverless Jenkins.
James S.: Aww, shit you...
Michael: He promised he would say that.
James S.: Oh, okay. Do you wanna talk about Serverless Jenkins?
James R.: You go first.
James S.: Well, there’s two things—Serverless Jenkins and?
James R.: Prow. Prow enables Serverless Jenkins. So, what it is, it’s part of the Kubernetes ecosystem. We talked about it briefly. There’s so much amazing stuff coming out of there. One of the things, which kind of is a bit underestimated, I think, or not as well known, is a project called Prow. On all the Kubernetes repos, they're actually using Prow, and it still uses it in a couple of others. What it does is, Prow is essentially a webhook event handler. The whole Jenkins thing, just to take it a little step back—Jenkins X is a journey for Jenkins moving to the cloud. We want to be using, leveraging more of the cloud, as much as possible, and also other projects that actually make sense.So, what Prow is actually enabling us to do is, rather than using, say, the webhook event handling Jenkins, we can actually do, defer that to a lightweight Golang project which is running all the time. Highly available, scalable.
James R.: Which means we can actually then look to Serverless Jenkins, which—
James S.: Or tear down the Jenkins when it’s not being used.
Michael: Yeah, so when it’s night time—or daytime as the case may be, or a long, boozy lunch? There’s no resources being consumed.
James S.: And you don’t lose the webhook, right?
Michael: Yeah, webhooks.
James S: So, you can take the Jenkins down, you still get your webhooks, and you start it back up when you need it. So, we can shave loads of money off your infrastructure costs, right?
James R.: That is huge, because we've heard stories—I mean, they're always happening, but we've heard stories of—
Michael: I just have to take a photo, sorry.
James R.: Yeah, carry on. People running 100 masters, people running 100 masters of different things, right?
James R.: And if builds aren’t happening, you don’t really want a 2 Gig JVM running, shooting up your cloud bills.
James R.: What you want is to have, be notified when that needs a build, when you need a build, and spin up that part, do the build, and then go away. We're doing some great things around Cloud Native Sig as well.
Michael: Yeah. That’s going to...cloud native Jenkins.
James R.: Exactly.
Michael: People have heard about it this morning. We're actually gonna...
James R.: Yeah, yeah. So, in answer to your question, there is huge excitement around the capabilities that the cloud is giving us. And the journey of what’s next on both Jenkins X and with CloudBees Kube CD.
Brian: Awesome. Thank you. That is make for a great segue. We're gonna do a lightning round. We're gonna go around and get final thoughts from everybody. Since you came last, you have to go first—last in, first out. So, Mr. Strachan, any final thoughts for the audience here at DevOps/Jenkins World, or those pursuing a DevOps transformation?
James S.: Firstly, I think most people are gonna be on the cloud soon. They could be on-premise, they could be private cloud, they could be public cloud. As soon as you start even talking about the cloud, Kubernetes is the thing you should be using to deploy your apps. So, Jenkins X is what you should be looking at first. Don’t try and figure this stuff out by yourself, use our automation. And then, longer term, a Serverless Jenkins feature we're looking towards is really exciting.
Brian: Yeah, that does sound...
James S.: Build your CI and CD infrastructure in a serverless model. Pay for what you use.
James S.: Don’t run loads of Jenkins masters when you don’t need them.
James S.: Pay for what you're actually using.
Brian: Okay, awesome. Thank you. Next, we're gonna go to Mr. Davies. What are your final thoughts?
Rob: Yeah, so Jenkins X has been on this journey, and so has Kubernetes, but the variety of ecosystem hasn’t stopped, so we're introducing new technology all the time. I'm gonna be watching that, and we're gonna be incorporating that in the Jenkins X project. I'm thinking of particular projects like Istio, which is 1.0, just recently. We're gonna be weaving that into how Jenkins X works and how we can shape traffic to get better feedback on preview environments, et cetera, to applications. So, it’s a journey, and we just started.
Brian: Okay. A lot more to come, I am excited. Mr. Rawlings?
James R.: Oh, I was just trying to think then and think of something really good to say, and I think... I think one of the main things around Jenkins X which has blown me away over the last eight months is the adoption we're getting around the community. I think one of the things I wanted to call out, though, is we have a lot of attention in the Kubernetes community. This is a story that’s gonna affect all of us, whether that’s traditional Jenkins users or the Jenkins community. What we're gonna need is almost like a shout-out for help and for all of us to be involved in this cloud native journey going forward. So, I think it’s an amazing—truly amazing technology, the things we're actually using. The potential going forward is so exciting.
James R.: The innovation that’s happening, people can be a part of it. And it was really just a shout-out to people to get involved and actually come together because there’s some really amazing things that we're gonna be doing.
Brian: That is a great point—awesome, awesome. And then next, to Michael, and I also wanna, before you start, I also wanna remind that, as you said earlier, you're a co-founder of CloudBees. So, as we come here and we're talking about Jenkins X and cloud native and the event is moving to Moscone next year—as you talk about final thoughts for Jenkins X, I’d also like to hear your observations about the journey of CloudBees and the DevOps movement.
Michael: Alright. Well, firstly, on Jenkins X, I guess my final thought is more of a recommendation, like, if you're a developer, you should go to jenkins-x.io and look up DevPods. There’s something called DevPods, which is something that James invented that was sort of, it seemed like a good idea, and I think it'll change your life. Like, it’s—you just gotta look it up to understand what it’s all about.
Michael: I think that’s something, if you're a bit skeptical, look up Jenkins X and DevPods.
Brian: Jenkins X, so go to jenkins-x.io.
Michael: Jenkins-x.io, and then DevPods is somewhere in there. Or just Google Jenkins X DevPods, there’ll be a blog or something you'll find.
Michael: It'll change your life.
Brian: Awesome. My life needs changing, so I'll go check it out.
Michael: I mean, on the CloudBees journey, yeah, it’s been pretty crazy. Like, it’s—you know, part of me thinks, “Why does anyone care about all this stuff? It’s just software, it’s just building stuff. Why are there thousands of people here all obsessing over building software?” But, you know, it’s a thing. There’s software in everything now, you can’t avoid it. It’s in your light bulb, it’s in your thermostat, it’s in your car, it’s in your blood pressure monitor, and everyone wants to update it all the time, every day. Things that—people don’t build hardware any more, they're just building software. It’s kinda terrifying when you think about it, but boy, it’s kinda great for business, too. But it’s where the industry’s going, so I think this is a reflection of how important software is, so.
Brian: Well, I think you bring up—it’s kind of terrifying, but you're doing something. The company you helped found and you guys all participated should help reduce some of that fear.
Michael: I hope so, yeah.
Brian: And, we hope, make software better, right?
Michael: Yeah, yeah, yeah. I think so.
Brian: So, hopefully, you've done a good thing.
Michael: I think so, I think so.
Brian: You think so? You mean you know so.
Michael: Oh, my God, yeah.
Brian: Anyway, with that guys...
Michael: What have I done?
Brian: If you guys would like to say bye, we're gonna go ahead and sign off. Thank you, guys, for the great stuff you've done with Jenkins X. Thanks for your time, here. You guys have not only made this podcast special, you guys have made DevOps World and Jenkins World 2018 special as well.
James R.: I would also counter that—you have, as well.
Brian: Alright. Signing off. Thank you, guys.
Announcer: Like what you've heard today? Don’t miss out on our next episode. Subscribe to DevOps Radio on iTunes or visit our website at CloudBees.com. For more updates on DevOps Radio and industry buzz, follow CloudBees on Twitter, Facebook, and LinkedIn.