James Strachan - Introducing Jenkins X, Taking CD to the NeXt Level

Andre Pino: I’m joined by James Strachan, a member of the Jenkins community, a contributor to the community and a senior architect with CloudBees. James, welcome.

James Strachan: Thank you, it’s great to be here.

Andre: James, you’ve got a very strong history with successful open source projects. Can you talk about that a minute?

James: Yeah, I’ve been a long-time open source developer, I guess for about 20 years now. Someone reminded me recently that I created the Groovy programming language about 15 years ago, and something called Apache Camel 11 year ago. And now I’m working on a new open source project called Jenkins X, which – ironically, I learned on Twitter yesterday, was announced exactly 11 years since I released the first version of Apache Camel. Just kind of spooky.

Andre: Yeah. Well, great. Yes, Jenkins X was announced yesterday to the community. And can you give us some background on Jenkins X?

James: Yeah, so a lot has happened really in the last about five years, in what it means to develop software, what it means to package it, run it, scale it, manage it, operate it – what it means to do running upgrades, to use things like immutable infrastructure. Pretty much almost everything seems to have changed in how we approach delivering software and moving quickly. A big change has been containers and container images and technologies like Docker. And then Kubernetes came along, and that makes a huge difference. Kubernetes is a way of running containers at scale, kind of like how Google has done it for the last decade, so that anybody can be a kind of a Google and run containers at scale elastically and so forth. And so a lot of these technologies have meant we have to change how we do things. We have to package software differently, we have to define how to run software differently on Kubernetes. And I think kind of parallel, there’s been these other driving forces like the move to microservices. I think across the whole software spectrum, we’re all highly motivated now to deliver business value faster. I think everybody’s worried about being hit by the next Airbnb or Uber or other business that’s especially figured out how to iterate quickly with software, and that’s disrupting your business. So we’ve all really had to raise our game and move quicker. So the best practice today, to go quicker, is to use microservices, which means releasing lots and lots of small pieces of code quickly, so rather than releasing every year, we have to release every few minutes. And all of these things together means, as we’re adopting the cloud, as we’re moving to microservices, as we’re moving towards Kubernetes and containers, pretty much we have to do things differently. And for continuous delivery, that makes a massive difference. So what we’ve really been trying to do with Jenkins X is look at the new reality of what it is to build software. And we’ll look backwards into how can we automate building container images, deploying software on Kubernetes, doing continuous integration and continuous delivery? How can we automate that entire software production pipeline in a way that developers don’t need to really know or care about what Docker is or how it works – what a Dockerfile is, what Jenkins pipelines are and how to write one, or even how to manage multiple different environments on something like Kubernetes. So we’re really trying to just automate continuous integration and continuous delivery, so that any development team can go quickly. Over time they can maybe gradually learn what containers are and what Kubernetes is, and maybe over time learn the intricacies and details of what GitOps is and all of the various pieces inside Jenkins X. But really Jenkins X is really trying to be a huge accelerant so that your business can go quickly, deliver value and software to testing and then staging and then production, but without really having to worry about how that even works. We’re seeing a pattern historically with lots of – the way lots of people use Jenkins is often there's one or two people who look after Jenkins. And then developers often kind of half ignore it and just sit and they’re kind of doing their own thing. As things become more cloud native, all developers need to be kind of working with the cloud. So that we kind of want it where any developer can spin up any new project and, automatically, that is then deployed, released, staged in an environment and promoted without having this interaction between the Jenkins X ______, and so forth.

Andre: So essentially, what you’re doing with is you’re abstracting away from the developer the complexities of the underlying technologies and allowing the developers to focus on the code that they really need to build.

James: Exactly, yeah. I think in some ways you can see what we’re doing is a little bit – like “serverless” - is a trendy buzzword at the moment. It’s a little bit like serverless, as in where we’re hiding Jenkins and we’re hiding where the pipeline is and what a Docker file is and how you package software in Helm charts on Kubernetes. We’re hiding a lot of these details. So we’re just providing really simple, easy-to-use tools that developers can just spin up applications really quickly and get stuff done. And then if developers really want to learn more details – almost everything that defines your application and how it’s packaged is a Docker image, and how it’s deployed on Kubernetes – these are all declarative configuration as code. It’s all checked into a Git repository via source code. So everything is there and you can look at it and modify it if you wish. But in many ways developers don’t need to worry too much about that. We automate everything for them. And if or when they ever wish to modify their pipeline or modify their Docker file or modify their Helm charts for Kubernetes, they can get down and dirty and do that if they wish, but there’s no need to do that initially. So it’s a way of going quickly with all these different technologies, getting complete continuous integration and continuous delivery across multiple environments, and then iteratively teams can then become more and more aware of these different technologies.

Andre: So you’re providing a highly-opinionated environment for developers that extracts away the technologies, but also provides them a fast path to get their code into a production system.

James: Yes.

Andre: And if I understand you correctly, the users of don’t even need to know anything about Jenkins. Is that correct?

James: Absolutely. A user doesn’t need to know anything about Jenkins and Jenkins Pipeline. They don’t need to know anything about Docker or Kubernetes. To do promotions we use a technology or approach called GitOps, which is where we manage environments, like the staging environment or production. So a Git repository and configuration, to describe which versions of which applications are running in production. So to move from version one to version two in production, we actually do a pull request and a Git repository under the covers. And again, you don’t even need to know anything about that. The promotion can be a black box to start with, and then over time it might be that your operations team decides that they want to use GitOps to manage their configuration and production, if they deal with secrets or databases or whatever it is – caches or other middleware. So the basic idea is, we’re trying automate CI and CD for people using the cloud and using Kubernetes, without people needing to know anything about these technologies. But if and when you’re ready to in any way customize those things, they’re right there in source control and you can just go right ahead and make a pull request and change things.

Andre: That’s really interesting. And so, therefore, what is the relationship for a current Jenkins shop or Jenkins user? What’s the relationship between their existing Jenkins environment and Jenkins X?

James: That’s a really good question. And so what we’re really pushing with Jenkins X is a very opinionated way of implementing CI and CD, continuous delivery, using best practices, that promotes declarative pipeline as code. So we tend to – we only really use declarative pipeline as code, and use GitOps for promotions. Now Jenkins is widely used for lots and lots of different projects. So we’re expecting lots of customers to have a kind of hybrid environment of a lot of their classic jobs in Jenkins – which maybe were written before Pipeline even came along, and even if they’re using Pipeline, they might be using scripted pipelines rather than declarative. And so we’re increasingly moving forward on things like declarative pipelines in Jenkins X, and trying to push forward the cloud native aspect of Jenkins. What we expect, is customers are going to have a mixed set of different Jenkins, right? If you have Jenkins jobs for building mobile apps or something, leave Jenkins exactly as it is for those kind of jobs. For the new Kubernetes CI/CD, then use Jenkins X. Under the covers it’s still Jenkins; it’s the same Jenkins that we know and love. We’re really just packaging and configuring and operating Jenkins in a slightly more cloud native way. I think over time we’d like to consolidate all of the different Jenkins configurations and service together, so that it looks like a seamless Jenkins cloud. Today, right now, they’re kind of separate Jenkins that you look at. What we’re hoping to do is unify the two things together, so it looks like one, big, elastic, happy Jenkins cloud with old versions of Jenkins, new versions of Jenkins and the Jenkins X version of Jenkins, all in one seamless experience. So that’s kind of the way we’re going.

Andre: Wow. So what’s the current status of the project?

James: So, the current status of the project is – we’ve been dogfooding Jenkins X, so we’ve been running Jenkins X ourselves on public cloud, to build Jenkins X, and we’ve been doing that for about two months now. So we know that you can be really successful developing and releasing and promoting software on the public cloud through Jenkins X. So we’re pretty happy with it. Although, like any open source project, we’ve only just announced it, so now’s the time people are really starting to hammer on it and download it. People are hitting the odd dumpling there. So we expect in the next couple of months to be really about making it really, really slick and solid so that it will work for almost anybody. You know, we’re finding already a couple of little gremlins in different kinds of clouds that people hit, depending on how they’ve configured Amazon or Google or whatever. So we expect most things to be usable right now, today, but you might bang your head a little bit. Give it a month or two, and we’re hoping it to be really, really rock-solid and for the docs to be – to have ironed out all of those gotchas that the early adopters hit. But it’s certainly absolutely usable today to be really useful in your infrastructure. We highly recommend people give it a try, because we think this is really awesome. I mean, I use it everyday and I love it. Let me give you one feature that we do, that once you’ve seen this, you can’t live without. Whenever you’re building a web UI or a REST endpoint, you often have lots of people working together on that thing. You might have a designer working on the CSS, you might have someone wishing a new form of the UI to be developed. So you have all these different people with different disciplines and focuses working together. What we do in Jenkins X is, whenever you submit pull request, by default we’ll deploy the pull requests. So before this code has been merged, we will build a version of the application and deploy it to a preview environment. We’ll make that application live. We’ll make it live and give it a URL so you can click on it. Then we’ll comment on your pull request to tell you there is a preview version of this application at this URL. “Click on it to try it out and give your feedback.” Now we’re using this all the time to build our own UIs, and we find it insanely useful to be able to review any code change visually with a running app before it’s merged. You might find people try and solve a particular UI problem, and the way they’ve done it you don’t like at all. So rather than committing something to master and then trying to revert it or change it, we could break other things. You can then discuss things on the pull request before it’s merged. So you can keep master completely rock-solid and clean and ready to release, and have human approval and discussions on new things on the pull request, while getting instant feedback at that preview request. I mean, just preview environments are so useful to help teams go faster, right? So you don’t have to just watch the code change and imagine what it’s going to be, right? You can just see it early, which is really, really useful.

Andre: Sounds like a great vehicle for collaboration amongst a team.

James: Absolutely. I mean, increasingly I think the whole point of things in the last few years has been, how can we iterate faster? How can we get faster feedback? How can we get new code changes in front of our team or users as quick as possible? So we want to go from hypothesis to idea to code to testing it to using it as quick as possible, and preview environments are an awesome way of doing that.

Andre: So if people want to learn more, where should they go?

James: There’s a website called Jenkins-x.io, or go to the Jenkins.io blog, where the announcement was just made. There is lots of documentation and demos of how to get started, what Jenkins X can do for you. We’ll be writing a lot of blogs over the next couple of months on all kinds of different features, describing how promotions across environments work, how working with Kubernetes works, what’s different about Jenkins X versus Jenkins, how we use things like Helm and Kubernetes and so forth. So definitely check out the Jenkins X website. There’s a Jenkins X Twitter handle, which is not terrible active today, but it will become more chatty. By all means follow me on Twitter. I tweet probably more than Jenkins X does. Usually always about Jenkins X, by the way. So yes, I’d definitely check the website, and we’ve got a community URL. If you go to the website there’s a community button. We have a community on Slack. Most of the Kubernetes open source ecosystem is on Slack. People who use Kubernetes itself or related open source projects like Helm and Draft and Istio and various other things. So Jenkins X has a Slack discussion area if you like Slack. We also have IRC, because Jenkins has always used IRC, so you can chat there. We have an issue tracker, if you want to just give us some feedback on an issue and carry on about your day, then we’d love that feedback. We also have an e-mailing list if you want to – if you prefer e-mail, then issue-track us. So however you wish to engage with us, please enjoy the community and let us know what you think. Give it a try. If there’s things you like, tell us; if there’s things you don’t like, tell us. And hopefully we can continually improve.

Andre: Sounds like you’ve got all communication channels covered except for smoke signals.

James: Maybe that’s in the new version.

Andre: Well James, it sounds like a really exciting project. Any final thoughts?

James: I guess I’ve been involved in a lot of open source projects over the years, but this is absolutely the most fun I’ve had in a couple of decades, really. Just because working with or using Jenkins X can help yourself go faster, right? It’s all about, how can we all improve and go quicker? So what I want to say is just check it out and play around with it, because even something like preview environments can make your team more effective, happier, have more fun. But we’ve only just got started. There’s so many things we can do to go quicker, improve how we do things, improve the feedback, get quicker responses and so forth. Just a little quick example: whenever Jenkins X promotes a new release to the server staging environment, it knows all the issues that were fixed in that version. And it goes to all of the issues and comments on the issues to tell anyone who’s watching why there were these issues now fixed in staging. And here’s the URL in case you want to try it out. So that increase in kind of chat and feedback is really, really useful when you’re developing software. A lot of developing software is really about communicating with your team members and with the outside world and customers. And anything that can optimize using technology, going quick, getting feedback and doing faster iteration cycles is a huge win and it makes everything more fun.

Andre: Well James, this sounds very exciting. Just the fact that it’s part of one of the most active open source communities in the world is – I think it probably means that it will improve very, very rapidly. So check it out: jenkins-x.io if you’re interested in learning more. James, thanks so much.

James: Thank you. It was fun.

Andre: Bye-bye now.

James: Bye.