Ishu Gupta Brings DevOps and Jenkins to Capital One

Brian Dawson: Okay, hello. I am Brian Dawson. I'm guest hosting DevOps Radio live here from DevOps World/Jenkins World 2018 in San Francisco and I have another one of our awesome presenters and attendees here with me today on DevOps Radio, Ishu Gupta. Hello Ishu. Can you tell us a bit about yourself please?

Ishu Gupta: Hello, yeah. This is Ishu Gupta. I am a senior software engineer at Capital One and I have been in the information technology world for around 11 years and I have been a coder.

When I started my career, I was then a developer, Java developer, for around five to six years. Then I moved into sys ops space and then I started provisioning servers, started hosting applications on them, particular WebLogic applications, and then I moved to DevOps. I learned AWS skills, I passed the certification, and slowly I moved to AWS. Right now I'm a full stack DevOps engineer working on CI/CD pipeline hosting applications on the cloud.

Brian Dawson: So within Capital One you've taken that skill set that you've explained to us, that you've given us an overview, and you're actually helping your application development teams implement CI/CD pipelines?

Ishu Gupta: Yeah, so we are sort of – the team I am part of is kind of a horizontal team to a lot to developers so we provide all the DevOps to them, to various multiple teams. We provide an easy path and developer-friendly tools so that teams can focus on writing code. Our biggest goal is to actually move the red tape out of all the processes and provide them the best possible way to write code and deliver it in cloud and make sure that they are compliant and they are failing fast and getting the fast feedback as soon as they can.

Brian Dawson: Okay, awesome, awesome, and I'm curious what role has Jenkins played in that?

Ishu Gupta: Yeah, Jenkins has been pivotal. We have been using Jenkins ever since we started using DevOps. I think since 2012 at least I am using Jenkins from the last six years. Jenkins – we use Jenkins – at a very large scale. Most of our CI/CD pipelines are in Jenkins. We use other CI tools as well to provide all sorts of options to the developers, but the majority of work is being done on Jenkins through CI/CD.

Brian Dawson: Awesome, as engineers so you've used Jenkins for over half of your career in development and you've been a Jenkins user, very similar for me, though I don't want to date myself. My career has been a little bit longer, almost triple, but Jenkins has been prevalent there. So I wanted to ask you and you'd actually be a perfect person to answer this, this year's theme at DevOps World/Jenkins World is transform. How has Capital One transformed its offerings through DevOps and how have you participated in that transformation?

Ishu Gupta: Yeah, luckily I have been a part of the journey of this transformation. When I started in 2012 it was like a huge process to move their code from their local machines to actually an application server. I still remember the day in 2012 when I was a part of the team, which was actually ordering servers, getting it on-site, provisioning them, setting them, and setting domains on them, deploying the applications on them. It used to take like if I don't overshoot it, it would at least take two months to at least have an application in...QA but now literally it's minutes. That's the biggest piece and the second piece is providing all the toolings to the developer, like the developer doesn't have to go to a systems team to get their servers made up. They can just click a couple of buttons and create a...for their own self. They can do innovation very faster. They can actually customize the CI/CD pipeline the way they want to actually put the code in development as soon as they can. I think that's the biggest incubator of every single thing is the protocol to put their idea in a development in a QA environment as soon as they can.

Brian Dawson: As fast as possible.

Ishu Gupta: Yeah, as fast as possible. Speed is the biggest piece that was – that was the biggest enabler of putting things and putting releases and putting ideas into production.

Brian Dawson: Interesting. So first to go back and to continue to date myself, so I do remember when we wanted a server I remember we used to actually – this sounds bad – we used to order from physical catalogs, I want this server with this spec.

Ishu Gupta: Absolutely.

Brian Dawson: And go through that two-month process so you guys have now within Capital One transformed to enable your developers what sounds like almost instant access to environments.

Ishu Gupta: Absolutely, yeah.

Brian Dawson: How have they reacted to that? What is the feedback that you've received from your development teams in response to this increased speed?

Ishu Gupta: Yeah, so it has been an incremental progress. We have had various trainings and a lot of different tool programs, internal trainings and we also encourage various developers to go and get the cloud certifications. We offered incentives to get those cloud certifications and things like that. So through the period of time the developers ... to put their application into the cloud because sometimes when you make things very simple it also encourages bad behavior. People can just abuse it and not do the right thing so we have to also put the controls around it to make sure that the things they are using the cloud in the right way. They are not exposing the data to the wrong people, they are not overusing or they are not underusing their infrastructure to make sure that we also make sure that the Capital One data is safe and the organization is not at-risk and we are using the resources to full capacity.

Brian Dawson: Okay and it sounds like to make sure I'm capturing it correctly you guys – so you've incrementally sort of made this improvement and access environments and part of safeguarding this progress of giving them the speed of cloud is you've not only had to put sort of guardrails and rules in effect around cloud usage but you've also educated and trained developers so they themselves know how to properly use it. That correct?

Ishu Gupta: Absolutely, yeah.

Brian Dawson: Awesome. So this seems like a good transition to this next question I wanted to ask and that's what tools or processes have you found are necessary for organizations who are undergoing a DevOps transformation? It sounds like cloud may be one of them, right?

Ishu Gupta: Yeah, so I think cloud is definitely a big, big role. It has a big role to play in all this journey and I think our CI/CD tool like a Jenkins or I can I'm even okay with Travis CI or any of other big tool and our repository like RD Factory or Nexus kind of tool where you can store all the data. And above all an enterprise has to definitely have analytics and auditing capabilities so that they know that the things which are happening are being driven in the right way. So I think these are the tools which are necessary.

Brian Dawson: Those are very important. So CI/CD server to orchestrate and automate, you don't mind Travis but Jenkins and then source code repository, which interestingly enough a lot of people are still on legacy source code repositories, right?

Ishu Gupta: Yeah. So we had to also move from the legacy source repositories. I think we have used all sorts of tools like IBM Clear Case and SVN and CVS and now GitHub is definitely one of the big partners that we use.

Brian Dawson: And then I'm sorry, take me back into the last one, so we said CI/CD tool, source code repositories, and you said auditing and monitoring, right, so that you're – so sure, we've set up a pipeline of sort of checks and balances but we need to be able to report to ensure that we've gone through those gates?

Ishu Gupta: Yeah, that is correct. So I think the biggest piece of all this CI/CD that we generally lack sometimes, is providing flexibility to the developer. Sometimes DevOps has a big battle internally, which is going on between the developers and the sys admins where sys admins only think about the controls and the developer only things about the code.

Brian Dawson: You're right.

Ishu Gupta: I think the perfect balance between making sure that hey, you are writing the software, you move fast, but if you need the basic principle like security, code quality, everything is good. I can give you enough flexibility. I don't stop you from advancing... I don't fight with you on your versioning of your code. I just make sure that you are not creating something which is not right for the company, you are not following the basic coding principles. If you abide by those principles I think you are good to... the different environment.

Brian Dawson: Well that's a great point. I don't know if you caught the keynote today but that's very similar to what the CEO of CloudBees and Chief Product Officer of CloudBees Sacha and Christina talked about. This idea that businesses that need to innovate have to provide developers freedom while still maintaining governance, and it sounds like that's been your experience at Capital One as well, right?

Ishu Gupta: Yeah. A lot of times the sys admins actually have their own ideas in their mind like, "Hey, we need to build this control. We have to make sure that the build and artifact only goes this way to production."

Brian Dawson: Right.

Ishu Gupta: But the way I think the developers think is they want to put the idea right into our environment, like it's dev, QA or something. So I think for a company to grow and to make sure that they release things as soon as possible, they have to find the right balance between both their approaches. They shouldn't – they should be able to provide a highway to the developer so that the developer is not frustrated. They should be able to easily commit and build and deploy the code. But at the same time they also are not creating a risk to the organization. They are scanning a code with a security scan, they are doing the static code analysis, they are doing the right thing by writing unit tests, a certain amount of coverage is met, and then they are doing the...really like they are writing functional tests, performance tests wherever they can.

Brian Dawson: Right.

Ishu Gupta: So I think that perfect balance, if you provide them an easy path to build all these things they'll be happy to do it.

Brian Dawson: Awesome. So going on a tangent, going off script here a bit, Capital One is also well known for bringing forth the open source project Hygieia, which gives metrics and insights into what's happening into your delivery pipeline. Any comments? So I guess one, do you guys use Hygieia internally?

Ishu Gupta: Absolutely, yeah.

Brian Dawson: Okay and what value has that provided? What's the importance of Hygieia for Capital One?

Ishu Gupta: Yeah, glad that you brought up open source....so going back to the keynote this morning when the CEO said that hey, you have to be a software company; otherwise you're going to be going to be extinct from this whole space so Capital One is a software company, which is now in the financial area. That's the model of the company right now.

Brian Dawson: Right.

Ishu Gupta: So I think you can't be a software company if you're not in open source and one of the biggest projects that we have in open source is Hygieia and Cloud Custodian and various other project. So Hygieia is definitely one of the best tools that we have right now and Walmart and Verizon and multiple other companies are using it for getting the state of their DevOps. So it's basically a DevOps dashboard, which is completely free, open source, you can easily install it, download it, and set it up in your organization. You can emit even through your pipeline, get to know how many builds you have every day, ...what is the code coverage of your particular component, are there any security issues, what is the functional test and performance test look like for a particular component. It can also be configured to basically give a particular score to your component to make sure that hey, it looks like it's 8 out of 10, something like that so that's a great insight to have. And it can be customized based on every component need so it definitely provides value to a developer and Hygieia also has a great feature of providing executive view as well. So let's say an account executive can see the overall health of development under them. So let's say an executive has 100 projects going on under him, him or her, they can easily just to go onto the Hygieia dashboard, see here how is that trending look like, are my components under me writing unit tests properly, are the unit tests basically increasing day-by-day as they're delivering features, or is it a static number? So it is really useful for all levels of people.

Brian Dawson: So it sounds like Hygieia and those insights are a key part of providing the visibility and confidence to enable developers to have the freedom to deliver the way they need to at the speed they need to.

Ishu Gupta: Yeah.

Brian Dawson: So actually this leads well into the next question. How do you know if you're succeeding at DevOps? What are some indicators of successful DevOps transformation?

Ishu Gupta: Yeah, I think the biggest indicator that we have is how many releases do we have now every day. So if the team is able to deliver core faster production I think that speaks for itself. So if a team is taking two weeks to deploy something to production in the past and now it is only taking two days or a single day then that's it. That basically speaks that hey, the team is confident in building and delivering the software. I think that's a big input.

Brian Dawson: So part of your transformation I think may be speaks to the speed of delivery is that Capital One has made a push to the public cloud. What steps have you had to take to achieve that?

Ishu Gupta: So yeah, first of all I had to get trained to be on the public cloud. We had to find the right applications to be on the public cloud, we have to make sure that every application that we are using, every Amazon service that we are using on public cloud is properly scanned and approved within Capital One to use. Once we have that then we make our informed decision and go to the public cloud basically. So we are one of the very few banks or I probably can say that we are the first bank to make a major push to the public cloud as a company.

Brian Dawson: And what drove that? Why is the public cloud a priority?

Ishu Gupta: I think the public cloud gives you so many different features that you have to – that a person who's not exactly have the skill set of a particular technology can still be able to use public cloud. For example, if you want to build an E2 instance, you don't have to understand what exactly...how to segment the memory on it so...it's very UI driven and it provides you a great amount of features. It provides you a great amount of other scaling capabilities. I think the security is pretty high in that as well. So there are industry experts who are investing a lot of effort in making sure that the public cloud serves the right purpose and we as a company are leveraging that.

Brian Dawson: Get a benefit of that. Yeah, that's an interesting point. So there's a lot invested in making the public cloud generally consumable, which it sounds like you're saying also makes it inherently easier to use for your customers that want to adopt the public cloud.

Ishu Gupta: Yes.

Brian Dawson: And you mentioned earlier before we start to get to the wrap-up question I heard you say you guys had to identify which applications are right for the public cloud.

Ishu Gupta: Yeah.

Brian Dawson: Can you share with us a couple of characteristics of an application that's suitable for public cloud deployment?

Ishu Gupta: Yeah, I think at the end of it we found that all the applications can be moved to public cloud and we initially started with a smaller applications, which are not business critical and after that we made a push. Once we were comfortable and convinced that hey, this is the right way to go, that's when we made the major push, yeah.

Brian Dawson: Okay, okay. And so you had a talk here at Jenkins World, DevOps World/Jenkins World, right?

Ishu Gupta: Yeah.

Brian Dawson: And what was the title of that talk?

Ishu Gupta: So the title was Delivery Speed Software Confidence, Re-imagining Your Pipeline. So what we did was like we – the team I'm part of, we don't think that a pipeline is just CI/CD. We think that a pipeline's job is to make sure that you are doing the right thing. You are also empowering the developer and the sys admin people to put the code from there to production. It also involves making sure that you are building the infrastructure, you are hosting the infrastructure, you are following the right patterns, you are making sure that the infrastructure is not underutilized, making sure that you provide an easy path for the developer to have access to various enterprise tools like hey, if you want to have login access to a particular system why don't we just provide that out of the box? Why do you have to talk to five people? So how do you remove that red taping and make a process, which is four weeks to let's say 15 minutes to make sure they are empowered to build and write code as compared to talk to different teams and getting the access part.

Brian Dawson: Awesome, okay, so it's really about like you say re-imagining, looking at the pipeline as almost more than just a simple click, click, click automation process but actually something that builds and provides and experience for the developers and sys ops.

Ishu Gupta: Absolutely. So we broke the pipeline into four major experiences so if I can summarize them quickly.

Brian Dawson: Yes, please.

Ishu Gupta: Yeah, so one of the experiences was an on-boarding experience, taking all the imports from their developers saying hey, what's your release range, what's your snapshot range, what is your...team name, do you want stickiness on your ALB things like that, all the technical details. The second piece is hey, what kind of CI/CD – providing them a world class CI/CD, making...

Brian Dawson: Process?

Ishu Gupta: Not the process but the whole CI/CD pipeline itself making sure that we we enable our web hook from the Git repo to the Jenkins. We provide them easily manageable and configurable Jenkins file. We write the CLI library so that they get the best out of the Jenkins. The third part was the monitoring experience so we wanted to make sure that we built the monitoring within the pipeline. It shouldn't be that hey, now I'm ready to put my application to production, or the...environment, let's build monitoring.

Brian Dawson: Now let's bolt on monitoring, yeah.

Ishu Gupta: Yeah, so we said okay, let's build it out of the box. What does it take to emit business and take events out of your application, how can I verify I have 100 transactions happening for my component today? How do I verify the traffic is high? If I find out hey the traffic dropped at a particular point why did it happen? What was the correlation at that time? And the last part was release experience, like how do we provide easy paths for the developers to put their code into production but at the same time have a couple of...files, someone is approving it, someone is looking at the test result, someone is looking at the security scan right at the time of production deployment so they are comfortable and make an informed decision to put it into production and if it fails in production how soon can you roll back and perform different pieces with it?

Brian Dawson: Okay, that's awesome. I appreciate you taking the time to share that. So just for the listeners so that is the on-boarding, you've broken down the overall pipeline experience into four experiences: on-boarding, CI/CD, monitoring and release experience such that people can release quickly and easily but the right eyes are on it approving it.

Ishu Gupta: Yes, definitely.

Brian Dawson: Awesome. Thank you for sharing that with us. So before we wrap up do you have any final thoughts, words of advice to share with attendees of Jenkins World that are in the midst of or embarking on DevOps transformations?

Ishu Gupta: No, I think the biggest advice is making sure you find the right balance between the development and the sys ops. For the developers, make sure that you debug the pains right before jumping into conclusions, make sure that you build the code locally on your system first before you push it to production, try to containerize your code so that it's exactly the same way on your dev, QA... and even on your local machine. I think that's the tech advice and the process advice is make sure that you don't create too many layers for a developer to be successful. I think the biggest – find out your customer – as the DevOps team, as the developer, make them happy.

Brian Dawson: And make it easy, make them happy.

Ishu Gupta: Make it easy and happy.

Brian Dawson: All right, that's great advice. Ishu, thank you for your time. I hope you enjoy the rest of DevOps World/Jenkins World.

Ishu Gupta: Sure, you too. Thanks a lot.

Brian Dawson: All right, thank you. Bye-bye. Recording: Like what you've heard today?

Don't miss out on our next episode. Subscribe to DevOps Radio on iTunes or visit our website at CloudBees.com. For more updates on DevOps Radio, follow CloudBees on Twitter, Facebook, and LinkedIn.