Episode 72: United States Air Force’s Nicolas Chaillan on DevSecOps in the DoD

Brian Dawson: Hello. This is Brian Dawson with another episode of DevOps Radio. Today, I'm excited to be speaking with Nicolas Chaillan, the Chief Software Officer of the United States Air Force. Hello, Nicolas. 

Nicolas Chaillan: Hey, good morning. How are you?

Brian Dawson: I'm doing well. I'm doing well. Yourself?

Nicolas Chaillan: Very good. Thank you.

Brian Dawson: Good. Well, let's go ahead and kick it off. First, actually, let me say thank you. I'm excited to have you. It's an honor to get a chance to talk DevOps and DevSecOps with you, someone with your background and experience. So I expect this to be a special conversation.

To kick things off, can you give our listeners a brief overview of what your background and experience is? And what led you to be the first Chief Software Officer in the United States Air Force?

Nicolas Chaillan: Absolutely. I think it's pretty interesting that you have a French speaker, at least someone with a French accent joining the US government to try to make a difference and push DevSecOps into the DoD. I started back 20 years ago. When I was 15, I created my first company back in France doing software innovations. I was part of the team that helped created PHP, the programming language. 

I ended up created 12 companies that I sold. I was lucky to make some money, to be able to first join DHS, where I was the chief architect at DHS, at S&T. Then I joined FEMA as a special advisor for cyber security as well.

Then I got a call, trying to see how we could do better when it comes to software and rapid prototyping innovation by OSD. They wanted to try to bring DevSecOps to the DoD, kind of following the success of Kessel Run in the Air Force. They were trying to find a way to scale that. 

Having a lot of years of experience in rapid prototyping innovation and building DevSecOps stacks across the world really, 12 countries, in my company with different cultures and different needs, that's been very exciting for me to help the DoD move faster. So we created the DoD Enterprise DevSecOps initiative when I started in August 2018.

Brian Dawson: Awesome. Wow. I spent some time looking at your previous experience and was blown away at the number of companies founded and sold. First of all, congratulations on those, and congratulations and thank you actually on taking on this role within DoD, to help make software central to what we do and help make the Air Force, the DoD, and the federal government operate more efficiently. 

So as I said in the introduction, you're the first chief software officer and, I'm sorry, not just in the Air Force, but in the federal government. I'm curious, one, how did the role come about? Did you create it? Was it created and then they sought you out? But then, more importantly, now that it does exist and you're filling that role, why is it so important in the federal government? 

Nicolas Chaillan: It's actually interesting because, when I started, I was really trying to push the DoD to re-embrace a commercial mindset and not do anything special. I think the mission of DoD is special, but the software is really just software. There's no such thing as a chief software officer on the commercial side, so I was actually not a fan of the name or the title, but we do have a lot of silos and we're pretty big in the government.

So we had quite some success with the beginning of the DevSecOps initiative. We wanted to find a home for this outside of OSD, and often we provide that into one of the DoD services like the Air Force. Because of the existing success of the Air Force, they ended up picking the Air Force to be the executive agent for DevSecOps for DoD. They ended up also then creating that chief software role.        

At first, I was not embracing the title, again, because the title doesn't exist on the commercial side, so I wasn't sure it was actually needed. I wasn't sure why we would do that. But also, you have to realize how the government is structured and how we deal with the silos inside of the government. 

A CIO in the government, unfortunately, is also not empowered the way a CIO is empowered on the commercial side. Many times they end up being policy shops and not being able to execute the work or manage the people, or even pick the people or hire. So that creates, obviously, silos and creates a massive bottleneck when it comes to execution, and makes it very hard for CIOs to be successful.    

The fact that they decided to put the chief software office inside of the acquisition side of the government is actually the right idea, because we end up having access to the funding and the money. Obviously, that helps drive decisions and push things to the right level of attention. 

So I think it's been quite exciting to see this role evolve. It started with really building some culture and understanding the lay of the land, but now it's driving several teams that are providing enterprise services to DoD with Cloud One and Platform One. 

Brian Dawson: Thank you for that. So how would you define the role? As you said, there's not a chief software officer role in the private sector. So to help me better understand it, how would you describe what your primary charter is? I'd be particularly interested in if it changed. What was it on paper when you started and what do you see it being now?     

Interviewee: I think it's really about empowering the organization to leverage software to become more efficient, more capable. Of course, that's going to change based on the kind of organization we're talking about. In the Air Force, of course it's going to be about the entirety and making sure that the _____ are all getting the technology they need at the pace that they need, and I think timeliness is becoming more and more critical.

I would say it's very much the same on the commercial side. Most companies innovate through software. There's very few innovations that are pure hardware defined. You will see way more continuous innovation on the software side, and that is the difference between having a successful business and losing against their competition. Clearly, having someone dedicated to empowering the organization to move at a pace of relevance and bring DevSecOps capabilities, and really also provide those as a centralized managed service. 

I think people often don't realize the complexity of providing an enterprise scale capability, beyond just the basic of cloud. When you start talking about continuous integration, continuous delivery by appliance, and all the complexity that goes into building those and providing flexibility and tools between AI, machine learning, deep learning, and the 16 program languages which support 23 different bases, and GPU needs, and all the different real time need for software on weapon systems, that becomes really a full-time job to make sure that, one, we can prioritize the work to bring these enterprise capabilities to the organization.

But on the commercial side as well, you end up having teams often building these capabilities instead of focusing on building their software and whatever it is they're trying to build, and you end up having siloed teams that are building their own stack and their own CI/CD pipeline, which I think ends up as a massive _____ and will slow down every single team. 

So I see the chief software as re-empowering and bringing kind of enterprise scale DevSecOps, but also streamline cyber security _____ and making sure that the process is understood, baked in security is there. That's why we call it DevSecOps and not DevOps. For us, it's very important that the baked in security is part of the continuous monitoring lifecycle. Really, that's what we've been doing for the last year. 

Brian Dawson: There's so much there I want to dig into. I think I want to start though with – so I'm surprised, because I'm sitting there thinking that Nick has it better than any other VP of DevOps anywhere else because you're working in the Air Force. You're the chief software officer. So you can just go in and say, "Everybody use the same tool, use the same database, use this process." Right. No negotiation, no having to embrace other teams. 

Nicolas Chaillan: I would say that's really when people fail. I think it's all about bringing options and not having a one size fits all. If you look at the stack we built, we didn't want to have everything based on Kubernetes and containers. So we have a single abstraction layer and we can deploy the software on any environment, whether it's a classic private cloud or public cloud or edge use case, or even on a weapon like F-16, where we deployed Kubernetes and containers on F-16 jets.

So we wanted to have that abstraction layer and that's why I picked Kubernetes, and yes, we did _____ Kubernetes, but that gives us access to multiple companies that can provide the service, whether it's Red Hat with OpenShift or VMware with TKG and D2iQ and Rancher and many others like Oracle. So you end up having the same abstraction layer and the same flexibility, so your software can move to different environments, but yet, you're not pushing one solution over another.

What's also interesting is we always want to bring options when it comes to tools. We're not saying, "Hey, you all have to use the same stack." First, DoD is too big. Each program in DoD could be equal to 1,100 many times, so you don't want to start pushing everyone, whether it's a business system, cyber defense system, a weapons system, a space system. You don't want to start pushing the same tech stack.

So we wanted to provide enterprise support and scale for 16 programming languages, 23 databases. We wanted to make sure we were bringing options to the teams. Let them pick. Do have a set of options already accredited and approved to be used at scale, and streamline that access, both to the contract, so they can acquire and get access to the licenses as fast as possible, but also do the same when it comes to scanning and approving open source software as well. 

Brian Dawson: Thank you for that. As you know, some of the listeners may know I have terrible humor, and earlier was a bad attempt at sarcasm. I think I'm funny, but I'm not. I'll keep trying. 

Nicolas Chaillan: I think it's funny.

Brian Dawson: So I really caught onto the 16 programming languages, 23 databases, the focus on service, which I think is really interesting and, as you did, you compared it to the commercial sector. I think when we try to standardize on certain software development and delivery practices such as continuous integration, continuous delivery and DevOps principles, I'm of the feeling and finding in a lot of conversations that in the commercial space it's extremely important that you do develop an abstraction layer and you provide teams with flexibility, so they can choose the tools that they need and the processes that they need to work more efficiently. 

I find it really interesting to know that even though you're close to the funding and you're the chief software officer in the federal government, I would assume with a little bit of authority to use the stick instead of the carrot when driving DevOps adoption, but it sound like, just like in the commercial space, you're saying it's fundamental that you apply some abstraction and flexibility in the federal government, right. So I just really wanted to kind of underscore that. 

Now shifting, Nicolas, I want to get on a common question you've probably been asked a lot. You called out having the baked in security in DevSecOps. Let me ask you, why can't it just be DevOps? Is there some value or importance to leveraging the term DevSecOps when implementing and promoting these best practices? 

Nicolas Chaillan: Well, I see DevSecOps as an evolution of DevOps. Some people think of the Sec of DevOps as just being the traditional static and dynamic crisis that we see everyone do, and that's not what we mean by Sec in DevSecOps. 

For us, it's really about continuous monitoring, particularly when it comes to behavioral detection and zero trust. We have a full zero trust stacked onto the container level using service mesh called Istio. That's how we enforce need to know and reduce the attack surface. 

Then we bring continuous monitoring tools with behavioral detection in real time to detect behavioral change of the containers, and proactively kill the containers using Chaos Engineering. So we're really pushing the next-gen security stack. 

For us, we had to find a way to insist on that requirement to be able to get to what we call the continuous authority to operate, the continuous ATO, where we can push our production multiple times a day continuously, but it's not just because we have a pipeline. The key aspect to get this accredited and this reciprocity going DoD-wide, like the way we built it, was to convince the cyber teams that we were bringing this continuous monitoring stack in production as well, to provide insight as far as what's going on continuously in production, providing centralized logs and telemetry, behavioral detection, zero trust, all as a turnkey capability we provide as a service to all these teams. So the Sec piece is not just to bring an old static/dynamic _____ of the house.    

Brian Dawson: Thank you. So as you've promoted DevSecOps, as you've worked to establish shared practices, promote and deploy those, are there any best practices or methods for driving that adoption that you've learned along the way that you could share with our listeners?

Nicolas Chaillan: Well, I think it's all about having the cards, like you mentioned. For us, it was really all about making sure that we could provide a continuous ATO, which allowed teams to produce software multiple times a day or as often as they wanted, and push to pull action, remove the _____ in that rotation process, and then making sure that data creation could have a DoD-wide reciprocity.

So the big thing we tackled was the reference design for DevSecOps, which we signed a year to the day with complete coincidence of my start date. So we signed it August 15, 2019, a year to the day, to explain to the different teams exactly what was required in a DevSecOps stack, to be able to have this DoD-wide reciprocity to be compliant and secure when it comes to the Sec piece of DevSecOps. 

Brian Dawson: So going back to your commercial experience, your experience in the private sector, where you also implemented a number of these practices and technologies, you have touched and worked on or implemented and deployed a technology in the field such as cyber security, DevSecOps, mobile, IoT or Internet of Things, big data, mixed reality, VR. We heard you mention some of those that you've actually deployed within the DoD.

But as you look to drive what I would call new, modern or even emerging and, in some of these cases, bleeding edge tech, looking at the time that you were kind of rolling it out, how do you ensure that you have the right talent and culture, or did you, in your organizations to make adoption of these technologies successful? 

Nicolas Chaillan: Well, I think there are two sides of that. One is we have a lot of great talent already. They are mostly very not empowered to do their job right. So the number one thing was to remove impediments in front of them.    

So we spent quite some time visiting teams and trying to find these kind of unique talent teams, and tried to bring them together into what we call now Platform One. So we kind of merged a lot of the top talent, building platforms across the Air Force, to become that central team that can provide it as a managed service to existing Air Force and DoD teams. That is now kind of the DoD-wide enterprise service for the SecOps, managed service.  

The second piece is how do we bring others with us? We need a few of these key set of people. But what happens when they leave? What happens to scale this? That's where we decided to look into training and spending quite a bit of time on understanding the best way to do that is scale across with various onsite training. That has not been possible anymore, but we have to train 100,000 people in one year.

That's a mix of acquisition/program manager side of the house, with a mix of technical teams and cyber teams and testing teams. So really trying to bring self-learning capabilities with nonbiased content. A lot of the training content up there has been designed by companies that are trying to promote their product, and we just cannot get locked in or have the teams just learn with that content without always getting other opinions and making sure we're not getting locked into a single product. 

So we've partnered with quite a few teams. The Linux Foundation was one, to make sure we had some content coming directly from the foundation. That's also why we picked Kubernetes with a CNC Foundation, the Cloud Native Computing Foundation. We became a member of the foundation as well, so we can vote. That's been very helpful. We just released a case study as well to try to show what we're doing on F-16 jets. 

So I think it's about getting people excited, people from the outside to join the Air Force so they can see, hey, we're leading the way here. We have dozens of companies that are reaching out to us because they want to reuse what we're doing. In fact, the entire stack, the DevSecOps stack we built is completely open source. So we have quite a few companies already using and consuming the infrastructure code and the containers we built. So that's been very exciting to see the adoption, which is helping the culture spread and get people excited to come and work for the Air Force as well. 

Brian Dawson: That's interesting because you touched upon, in the federal government or the public sector, something that I think is an under-addressed area of focus in the private sector, and that's up-scaling, training, but then also building a culture where you can retain your best talent and you can gain top talent. So it sounds like, if I understood it right – you can correct me, Nicolas – one component of what you're doing to build a culture is to outwardly and externally, publicly promote and celebrate some of the advanced work that you're doing.

It also sounds like you're empowering people within the Air Force to work on some pretty cool stuff and get recognized for it. Is that fair to say? 

Nicolas Chaillan: Absolutely, yeah.

Brian Dawson: Well, I'm going to get a little personal with you. I'm going to go off script. You got some questions to kind of prepare you, but as I warned you before we started, Nicolas, I would probably go rogue.

So the semi-personal question is the chief software officer in the Air Force, or really in your role as any chief technology officer, what are your fears? What keeps you up at night? Then what are your motivations? What gets you up and gets you excited in the morning, in terms of the role and the space that you work in? 

Nicolas Chaillan: Well, I think my fears changed after I started my work in DoD, of course. That's when you realize that everything we've been doing, while we were able to make some good money, we did not really have the impact we would be having spending more time in the government. I see that now as this kind of obvious when you look at the mission and the importance of what we do to protect the freedom we're enjoying every day.

People sometimes miss that. They don't realize that the reason why we can do some of the things we're doing on the commercial side is thanks to the military and _____ protecting all of us. So I think that's something worth keeping in mind. 

What keeps me up at night is often related to cyber, slash, timeliness, meaning every time we have bottlenecks, every time we make decisions or policy decisions that are either reducing the ability to compete against some of the other nations that are not always the most friendly nations out there, and when we get into a situation where we let governance, slash, policy compound the delay in delivery of capabilities to the _____ when we take risks that we should never even think about doing.

Really, I think it's been very scary when we see sometimes people trying to do the right thing for the right reasons, in terms of cyber or in terms of compliance, not realize that the downside from that could be a component down the road 20 years behind other nations, which often they'll have the same limitations or care about these kinds of things. That of course is why DevSecOps matter.

When we realize we're saving about 18 months of program time for every program moving to the SecOps for every five years of time compared to what was planned, this is what's happening. By adopting now multiple dozens of programs through DevSecOps, we are now saving 100 years. Already we saved in one year one hundred years of program time across the DoD. You can imagine the impact of that. It's not just financial. I don't think it's primarily a financial problem. I think it's really about time.

Brian Dawson: So there's an interesting thing I want to dig into there. If I heard correctly, to kind of suss out or interpret some of what you said, part of what keeps you awake is the balancing of speed versus security. But then as you progressed in your answer, it sounded like, Nicolas, that you may be making the case that you don't have to balance speed and security. In fact, speed may enable you to be more secure. 

Any comments on that? Can you correct me? I'm sure I got it wrong.

Nicolas Chaillan: No. I think there is a difference between compliance and security and governance and egos and security, and cyber is very different from all of that. So I care very much about cyber, and that's why we're pushing for this baked in security. 

I would say it's okay to take risk if you can correct it very quickly. For example, if there is a zero day exploit in Linux and I have to update all of my containers, I can do that within minutes or seconds across the entire stack, because we all use the same container repo that we opened, Repo 1, which is public to the Internet and doesn't _____ outside of DoD, including on the federal side, but also on the commercial side, while using these containers right now. So that gives us the ability to react fast.

So you can take risk and move fast, if you can react fast and think very quickly and learn fast. My main motto is fail fast, but don't fail twice for the same reasons.

So part of the answer is cyber matters, maybe compliance and governance a little bit less. Sometimes they are there for the right reasons. Sometimes they are not. We have a lot of silos with people trained to have a say into every stage of the _____, the weapons system. The more silos, the more steps and gates. We realized that compounds to years of time wasted across DoD programs.

So we always have to take a step back as leaders in the government, when our ego or what's best for the chief software office or whatever other office, it might not be what's best for the _____ and the DoD. So adding more governance, more compliance, more requirements could become a massive impact to the delivery of capabilities to the _____.

At the same time, you don't want to do a bunch of nonsense and just have every team do whatever they want. There's nuclear – [indiscernible] – we are flying weapons. I mean this is real. It's not just a freaking mobile app on a phone, like I've done for 20 years, right. It's a different type of impact if you do it wrong.

So that's where we have to ban things. I get it, right. When we touch nuclear systems, which we do with multiple programs, we do have to be extremely careful. And having manual gates is completely fine. The key is for the gates to be as automated as it can be, but also to keep in mind that if we do have a gate, is it really necessary for actual safety, for actual cyber security, and not just checking the box and pure compliance reasons.

Brian Dawson: Right, that it's logical, smart, sensible, manual gates that aren't frivolously slowing things down.

So I'm going to want to shift and ask you some questions. I'm going to ask you to predict the future. But before we shift to predicting the future, I know one of the common things that you have to deal with in the Air Force, DoD, and federal government is the authority to operate, and the process or requirements to achieve authority to operate. 

What have you been able to do or do you have any take on automating what are those, I beg to say, kind of rigidly defined checks that are so critical to the process within the DoD or Air Force? And of course correct me if I have it completely wrong. Maybe ATO is not a focus of yours. 

Nicolas Chaillan: It is absolutely. I think we inherited a lot of great work by Lauren Knausenberger, who is the Chief Transformation Officer for the Air Force. She's the first AO, authorizing official, who signed the first continuous ATO for Kessel Run, and then signed the one for Platform One, the second continuous ATO in DoD.

It's changing the way we even think about _____. Continuous ATO is not about moving fast. It's not a fast track like _____ ATO. It's a very different concept. The concept is about creating the factory, the pipeline and the gates, and defining those gates, and really understanding the gates, and the software that comes out of the factory is automatically accredited by passing the gates.    

At the same time, we have created the team within the factory, so it's not running in a vacuum and teams understand the risk, and understand the cyber responsibilities of that team to make sure that they're not bypassing the pipeline. So there's kind of a two-step process of accrediting the factory and the accrediting the team using the factory, which is why Platform One can bring a factory that's already compliant and accredited, and now teams can just inherit that and customize it as needed, picking programming languages and tools they want to use and so on, and then be accredited to use it by receiving training to learn how to end up using the gates within the factory. 

Brian Dawson: Okay. I have to ask you. You mentioned it a couple of times and you actually explained Platform One. But I am curious. What are the components of Platform One? What is Platform One in terms of tangible things?

Nicolas Chaillan: I think any company or organization out there needs to have the same concept of what Cloud One and Platform One is, and I'm going to go Cloud One first and then Platform One.

Cloud One is really a cloud office that brings both, for us, Azure and Amazon government clouds to the different teams that need to use it, with an ATO baked in. So they get day one access to an enclave that's accredited and they consume bother Azure and Amazon. We decouple both by having single sign on across both clouds, so the same accounts, and the ability to consume both services, so you don't get locked into one cloud or another. That's Cloud One, really streamlining access to cloud.            

The Platform One is agnostic to the environment. So they do use Cloud One, but they also can deploy on 22S, which is the classified cloud from the AC, also any on-premise edges case, weapons system. They can deploy anywhere, really completely agnostic to the environment. 

Platform One is kind of a managed service team that can bring as little or as much as the teams want. That's really critical to _____ as a company providing a service, even though it's a government team with contractors as well. But the _____ program is on the customer and we're not telling them what to do. We can help them if they need help or advice. 

At the end of the day, the program office of each team gets to pick what they use, as little or as much as they want. They have access to the repo of containers. So Platform One manages Repo One, which is the repo of containers centrally accredited that has been scanned and approved to be using DoD for containers, both open source and commercial products. So multiple companies can provide their container to Platform One, and we accredit it and we have a central place to push it to all classification levels. 

So a very great use case for smaller startups, too. If they need to start doing business with DoD, they can come to Platform One and within a couple of weeks of getting their containers, they can get accredited DoD-wide, all the way to the highest level of clearances. So that's a great way to empower smaller business to do business with DoD as well. So Platform One is leading and building the Repo One.  

Then Platform One also builds all the code and automation and flash address code, configuration as code, to bring up from scratch the DevSecOps stack, including Kubernetes with the multiple distributions like OpenShift, TKG, Rancher, D2iQ. In partnership with these companies, they give us their code and we make sure everything is automated and can run on all these different clouds and disconnected air gapped environments and weapons systems. So we make it easy for _____ to adopt all of these different tools. Then we have examples of pipeline that they can come and consume.   

The key two services of Platform One are what we call the party bus or the big bang. The party bus – [laughter] – yeah, they always have cool names. And you have to see the Platform One logo. It's pretty cool. 

But if you look at the party bus concept, it's a multitenant environment. So we picked a bunch of tools for SecOps with quite a few options. We run it. We manage it. They get access to it as a development team to consume it. They don't really become admin users. They just because a developer user to consume those services and have access to all these different tools. That streamlines the access to DevSecOps, particularly for smaller teams or teams that need training or are getting started in DevSecOps. They pay per user, and they can consume and have access to a pipeline at different classification levels that are already accredited and with all the tools they need. 

Then the big bang is what I think is also critical for any organization, the ability to spin up a dedicated enclave, very much like we did for the party bus. In this case, it's a dedicated enclave per program, for example, per office or however the customer wants to cut it. We can spin up a dedicated enclave with all the tools and we manage it. We maintain it.

We can go all the way from here, as we deploy it, and it's your problem to manage the day two of the environment and continuous updating of that, or we can continuously manage and operate, bring the cyber security and the _____ to run it. So that streamlines access to a dedicated enclave for DevSecOps.

Brian Dawson: Okay, great. Wow. There's so much to cover and, as I probably say way too much it's cliché, I wish I wasn't up against the clock because I could really dig in and would love to learn from you more.

I want to shift from talking about the present and your past experience to now talking about the future in terms of future. My first question is, what do you see the future of DevOps being? What is it going to be in two, five, ten years? Or maybe even I think a more apt question than what is the future DevOps is what the future of software development and delivery over the next, two, five, ten years? 

Nicolas Chaillan: I think the key aspect is going to be abstraction and being able to be decoupled from the platform and be able to run on any environment. I think we're getting way too locked in. We had a massive wave of SaaS services, software-as-a-service, that were completely dependent on APIs of cloud providers. I think that's a massive mistake and _____ that many companies _____ because of the flexibility and the ease of adoption of these services. 

If we look at how we build software now, when we want to be agnostic to the environment, we bring everything with us as containers. So every service we're going to use is going to be containerized, and I think it's going to be an easy way to ship and deliver software and making sure that the software is immutable, going back to everything is going to become code, infrastructure as code and configuration as code, and everything will be – your source of truth is you'll get repo, multiple sets of eyes on code, making sure that you do your change management and your security there by having multiple sets of eyes on the source code _____ request and things like that.

So I think the GitHub model of everything is code, and you pull – not push, you pull. Kubernetes will be able to pull from Git to enforce your desired state in production is going to be the way to go. That's what we use today with Argo CD on the DevSecOps stack of DoD. With Argo, we can pull, ensuring that there's no CIC tool that have keys of production systems.   

So everything is code. Any change is code. You don't just bypass the stack and go look into SSH in production to make changes. Everything we do is going through the pipeline and redeploys, and the pipeline will be able to do its gate checks, and then the production system will pull once it passes the different gates and _____ request approval from the different users that have _____ _____ code to approve the change.    

So everything from _____ rules to networking to the service mesh, so massive adoption of service meshes. There's going to be big federation, a lot of _____ products that will come out on the GitHub side to enforce parity and no drift between code and production. So if you have anything that changed in production versus working your source code, then you have a drift and that's not acceptable. That means someone went into production and made changes manually, which should never happen.

You should really create everything as _____, right, so carrying down everything and being able to bring back up with GitHub is fairly easy. You push a button, deploy, and you can carry down every night or multiple times a day, and bring it back up with no downtime, if you do it well. So I think for cyber and reducing the attacks _____ and making sure that we have moving target defense, so teams cannot – bad actors cannot laterally move within the system and we don't even know they've been there for years. If you tear things down and bring it back up multiple times a day, go back to a mutable state, that's going to reduce a cyber risk. 

So I think a lot of these principles will be a massive enabler when you look at the cyber risk today, and the attack of bad actors being inside a system for years before the system owner is even aware of the incident. So all of that is kind of the future, containers, functions, _____, Istio. Anyone doing software the old-fashioned way with _____ application, not flexible, not able to be modular and Lego blocks concept, when you can swap things around and use the best tool to get the job done, and be able to swap to different programming languages and that is based on your mission needs. They're just going to get behind very quickly. 

Brian Dawson: Right. So basically, you're telling me you have no opinion of what the future looks like. [Laughter]. I might be wrong. No. I mean those are all good bets. You covered a lot there, a lot of practices, tools, and technology.

Let me ask you, in terms of emerging tools or technology, what's something that you're really placing your bet on or you see developing and you're excited to work with. Is there any tool or technology that you're particularly excited about?

Nicolas Chaillan: Yeah. It might sound a little boring, but I think there are two concepts. The GitHub new products are there that are able to push the desired state in production automatically and make sure that they apply the change. You don't have to tell how they're going to make the change, but you just tell your desired state and Kubernetes will be able to make that happen magically. That's pretty cool stuff. And everything is code.

I think on the cyber side, I see a couple of companies looking into scanning your source code and scanning your production workload, and comparing the two and finding drifts. Anything that changed between your source code and your production workload could be either a bad actor, someone got a foothold into one of your environments and is spinning stuff that you shouldn't have there. Or it could be drift, a human connecting to the system and making changes, which is a big no-no for us. You want to stick to immutable code. 

So products that can detect this at a fast pace and find someone connecting to the _____ console and typing things and changing opening _____ or whatever manually, right, which is a big no-no again. So entering, we have all these gates. It's great. 

Then the last one is really the service mesh, so things like Istio or Linkerd and many others, console. They all have a different concept. For us, the key aspect was mutual _____ and zero trust enforcement, so having the ability to whitelist traffic between containers, denial by default, and reducing the attack surface, so the teams can laterally move between workloads without whitelisting the traffic. So now you have a service mesh team that whitelists access to Container A to talk to B, which reduces the attack surface. So if a bad actor gets access to Container A, they only can see what B is doing and that's it, and not see the _____.

So that kind of concept of service mesh I think is going to be game changing and we're going to be seeing more and more features that will allow teams to adopt more easily program languages, so they don't have to coordinate releases and walk in tandem between teams to organize and plan for releases, which is really an outdated concept that should not exist anymore, so we can have teams that release at their own pace.

All of that compounds to time, which all of that compounds to being able to complete any business on the planet. In the next ten years, we'll be either highly successful or just behind. Ninety percent _____ on the DevSecOps understanding and automation. If they don't have those tools and concepts in place, they will not be able to compete. So that will empower teams to either fail or succeed.

Right now, it's like a 20, 30 percent empowerment to move faster. I think down the road, with more people adopting this concept, the difference will be being able to win the next battles or not. 

Brian Dawson: Right. Fantastic answer, by the way, the last two answers, fantastic. I'm going to have to dig in and do some research so I can keep up. 

That actually leads me to ask – I know I promised I'd start to wrap up. Forgive me. We'll get to some of the fun questions in a minute. But are you familiar with the concept of progressive delivery? If so, where do you see progressive delivery playing into the future? Feature flag management, feature management. Are you focusing anywhere there? 

And to convolute the question more, clarify and convolute at the same time, there's been a lot of discussion about elevating. We deal a lot with the technical concerns of creating a build, deploying that build, basically moving kind of what I would call a tangible artifact from code through the pipeline into production. But above that, right at the end of the day, we're delivering features and capabilities. I know progressive delivery and feature flagging speak to taking a different view on addressing more of what I would call the business concerns, to use a generic term.

So to rephrase the question, do you see a move to elevating the focus on the problem that you're solving, and do feature flagging or progressive delivery come into play in your mind at all? 

Nicolas Chaillan: Yeah. Funny enough, I think the similar concept we've already _____ for 20 years. It just had a different name and different concept and less information, but very much because they had different environments with different needs, they had to do some type of feature flagging and enabling or disabling things. We have special flags for a classified environment versus an unclassified environment and so on. 

So it's interesting how it's been there and we just always like to rename things and come up with a fancy thing, which, in this case – in your case, there is a concept around the automation that brings value, too, and the ability to use the flags to try things out with _____ releases and progressive deployments, where you see the adoption from your end users and see whether or not they like it. 

It's a concept that I've been using on the commercial side for years. Certainly on the DoD side, it's particularly interesting to start looking at adoption and whether or not something is successful.

What I also like is when you do not enable a feature publicly when it comes to the user seeing the feature, but you can still enable the feature and see how the system behaves when you start taking a load of millions or billions of users. That's what Facebook did. With Facebook Messenger, they didn't just turn the switch and add a billion users overnight. They had the feature enabled, and I think fake chats and video and stuff for months, without the user even knowing, and pretty much using people's compute at no cost to do their own testing, until they actually enabled the visibility of the feature, which was already actually activated for months. So it's kind of a hidden feature concept.

That kind of feature is game changing. How do you know, particularly when it comes to scale and performance, that you're going to be able to take the load? Of course, Kubernetes helps when it comes to auto-scaling and that kind of concept of CPU memory scale. But, again, you want to make sure you can test that. What a better way than doing progressive deployments and be able to see the impact on your infrastructure and your systems? 

Brian Dawson: I know I promised I'm going to get you out of here, and I realize I was so engrossed in what you were saying that I lost track of time. So I'm going to shift now to ask you about a DevOoops moment. We ask all of our Devs guests this. And that's D-E-V-O-O-O-P-S, like Dev Ooops.

Really, I'd like for you to share with us a time in your career, where you or someone you know has made a mistake, but you were able to turn that negative moment or that mistake into a learning moment. Do you have something that you can with our listeners?

Nicolas Chaillan: Yeah. Of course we have tons of little things. The funny thing is we have not had a big oops moment, mostly because of the gates we're putting in and the ability to rapidly react to issues and _____ releases and things like that. I think the issue comes when people try to move too fast and they forget the basic principles of DevOps or simply bypassing the pipelines.

So you have all this security, and I've seen teams where we had a bunch of security in the pipeline and then, oh, we didn't stop someone from bypassing the pipeline and going around the entire security system, and they're pulling stuff manually, if need be, on the cloud. That kind of defeats the point. So that's what kind of an interesting moment, where we're like, okay, we put in all this security, but how do we make sure that we don't have people bypassing the pipelines – [indiscernible] – particularly a lot of time on a product that will be able to detect that, and looking at signing and things that the pipeline could do to sign artifacts. So that was the lessons learned in terms of that.

Then the next one is about signing as well, where we all have often the signing pipeline using the same key, and if that key gets compromised, you can get into trouble. So people posting – you know, we had a team that made a mistake and because, remember, everything is in code, they put the automation in code and they had forgotten to remove their key, and the key ended up publicly on the repo, and that's obviously a big no-no. So we use Sealed Secrets that can encrypt the key, or we have vaults as well, to be able to have the key stored on the vault. We even have Sealed Secrets that can store on the Git repo your keys publicly facing, but encrypting them. Then of course we don't put the private key, so no one can really encrypt that. So that's also a big _____.

You want to push automation. You want to push for everything is code. But you also want to not forget not to put your secrets and passwords publicly assessable. 

Brian Dawson: Yeah. It's funny. I've heard a form of leaking credentials, leaking secrets, leaking keys a few times. Thankfully, as it sound like for you, no harm done, and it was a reminder, as you're sharing with us, to be very careful with that. 

So my question for you, I promise here, is what is a book, a podcast, a resource, a learning resource that you use that you recommend our listeners consume, that they go out and get right now and it will be game changing for them. Do you have something like that that you lean on?     

Nicolas Chaillan: I think there is a lot of great content out there. Of course there are a ton of podcasts, but also books and audibles also doing a good job at bringing some of these books for people like me, who have a two-hour commute each way. That helps me listen to a lot of books.

Of course, for me, kind of the top two I would say on the DevSecOps, the DevOps side is The Phoenix Project and The Unicorn Project. I would say it's particularly helpful when it comes to opening eyes to some of the old school IT operation guys that are used to those silos and those kind of old school processes, to open their mindset to new, better processes like DevOps. I think those two books are particularly successful at opening people's eyes.

Brian Dawson: Thanks for sharing those. So The Phoenix Project and The Unicorn Project is what Nicolas referred to here. 

Nicolas, before I let you go, do you have any final thoughts, any final tidbits of wisdom to share with the people that are listening to you on our podcast today?

Nicolas Chaillan: No. I think the reason why we've been quite successful in the adoption of DevSecOps is really building this centralized team and making this a centralized enterprise service. I think people that don't do that and let each team do whatever they need is just slowing them down. I think you need to be able to bring that kind of enterprise service concept for DevSecOps, while not having a one-size-fits-all, so providing options, not too many, not too few, _____ that's difficult to understand. 

But I would say that's going to be the prime enabler for organizations to move at a pace _____. I always say for people that always think that things like Kubernetes or containers are not something they can use or not stable enough, if we can use them on weapon systems, they're certainly good enough for your business as well. 

Brian Dawson: That's a great point. Thank you. I will tell you, listening to you, I'm glad we have you in the role, helping empower the Air Force and the DoD with software. So thank you for the work that you do. Thank you for your time today. Thank you for all of your insights. Please reach back and let us know when it happens and how it goes. 

Nicolas Chaillan: Yeah. Thank you so much for having me.