Episode 102: Creating the Software Matrix with Ev Kontsevoy

Brian Dawson:

Hello, welcome to another episode of DevOps Radio. I'm Brian Dawson. And today I'll be your host and you'll join me in talking to Ev Kontsevoy, the CEO and co-founder of Teleport, and a man with a pretty impressive view on the current state and the future state of software development and delivery. Ev, hello, how are you doing?

Ev Kontsevoy:

Hey, Brian. Thanks for having me. I'm doing well.

Brian Dawson:

Yeah. Well, thanks for joining us. Our guests don't know, but you and I had had a chance in recent weeks to have an offline conversation together and I thought it would be awesome to get you on to share some of those thoughts with our guests. To get you introduced and get you started, of course, I gave a very simple introduction. Maybe you can give our listeners an overview of your role at Teleport or generally, what are you doing today?

Ev Kontsevoy:

Sure, sounds good. I'm a software engineer by training. There's a big difference between what I'm doing today and what I like doing today, but that's probably what is true with all of us. I love technology. I like building things, but at the same time, in my current role, I'm a co-founder and CEO of a successful and very quickly growing company. I am in charge of making sure that we continue to succeed, and I'm in charge of growing our executive team, in charge of getting our message out there. And it is very much enjoyable. So yes, I wish I had more time in front of a keyboard and the Vim editor, but it is what it is.

Brian Dawson:

Well, that is funny because I tell the story of... So for years, even though, I wasn't spending the majority of my time at a keyboard, I would always introduce myself as a software engineer or a software professional. It was a couple of years ago where I opened up a new laptop and I realized I didn't have an IDE installed. I hadn't open an IDE in way too long. I realized I couldn't call myself a software engineer anymore necessarily, but that is still where my heart's at.

Brian Dawson:

I tell my team, as we get into technical issues and building out environments, that I'm jealous. They don't believe me, but I miss the time. Now that said, you talked about an successful venture and helping ensure that you guys continue to grow. That of course, is Teleport. Obviously, we're not here to advertise Teleport, but help us understand what Teleport is and what problem you're solving there.

Ev Kontsevoy:

Teleport gives engineers, developers, DevOps engineers unified access to all kinds of cloud resources, that includes SSH, access to servers, access to databases like Postgres or MySQL, access to Kubernetes clusters, internal web applications. Anything you run behind NAT, Teleport connects you to all of these things using obviously, industry-best security practices, namely certificates and capabilities like single sign-on. So, it gives your command-line tools superpowers. It allows you to connect to everything you need to connect and keeps your security and compliance people at your company quite happy.

Brian Dawson:

Would you say that there's the bit of Teleport, well, I mean, obviously the name implies kind of what you covered, but it flattens the world.

Ev Kontsevoy:

Well, there is a reason it's called Teleport, right? Because if you...

Brian Dawson:

Right.

Ev Kontsevoy:

I'm very much inspired by the future of computing, the vision that has been presented to us in sci-fi literature like Neuromancer and Snow Crash. The idea of us using one giant multitenant computer, and they're all logging into it. It's basically what you see in the Matrix, right? The nice thing about being in the Matrix is you plug this thing in the back of your head, and you're in. And it's completely seamless, and security is actually quite amazing, right? Because whatever regular people can do, agents can do even more.

Ev Kontsevoy:

We felt that today's computing world, like the way we interact with all these different cloud computers, is extremely annoying, that you have to deal with different login mechanism, and role-based access control is different everywhere. There are like proxy servers and jump hosts. So, think of Teleport as the cord from the Matrix. You just login one time and then all of your command-line tools suddenly know how to connect to something. Yes, Teleport creates the illusion that the rest of your company's computers are in the same room with you.

Brian Dawson:

Yeah, that is phenomenal. On discussion for later is, when we reference sci-fi and that how we have to watch out for AI and ethics since we centralize all the computing, but I know in Teleport realm, that's conversation for later. Before we move on though, and dig more into that vision, I do want to get into your career and your path to where you're at, as we usually do here on DevOps Radio.

Brian Dawson:

So, as you said, you're an engineer. That's what you enjoy; spending time in Vim. But you've now founded in a running a successful company. What was the path between the guy that loves sitting down at a terminal or down at computer and coding to the guy that is creating a company that realizes the Matrix?

Ev Kontsevoy:

Whoa! I don't know where you want me to start. I just did... Mm-hmm (affirmative)?

Brian Dawson:

Well, anywhere. But we're curious, let's start with this: how did you get into computer science? How did you end up programming?

Ev Kontsevoy:

Oh, that's an interesting question. I was born in what used to be called Soviet Union. And as many of us, and probably including you, so I got exposed to computers through gaming, like really early when I was a teenager. And it became kind of clear that I want to do that. But in Russia at the time, they would say something like, you become computer scientists only after you fail at becoming a real scientist.

Ev Kontsevoy:

So with that kind of background, I decided that I needed to go get a degree in math, because it's the closest thing to computer science. Also, because Russia has a lot of fantastic universities that teach that. That's basically how I've gotten into it. Because prior to discovering computers, I felt that I wanted to be a aerospace engineer, because come on, flying things, isn't that exciting, right?

Ev Kontsevoy:

In terms of how you transition from getting into software, and then eventually, to co-found companies, I think you'd have to get lucky a few times. In my case, my first real job, like in terms of job I really wanted was I was a software engineer at a company called National Instruments. National Instruments was building virtual tools in a computer, that you would use data acquisition board plus some software to measure anything you want. So, that was basically one big giant idea behind the company. And I loved it. Having other engineers from all types of industries, from companies like Lockheed Martin, man, use what you do to build things that fly... It was just incredibly exciting.

Ev Kontsevoy:

And then, when I decided to explore what the rest of the world looks like outside of National Instruments, I was extremely not impressed with software that we make for people. It's like, really, I just realized that for me to be happy, I need to be building software for other engineers. Because I can put myself in their shoes. It's just something incredibly fulfilling, something satisfying about building software for engineers.

Brian Dawson:

That's interesting.

Ev Kontsevoy:

Right now, if you listen to me say that, it doesn't sound overly unusual like, "Yeah, sure. Go get a job. Build software for engineers." I'm talking about the year 2005/2006. It wasn't really clear what kind of job do I need to have? I got an offer from Microsoft at the time to go work, and they had a team called Developer Tools. So it's like working on CLR runtime or something like that. That meant I had to relocate to Seattle, right? I was in Austin, and I loved being in Texas. So that's really kind of my path, how I become became an entrepreneur because I needed a job. So I had to... I needed to eat. Yeah. Yeah. I got to-

Brian Dawson:

Okay. And you didn't want to... You needed a job and you... So I got a couple of things. One is you found empowerment in people using your solutions. You identified gaps in those solutions. You needed a job because you needed to eat and you didn't want to move. And those things came together to say, "I need to go out and create my own thing. I see a problem. I need to go solve it." Is that fair?

Ev Kontsevoy:

... Yeah, it paints me in a very positive light, like I was carefully scheming and planning all of that. No, no. It was actually an accident. So what happened? First of all, I didn't want to move to Seattle, I wanted to move to East Coast. That's really why I decided against to go into Microsoft. I moved to New York City. And around that same time, I went to an event like a developer meetup or something, where Jeff Bezos was talking to developers.

Ev Kontsevoy:

At the time, he was already a famous billionaire with these kind of giant personal bodyguards following him everywhere. But Amazon wasn't cool. It was not the kind of company where you wanted to run or host your application. It was a bookstore, launched EC2. I don't think it was called AWS at the time.

Ev Kontsevoy:

Bezos went on stage, and he did this presentation like Computing as a Utility. And at the end of that presentation, developers were asking him skeptical questions about, "Well, your cloud kind of sucks because of this and that," and they would kind of... Performance issues, or whatever. But someone asked him, "How come you cannot get any email in or out of your cloud?" Because without that... Because for spam-filtering purposes, I think they were blocking SMTP ports, or something.

Ev Kontsevoy:

I was sitting there in the audience, and that's really what gave me idea that I got to start a company. Because before, I thought if I want to build software for engineers, my mind was kind of locked in this mode like, "I need to sell libraries, software libraries" because that's really what software for engineers used to be. But that is like a dead market. Everything's open source now. So I would-

Brian Dawson:

Statically-linked SDKs, right? [crosstalk 00:11:50].

Ev Kontsevoy:

... Yeah, yeah, like some kind of SDK. That was kind of sad, so I felt like, "There is no way you're going to start this company." But then this whole exchange between Bezos and this random engineer made me realize, "Hey, I could just have an API and charge money for it. All these people who are going to start using Amazon's cloud, they will need the email capabilities." That's how the first company was born called Mailgun. So it was just me being in the same room with these two people exchanging questions. It's not like I was particularly prepared for that.

Brian Dawson:

Right, right. Particularly prolific and you had a grand vision. But no, that is actually a really interesting story. A couple of things to comment on there, one to squeeze in. You and I had a conversation. And I think I told you ultimately, I want to go start something on my own And I'm looking to build up the muscle. I don't remember exactly what you said. But your story encapsulates a bit of the response, which kind of came across as you just do it. It's not about... You didn't wait to build up a muscle to understand how to do it. You saw an opportunity, you saw a problem, and you went out and chased it, right?

Ev Kontsevoy:

Yeah, pretty much. I don't believe you can predict what kind of muscle you need to grow in order to do something big. Just like Nike says, right? Just do it.

Brian Dawson:

Just do it. But the other interesting thing that I just caught there, that correlates back to a conversation I was having with a gentleman from Dell Capital Investments. And he was marveled at how API... It's like, if you asked me 10 years ago if APIs as a service would be one of the largest growing segments in kind of the DevOps or software landscape, he would have never believed it.

Brian Dawson:

And if Tyler Jewell happens to listen to this, I know this not exactly what he said. But we were kind of marveling at that. Now, when we look at Mailgun, kind of this view you had and selling API access versus SDKs is actually turning it into one of the fastest-growth business models in our space now. There was-

Ev Kontsevoy:

Yeah, I [crosstalk 00:14:02] Twilio for that. Twilio, I think, was the first company that really, really fixed investors' minds. Before Twilio, I don't think we had a successful example of an entrepreneur raising a ton of money on a API idea. Maybe I'm wrong, but this is how it felt at the time. When I saw Twilio funding announcement, like series A, and then B, and so on, so forth, it felt like it was the first company that made investors believe, so Jeff has done a phenomenal job there explaining to them.

Ev Kontsevoy:

I failed. I went to some investor meetings and I said, "Hey, I'm going to sell you an API." I'm not going to name that gentleman, but I remember one of them just kind of, in a patronizing tone. He's like, "Ev, you don't understand. API, it's not a product. First, you build a product. And then second, you put an API in front of it." So, okay, whatever.

Brian Dawson:

... That's funny. Well, now, you can just silently send him some market reports with nothing else. So now I mean, look, that credits you with however you want to look at it. And not here to stroke your ego, there was a level of even though it sounds like you didn't know how to express it, or necessarily sell it, a level of vision you had there, that's manifested itself.

Brian Dawson:

Now, what I would like to ask, shifting a bit, is software engineer went out, made a company happen, had a reasonable... You exited with that, moved on to Rackspace, which gave you a new view of the world. And that ultimately led into Teleport. Actually, before I go into the next question that I have listed to ask you, tell me a bit about that transition. What did you see at Rackspace that led you to this view of future computing that's oriented around the Matrix and ultimately Teleport?

Ev Kontsevoy:

Well, maybe it wasn't so much about Rackspace. But with Mailgun, using that company technology as an example. So we started with a few bare metal servers, at SoftLayer. That was a hosting company in Texas that got acquired later by IBM. And then, the Mailgun team and Mailgun technology stack was basically following this cloud revolution, like 2009, 2010. '11, '12, '13, so on and so forth. So as all these different cloud providers were making new and new capabilities, we had to adopt that.

Ev Kontsevoy:

Just being there and witnessing how certain innovations making our lives better, and certain innovations make our lives more miserable, made me develop certain strong views on cloud computing. And then, when we got acquired by Rackspace, that gave me access to a lot of people who were in my shoes. I'm talking about engineering leaders at companies that use a lot of cloud computing, not necessarily Rackspace customers. So I met and talked to people at GitHub and companies like that where they were just sharing their pain points with me. And that basically kind of shaped my belief that the cloud computing soon it will be replaced with something dramatically better. That's kind of where my mind exists right now in the future.

Brian Dawson:

Okay. So, there's... But I have to say, somewhat controversially, I think I had shared this with you before. But it's, again, some wisdom to your approach. And that's that look, there's so much with kind of this DevOps movement and digital transformation and the motion to cloud, that you have a bunch of people out there speaking at a very high level, right? At a very macro level in terms of what it means.

Brian Dawson:

But I think sometimes we forget about kind of the tactical, on-the-ground bits and pieces that compose this movement to the future. And I say all this because when I first came across Teleport, and I'm like, "Yeah, well, SSH, that's not sexy, right?" And not to offend you, right? I mean, it's SSH. But there's a reality that this path to the future requires, that the tools we use, the processes we perform day-to-day, which may not be sexy enough to make it into a keynote presentation, actually, are the core of that movement to the future. Is that fair? And am I shocking you in saying discussions about NAT and SSH are not sexy?

Ev Kontsevoy:

Look, I'm used to it because at Mailgun people used to tell me, "Ev, why are you doing email in 2009? Email is not sexy anymore." I guess that's what I do. I just like my not-sexy things. But I have a way of making SSH sound sexy. Imagine if someone asks you, Brian, someone nontechnical, someone in your family who is, I don't know, into again, some completely different field, and they ask you to explain what this thing is.

Ev Kontsevoy:

So then, what I would tell them is that, look, you're interacting with the Internet, and everything that's on the Internet, using this thing called browser that's in front of your nose, and it has a protocol that it speaks, and it has this user interface. And there is like probably $1 trillion worth of value that being delivered to people who use browsers all the time.

Ev Kontsevoy:

But then, did you know that there is completely different side of it? So the Internet is split between people who are consuming it, and people who are making it happen. And those makers, those engineers in the shadows that make that possible, what do you think they use to get all that code out there to get all those configurations and get all the data in? They have completely different tool, and it's this secret browser you never heard about, and it runs on SSH, most of the time. So now suddenly, he's like, "Ooh, my God, it's almost like a dark matter of the Internet." Ha! Ha!

Brian Dawson:

Yeah, no, no, that is good. Right, yeah, there's all this background that people don't know. I talk about my kids a lot on the podcast. My son's an economics student. And he has to do some market analysis on SaaS. It's funny that you say that because we kind of spent hours... What I realized is he was asking, "Okay, what is AWS? What is the cloud, really? What is a VM? What is open source? What is Linux?" I realized that there is so much there that is hidden from the lay person, and that I didn't come up with a concise way to explain it. So you did it well. I will call back though, that you said, "I have a sexy way to explain SSH." I didn't know if the voice was part of it. No, that's a good explanation.

Brian Dawson:

All right. Well, let me move you to another question. So you are now a CEO and co-founder of two technology companies, both can be considered successful. What are some of the key lessons you learned. And we have a split of leaders, founders, practitioners, a widespread audience. But for that group, either those that are currently leading today, or those that aspire to one day, what are some things that you learned that you can share?

Ev Kontsevoy:

I can talk about maybe two things, and one of them is maybe more tactical and another one is maybe more macro. On a tactical level, and it's a mistake we've done when we started Mailgun. In the early days, we were overly influenced by the financial crisis. Because if you were already in the workforce, for those of you who are too young to remember that, it felt like there was no money in the economy. Investors were not writing checks, customers were not spending as freely. I'm talking like year 2009, maybe early 2010. So it made this kind of impression on us that we need to be extremely frugal, and we need to work super hard.

Ev Kontsevoy:

And ultimately, we basically worked ourselves into this situation where we felt extremely weak. This was one of the reasons why we said yes to an acquisition offer. In hindsight, it was a mistake. So when you starting a company, and you're starting a business, you must not treat that as a sprint. It has to feel more like a marathon or it actually has to be, not even a marathon, it needs to feel enjoyable, and it needs to be sustainable.

Ev Kontsevoy:

And so, think of a company as a way for you to build a pleasant reality around yourself. It allows your day-to-day life to be better, and your company needs to be a vehicle, how you're making your personal life also better. Because if it's not true, first it will be like a slight pain, and then eventually it will be agony. So, that was a tactical thing.

Brian Dawson:

Yeah, that's powerful. Before you go to number two, let me just ask you to clarify a word. When you guys worked yourself into a position where you felt weak, what do you mean by that? I'm assuming people were burnt out [crosstalk 00:23:32].

Ev Kontsevoy:

Uh-huh (affirmative). Actually, physically weak. I gained some weight. I started to develop some joint pains. This is what happens when you work crazy hours, when you don't have any time off. When you take every failure extremely personally, when you... I remember once I got an alert on my phone, that our database was dropping connections. It meant that someone's email was not getting delivered. And I remember feeling literally paralyzed waist down for a few minutes. But this just kind of sinking sense of panic. Don't do that to yourself. At the end of the day, it's just a job, so you have to manage your kind of psychology, all these things.

Brian Dawson:

And it sounds like at the end of the day, it's just a job. But if you truly care about it, if you put yourself [inaudible 00:24:26], at the end of the day, if that's the way you run your business, ultimately, you're not likely to succeed.

Ev Kontsevoy:

Exactly.

Brian Dawson:

Fair. Okay. And you had number two. That was awesome insight. And then, you said you had a more of a macro lesson.

Ev Kontsevoy:

I don't think it's a lesson but it's something that I've been thinking about it lately is that it's interesting how you phrased, in the beginning of this conversation, this kind of process of starting a company: you identify a customer need, you identify solution, and then you go and build a company as an answer to that solution. And I used to basically believe that that is indeed the case. Look, there is nothing wrong with it.

Ev Kontsevoy:

Most companies probably are founded this way, but if you want to inject some fun into it, and this is something that's probably happened with Teleport, to some extent, is I think it's more fun to create a company about a really interesting end state, about the future state. You want to change the world to be a certain way. And it could be something absolutely crazy, and being crazy is good.

Ev Kontsevoy:

Let's come up with really insane example. You've heard about this UFO case, right? Nimitz the aircraft carrier saw this Tic-Tac-shaped UFOs and all the pilots saw it. It was in the news. It was in the New York Times. All right then, so how do these UFOs move? People say that they bend the spacetime fabric. So then, you basically say, "You know what, let's start a company that will build a drive that basically, let's invent this propulsion method that UFOs use, something completely crazy."

Ev Kontsevoy:

And then, go and yeah, raise money to do it. Tell investors like, "If you want to be famous for solving this mystery, if we need to move away from burning tons of carbon to shoot one like 10 pounds of payload into orbit, let's go and do that." And by doing that, you will attract a lot of other crazy people too. They will join you to try to build this thing. But you know, the interesting thing? You'll probably fail. It's pretty, very unlikely, let's put it this way, that you will create this-

Brian Dawson:

You've got to learn how to bend the spacetime.

Ev Kontsevoy:

... but you will probably create a lot of other cool stuff in the process. That's really the thing. And not only your own life will be a lot more interesting this way, but I do believe that the kind of quality and the, not even quality, but the impact of things you will end up building will be way higher than anything that you could have done if you were just like trying to solve a specific customer problem. Something like this actually happened with us in Teleport. Teleport was originally started... Well, maybe we can talk about it later, but-

Brian Dawson:

No, go ahead.

Ev Kontsevoy:

... When you asked me about what did I learn in Rackspace? What is it I'm unhappy about? I am unhappy about state of the cloud. Because running software in the cloud today is incredibly complicated. The complexity of computing has gone in absolutely insane. It was for me, in the early years, that was why I abandoned Windows. Because building software for Windows was getting more and more complicated, but it was the same software, you just had to learn more APIs, you had to spend more time reading docs, you have to type a lot more. You just have a lot of typing to make simple things.

Brian Dawson:

For loving... So, Keith, I hate to interrupt because I always talk about this. And I always felt it was a shortcoming of mine. But as we got into object-oriented programming, and it grew in kind of the complexity underneath in computing, I used to get frustrated with the amount of time... To type a couple of lines of code, the number of docs I had to read, right? I had one of those rotating monitors that would sit and I was...

Brian Dawson:

I forgot, maybe it was C# at this time. But literally, to do anything, I had to have a monitor rotated sideways, and I'd spend half my day reading docs. Anyway, you just triggered a memory about some of the pain, but back over to you, right.

Ev Kontsevoy:

It is like some PTSD going on here. Absolutely. And then, I switched to Linux, and it felt simple, and it was awesome. "Hello, World!" started to look like "Hello, World!" again, like these little applications are possible. And then this cloud thing happened. We started to pile layers and layers and layers of abstraction. Like I remember I was at Rackspace, and I was talking to a company and they're asking me, "So what do you think our container strategy needs to be?" And like, "I don't know, what do you want them to be?" And they said, "Well, aren't containers better than VMs?"

Ev Kontsevoy:

I said, "Okay, if they are for your case, sure." And they were kind of happy. They said like, "Oh, it means we're going to replace all our VMs with containers." I laughed, and I was like, "Most likely, no. Most likely you're going to run containers on top of your-

Brian Dawson:

On top of VMs, on top of hardware.

Ev Kontsevoy:

... I just feel like we never tell people, "What are you making obsolete with this thing that you build?" Everything gets piled on top, on top, on top. And that is the craziness that we now have this industry. We call ourselves DevOps. We're DevOps people. Half of engineering budgets right now, it's companies are allocated to DevOps. What do we do?

Ev Kontsevoy:

Well, if you can go look into these technology stacks that we use, we have these layers like, "Okay, there's a Linux sitting on bare metal, that just hypervisor and VM sitting on top; that's another operating system inside. That VM doesn't exist in isolation. You use some kind of API to manage it, like EC2 API or OpenStack, whatever. That's another layer.

Ev Kontsevoy:

So, there is like a container running in it that has another image of an operating system. That container image came from somewhere, oh, there's a Docker registry over there. And all of it is managed by Kubernetes. That's another layer. So then, there is probably Grafana dashboard, and then there's MongoDB. And then there is NGINX. And there is Redis cache. And I'm not even talking about the actual application, but this stack keeps going up and up and up.

Ev Kontsevoy:

And now, think about it. Every layer is listening on a socket. Every layer has its own configuration file. Every layer uses some kind of encryption. Every layer has its own idea about what role-based access control is. And now, so you count all of this endpoints, all of these sockets, config files, and then you have to have people on your team with expertise on every single thing. That is insane.

Ev Kontsevoy:

The thing is you have to have it even when your application is relatively small. It's not like you can gradually kind of build up this thing. So it's insane now. You look at startups, like five-person startup, they just graduated from Y Combinator or whatever and they are already looking for a DevOps hire. That is just crazy. Because DevOps people basically run software manually.

Brian Dawson:

Well, yeah, and we have had some debates about [ESO 00:31:51], so hold your thought, I'm going to try not to take over too much. First, it's funny because you just captured. And I told you about this conversation I had with my son. It was exactly that. I mean, he's trying to wrap... I'm like, "Son, it's infinite layers, just take my simple explanation."

Ev Kontsevoy:

Turtles all the way down.

Brian Dawson:

But yeah, I mean, look we are trying to roll out a simple Django app or PHP app on the cloud, it is funny that you raise it. You literally can have a few hundred lines of code, and some configuration files. And it's deploying on this huge glacier of technology. Yeah, so that is... Forgot where else I was going to go. But yeah, that's really actually powerful, the way you call that out. Now, that explains the word, clunky that you used. Anyway, that's [crosstalk 00:32:39].

Ev Kontsevoy:

This is kind of like where I was going with this is to the realm of crazy ideas. So we decided that the world actually wants Heroku experience, it's pretty obvious. Like remember when Heroku showed up, it was like, "Oh, my God!" That it was like first true serverless platform that you could use without having to deal with all these layers. However, people want Heroku for their own use case. They want Heroku for their own scale. They want Heroku for their language of choice, whatever.

Ev Kontsevoy:

And that, we decided like, "Why don't we build a company that makes computing feel like the Matrix?" Because Matrix by itself, it's basically like cloud, right? It should be, should definitely be something like that; some kind of data center somewhere, some kind of servers. But when you interact with the Matrix, you don't feel presence of any layers, you get in, and then you basically execute instructions inside of the Matrix.

Ev Kontsevoy:

We felt, "Okay, let's build a company, and we're going to say that we will make your applications run anywhere in the world by themselves without DevOps people." And you might sound, "Well, that's a crazy idea. How can you make arbitrary application run anywhere in the world without DevOps people?" But okay, but what if we try? What if we try to build the stack? Where are we going to end up?

Ev Kontsevoy:

It's not exactly the new propulsion method to [crosstalk 00:34:05], but yeah, so we were not as crazy. It was more enjoyable, I'll tell you that. So if you start with something a little bit more abstract, it makes this kind of search for a new product much, much more interesting and enjoyable. And that's really where Teleport came from.

Brian Dawson:

And so, Teleport became the component problem of that larger propulsion vision, right?

Ev Kontsevoy:

Correct, so with Teleport, we realized that all of these layers of technology, they all have sockets they listen on. There is always security considerations. How do you control who has access to this database, to this SSH server, to this Kubernetes cluster, to this dashboard, whatever? So, if you were to manually configure access to every single thing, even if you use scripting, that is still a ton of work, and you need to have a lot of expertise. But most importantly, you creating a lot of opportunities for human error.

Brian Dawson:

Yeah, it's a lot of risk. It's a lot of risk.

Ev Kontsevoy:

Yeah, you will get owned. And so, what companies do, it's a combination of things they do, they either just cut access completely for their engineers and say, "You know what, no, you don't touch this from now on. You have to fill out a form." So you making people less productive if you do that. Or they will default to some hack like, "Let's have a shared secret, and we're going to use one password," or, "we're going to have this one SSH key that these five people can have." So they basically try to balance complexity with productivity, with usually mediocre outcomes.

Ev Kontsevoy:

And then, we've felt that, in the process of trying to build this Matrix-like experience, we did build these cordless phones, which is what Teleport basically does. It says, "I will natively speak all protocols that you use. I will give you credentials for all endpoints and all protocols using your identity, so you have to authenticate with Google or GitHub, or whatever it is your company use, and then Teleport will configure your command-line environment, so it's connected to everything you need."

Ev Kontsevoy:

But in doing so, not all engineers are super productive because they can access all of these things they want; SSH servers, Kubernetes clusters, databases. But the security people now don't have to configure every single socket, every single endpoint manually. It's all right [inaudible 00:36:30]. So, this is like a story, how we started with a relatively crazy idea: let's try to run applications anywhere in the world. And now, basically ended up with a new class of product, which we believe, called the access plane. You have a control plane, data plane, and Teleport is your access plane.

Brian Dawson:

Awesome. Yeah.

Ev Kontsevoy:

I like that, so if you ask me about these lessons you learned... And so, I definitely enjoy this experience of building something wonderful, but by trying to build something crazy first.

Brian Dawson:

Yeah. Yeah. I love that. And I forgot we were still on that seed topic. We forked on a bunch because I got a lot I want to dig in. Maybe we'll hit as we go further, but yeah, I do want to call out that that lesson both, so first, thank you for sharing those two lessons, right? One is when you're building a company, don't treat it as a sprint, treat it as a marathon or even better, ensure you're building it around something you enjoy doing, or else it compromises your sustainability and possibly puts you in a weakened position.

Brian Dawson:

Lesson two is shoot for the moon. Don't look to solve a point-today problem, but come up with something crazy. You'll fail, but in falling short, you'll land on something that really matters and you'll enjoy it while you're doing it. And I think there's somewhat of a commonality in both of those lessons, that I think if you go for the moonshot, you don't spend your time trying to make up a North Star that inspires people, that makes people want to get up every day and work and innovate to solve the problem. If you start with something crazy, the bright side is you have some inherent or built-in inspiration that motivates people when they show up every day. Is that fair?

Ev Kontsevoy:

Yeah. And I do think you get higher quality people joining you too.

Brian Dawson:

Yeah, all right, so now let's bring it back down a little bit. So some of what we hit there, in multiple points, we've talked about developers on the ground, practitioners, and how you talk to them and you identified pains they were feeling. Can we bring it back down a little bit and revisit what you're hearing from developers about their pains and how you feel you're addressing, or we need to address those things?

Ev Kontsevoy:

Well, are you talking specifically about Teleport or just in general?

Brian Dawson:

I think, in general. In general.

Ev Kontsevoy:

So, one thing that is now interesting, I'm not sure if it's a pain that developers are realizing, but it's definitely a question that we should be starting to ask ourselves. Why are we separating developments like front end and back end? Because if you think about it, so what is a front-end engineering?

Ev Kontsevoy:

So let's say, okay, using a browser. Let's just put mobile development aside, but okay, so using a browser to visit a certain website. As okay, the page is being served to you and certain code executes. So where is the code coming from? The code sits in some kind of server anywhere, regardless of is it JavaScript, or is it some Golang. The only difference between front end and back end, is that code is getting downloaded into a browser, and then it gets executed there.

Ev Kontsevoy:

And then, this one little silly fact, that just something gets downloaded, we basically have sometimes like two completely different recruiters inside of a company. One is, "Well, I'm hiring for front-end roles." And the other goes like, "I'm hiring for back-end roles." So I'm desperately waiting for this difference to go away. I don't think I'm the only one. We've been dreaming of [crosstalk 00:40:32] for a while, so trying to follow this Wasm space to see what happens.

Ev Kontsevoy:

Because I would just prefer as an engineer, I'm building an application, you could put like annotations to certain functions like you, you're going to run on my server side right next to database, right? You, you're going to run on a client, go into the browser. And you, you're going to run on edge and it's like a CDN PoP, right? And then, it's just same application, same programming language, same choice of libraries. I just want that experience badly, and I do notice that other people want it too.

Brian Dawson:

Well, I do know that that's a resurgence, so a bit of your precursor to Teleport, but well, I kind of generically put it at while you talk about the separation between front-end and back-end developers, gosh, there's a number of threads to pull there, I also hear a bit of, "Look, let's have one team" to put it grossly, right? Let's have one team that can write one codebase, taking a systems-thinking approach and deploy it anywhere, deploy it in multiple places.

Brian Dawson:

Let's focus on the logic, not the back-end infrastructure, the back-end deployment model, and I'm loading a lot into the statement, so I'll be curious about your feedback, which then starts to get to an area, not only is it extra coordination and complication, but when you segment teams in that way, when you segment teams by what's the platform they're deploying to, are you dealing with back end? Are you dealing with front end?

Brian Dawson:

You not only get additional complexity following kind of Conway's law, you get a broken user experience. Now, I know I heavily packed that statement, but let's see if you have any comments or thoughts there if I'm interpreting you correctly, and what's your take on this?

Ev Kontsevoy:

Maybe I need to step back and simplify it a little bit.

Brian Dawson:

Okay.

Ev Kontsevoy:

I am desperately craving for more kind of localhost simplicity. So how could we make the process of developing for the cloud similar to developing for localhost, where everything runs on localhost, and everyone is happy?

Brian Dawson:

You're not worried... Right, okay.

Ev Kontsevoy:

It's basically again, it goes back to this kind of same Matrix-like experience. So just imagine, imagine if all these clouds that we have today, if you made them feel like one giant computer, like one computer. And that computer has an operating system. And that operating system provides you with a stable API and certain guarantees. And that API of the operating system is incorporated into the runtime of the language you're using. It's basically kind of similarly like libc and Unix, right?

Brian Dawson:

Yeah.

Ev Kontsevoy:

So it makes everything extremely simple and enjoyable again. Are we going to get there eventually? I hope so. Because then, we would... Like, for example, people sometimes are proud. They would say things work, "We engineer for chaos. We treat our servers like cattle. Our software is fully resilient."

Ev Kontsevoy:

I'm bored when I hear that. I don't want to even think of servers. I don't want to think about... Okay, just taking the localhost analogy, it's like you, you have a laptop. Okay, so if you have a laptop, it has RAM. And let's just say, each stick of RAM is like a server in that cloud computer. So if you say, "Well, I treat my RAM like cattle, certain DIMMs just appear and disappear inside of my computer," would you really want to build software on a computer like that? It's just ridiculous.

Brian Dawson:

Yeah.

Ev Kontsevoy:

You want the operating system to provide you with a certain level of guarantee, so you don't even need to think of which RAM stick is buffer is [crosstalk 00:44:34]. It just needs to go away. I was really hoping that Kubernetes will be like a step towards that future. But it looks like it's going to end up another layer on top of a pile of other layers, and we have to manage that plus a bunch of other stuff. And so, we'll see. We'll see. But it feels like we're reaching the breaking point.

Brian Dawson:

Yeah, yeah. And I do think, yeah, I love the way you summarized it. I think my heavily-packed statement was saying things in multiple similar ways. And yeah, no, I think it makes sense. I agree. I'll be a little bit more radical than I maybe normally would be in saying that that aligns with your statement that not only will we end up in a place where we're not talking about infrastructure, right?

Brian Dawson:

And we're not managing layers and layers of complexity, but we're also going to stop talking about bespoke delivery pipelines and processes. At some point, we need to end up at a place where there's a focus, and this will be buzzwordy, a focus on the logic, a focus on creating value, just allowing software engineers to show up, create logic, and deploy it without worrying about what's in between. But no, I love that vision.

Brian Dawson:

Now, let me ask, Ev, what is the role of open source in this, right? We're talking about the future, the future of computing, we've used the word kind of that Matrix vision. Where's open source building as things move forward? And what role does open source have in the future of software development in your mind?

Ev Kontsevoy:

So, there're basically... I think we would have very different conversations about open source like from a business perspective, what does it mean for an open source company, how do they make money? Or if you're a consumer of software, what opinion you should have about the software being open source versus proprietary, like closed source? But that's on one side. And then, from the other side it's basically the industry: the process of creating software. How is that changing, and what role open source plays?

Brian Dawson:

Yeah, I'd like to talk about both. I'd like to get your comment on both. Let's start with the first one. If I'm understanding your view correctly, they're kind of intertwined, right? They're not wholly exclusive. But more adoption and use of open source, which I think right now that we always have to have the debate about do we get gain? Say, I'm an organization looking to deploy, looking to create a new solution. I have to debate the pros and cons of deploying open source. And oftentimes, there are security concerns. There's indemnification. It seems like this view of a Matrix world has the potential to eliminate those concerns with adoption of open source. Do you have comment on that?

Ev Kontsevoy:

Yes, and I think it's actually directly related to what we were talking about just in the last five minutes. So, we have this software in-house so enormously complex. And the difficulty of running something in production is now higher than difficulty of building it in the first place. Are you following me?

Brian Dawson:

Yeah. Yeah.

Ev Kontsevoy:

The industry is naturally embracing open source because if they say, if the cost of creating something, if it's not a commodity, so the open source is out there, we're going to charge you for what's really hard, and that is running it in production. So this is why you see these companies that have products that are completely open source and they make money on their hosted cloud offerings.

Ev Kontsevoy:

However, it creates this crazy incentives. Like if you were building an open source solution, it's in your best interest making it incredibly hard to run. And people will come to you like, "Sell me the hosted version of what you've built." So, it's interesting, I think about this a lot.

Brian Dawson:

Yeah, yeah.

Ev Kontsevoy:

But going back to kind of security space, the interesting thing you mentioned too, I just now realized that you did ask me about it. Teleport has made the exception to this rule. Teleport is trivial to run yourself. It's a single binary, by the way, it's just a daemon. It's like sshd, but you would run Teleport and it gives you all these other protocols. So why would companies pay us money then?

Ev Kontsevoy:

Well, because in security space, it's one of these, I'm not sure, it's probably an exception to the norm, is that no one ever, in some sort of meeting setting, gets up and says, "From now on, I want to be responsible for security and compliance." If we ever get someone who's like, "I'm the person you're going to blame." Never happens.

Ev Kontsevoy:

It does happen with other things. For example, let's say you want to use open source database like Cassandra, right? But you don't really know how to run it really well. So then you're probably going to go get yourself the DataStax license. And then, you learn, eventually. And when you do, sometimes a certain engineer will get and says, "Yeah, I'm an expert in Cassandra now, having dealt with it a for a year. It's on my resume. We don't need that license anymore. I'll just do it." It just happens. Engineers develop a thing to love for certain technology, and then they feel like they don't need the help. It never happens with security. No one ever says, "I'm in love with FedRAMP Moderate compliance."

Brian Dawson:

So, so right. "I will own FedRAMP [crosstalk 00:50:41]."

Ev Kontsevoy:

Yeah, that's basically what we sell. We sell our expertise that is manifested in code. You download Teleport, you look at its code, and I believe it's beautiful. So, I'm proud of our repository, go take a look at how everything is engineered. And then, people want to pay us money, they want to have relationship with the vendor, who guarantees that their engineer is going to be productive and they will have amazing security and amazing compliance. And so-

Brian Dawson:

You hit on something on that, while we talk about Teleport, and noting that we at DevOps Radio and you as well, Ev, it's not about pitching Teleport, but there's a really interesting view on the world that is exposed to this. So I want to continue to dig it on Teleport a bit. Now, is there... So you have an open-core or open-source component of Teleport? Or what role does open source play? And... Yeah, go ahead, go ahead, sorry.

Ev Kontsevoy:

... Teleport is basically 99% open source that-

Brian Dawson:

Interesting.

Ev Kontsevoy:

... technically, I should probably say we are open core, but it assumes that there is a small core and a bunch of kind of enterprise stuff you could buy. No, no, in reality, the enterprise version, it basically has a few things in it that most small companies don't care about anyway.

Brian Dawson:

Okay.

Ev Kontsevoy:

So yeah, yeah, because in enterprise setting, for example, like Teleport can run in this FIPS mode, where we make a modified binary that doesn't even support certain ciphers and hashing functions that are not listed on like NIST, a recommended list. Anyway, it's stuff that startups for sure don't care about. Enterprises do. Just as I said, for larger companies, by having a commercial relationship with us, they basically could get a peace of mind and some of these scale-related capabilities.

Brian Dawson:

Awesome, awesome. All right. So, I want to shift and you've been pretty open and pretty transparent, so I'm really interested in hearing your answer to this question. On DevOps Radio, we have a tradition in asking people about their dev-oops moment. Not DevOps, but dev O-O-P-S, like dev-oops, oh, smack! Where you faced a challenge in your career.

Brian Dawson:

Oftentimes, people will respond technically, it could be from a leadership position, team-builder position, but a place where, to put it, frankly, you've screwed up, there were consequences, but you took a great lesson away from that. Is there something that you've experienced that you could share as a lesson to our listeners?

Ev Kontsevoy:

Yeah, someone asked me this question just a few days ago. Well, it made me think. So, it was a software bug, which I actually did not find. I eventually learned what it was with the help of another engineer. But so, in the beginning of my career, I was working on this HyperTrend control. It's a UI component that plots electrical signal, it goes up and down, and up and down.

Ev Kontsevoy:

And the deployment target, where it was supposed to run was industrial automation setting, like it's on the production line, for example, right? So naturally, it's 24/7/365, basically all the time, it's on all the time. And to make sure that the control and all this other machinery behind it, because it had a local cache that was reliable, there were no memory leaks.

Ev Kontsevoy:

So I would leave it running like on my work computer basically all the time. If it's a family vacation, I would keep it running like every weekend, just to make sure that I don't introduce any issues in there. And one day, I show up and it's blank, like there is basically no signal. I check everything I could, there was no memory leak, no exhaustion of GDI resources, for those who did Windows programming.

Ev Kontsevoy:

And so, I was chasing that thing for weeks and weeks and weeks. And then, I couldn't even reproduce it, again. I had like a special machine in the corner plotting and it was working just absolutely fine. And then, few months later, the same thing happened, it got blank again. And I cannot claim that I ever solved this issue. But then someone else made an observation like, "Did you notice that the two times when you saw this problem, they were exactly like, I think, six months apart?"

Ev Kontsevoy:

And it probably has something to do with that date-time shifting, when there are basically time jumps by an hour forward or backward. And that was the bug. So when I was plotting it, I was using current time as a part of this formula to compute basically, which pixel needs to be highlighted. And when the time shifted in my logic, it made the control basically, it started drawing off-screen. And so, that was kind of the first couple of years of my career. That was my introduction into time handling for programmers, it's incredibly [crosstalk 00:55:58].

Brian Dawson:

So, I have an idea about kind of a macro lesson or learning from that. But is there anything that when you think about that very technical mistake that you draw from that in approaching other things in your career?

Ev Kontsevoy:

Well, things are more complex than they appear, which makes me again, scared about cloud complexity. So if something as seemingly simple as date could create these issues where you spend months trying to chase why a certain pixel not visible on your screen anymore, so I can't even begin to explain what kind of fears I have by looking at all these stacks that we have in cloud [crosstalk 00:56:45].

Brian Dawson:

Yeah, and one misusage, one, yeah, that is interesting, one kind of innocent, inappropriate function call; sitting down, it's like the pee under the mattress fable, right? It could cascade all the way up to the top of the stack. Yeah, no, I get it. Your rightful in losing sleep at night and sweating at the fear of an end of... No, I'm playing with you.

Brian Dawson:

Now, that is interesting. And I do think it both plays the other way. Like, here's where abstraction can be dangerous, but also at more tactical level where abstraction is important, right? You tied very literal to current time. Yeah, we can probably have a whole episode unwrapping the lessons from that. I do appreciate you sharing that.

Brian Dawson:

My other standard question for you that I'd love to hear is, observing that you are very thoughtful, you think about this space a lot, you're very informed, it makes me that much more curious to hear what your recommendation is for a book, a podcast, a resource, a person to follow, that you would recommend to our audience. And noting this doesn't have to be technical.

Brian Dawson:

It could be something outside of this technical sphere that has informed you and you think it could help others. But do you have some things that you would point me and the audience to? Is there a book that changed your-

Ev Kontsevoy:

Mm-hmm (affirmative). So, how about this, I would maybe suggest a certain trend, or just more general advice.

Brian Dawson:

... Okay.

Ev Kontsevoy:

Because you probably have all kinds of people listening to us right now. So, let's just make an assumption that if you are software engineers, try to get into something that's not software. Because what I find that the most interesting thing happens when let's say you are an expert in the field, I'm just assuming that if you have the energy and natural curiosity, and you love what you do, you will inevitably become better and better and better at software.

Ev Kontsevoy:

You will find your own books to read. You will figure out some podcasts. It's inevitability in your career. Fine. However, to make it even more enjoyable, entertaining, but most importantly, to make you a much more interesting human being, try to venture into completely different field. Like one example, in school, I think we all take class, again, depends on which school you go to. So it's this technique of simulated annealing in software.

Ev Kontsevoy:

It's for certain AI algorithms, the way you jump out of the local maxima for a function and you look for a global maxima is to simulate how metal is melting and cooling. So, that's an idea taken from metallurgy and brought into software. Whoever came up with it knew something about how molten metals, how they cool, right?

Brian Dawson:

Yeah.

Ev Kontsevoy:

So, that what I find most fascinating is when software people get into something else, if you interested and get into optics, learn something about light. Get into psychology, and then see how you're... It makes you like a miniature superhuman because it gives you a domain, where you can apply your software superpower into a completely new field. I find it fascinating. So unfortunately, I don't have much free time these days, but that's what I would absolutely enjoy doing, is to explore something completely different.

Brian Dawson:

Fantastic, unexpected advice, actually. And so, I'll have a couple of comments. But I'll phrase it as go follow a resource that has nothing to do with software and develop expertise there. I also think that's interesting one, because it goes back, because I started my career in console game development, right? Creating virtual worlds effectively, right? Where you were dealing with assembly-level programming, some higher-order C libraries, how to get a chip to do things. But at the end of the day, you were getting a chip to do things to model the real world. So you had to learn physics, you had to learn psychology, you had to learn optics, you had to learn...

Brian Dawson:

And I always have and always will, in line with what you've said, carry the utmost respect, generally speaking, for game developers because it is not just about solving a technical problem. Then why do I think what you share is really powerful? It's because, as we've kind of highlighted with some other things, when you're sitting down and you're coding up a function, or you're invoking a function, the reality is, especially as we move to your Matrix world, Ev, is you're solving higher-order problems. And not only can you get creative inspiration for what you do technically and in software, but it also ties what you're doing in software to higher-order problems that you're solving. Anyway, this is getting really esoteric, but I love that answer, Ev, really resonates with me. Well, that's a... Yeah.

Ev Kontsevoy:

Hey, Brian, if you want a suggestion, I would love to listen to someone who is a lawyer with a software background on your podcast. Because if you think about it, what do we do with programming language? We basically convert from English to a language computer understands. And then, if you'll talk to lawyers, what do they do? They translate things to legalese. That's like a special language that is run by a legal system like by courts and judges. So, that's your computer, they interpret their language like if you've crafted the contract.

Ev Kontsevoy:

So, I'm really curious if you were good software engineer, and then you became a lawyer, what similarities do you see? Can we transform our entire system of making laws and enforcing them into basically a programming exercise? I don't know about it enough. But I just thought about it. It falls into that same... Uh-huh (affirmative)?

Brian Dawson:

No, I love it. I love the suggestion popping up in your mind. And I actually think we will make that happen. I can think about a couple of people that I know or come across that have moved between the CS field and the legal field. So great suggestion, we'll have to remember to credit you when we get the guest on to do that. I'd love to explore.

Brian Dawson:

So, Ev, it's actually been, as I anticipated, a joy talking to you. I know all good things come to an end. But before I let you go, what I would love to hear is your final words or final thoughts for the listeners of this podcast.

Ev Kontsevoy:

Well, I hate to be a broken record, but I would encourage all of us, everyone who works in this industry to think about why we're doing what we're doing and try to recognize that we've built incredible amount of complexity into our systems. We need to start realizing that we accumulated so much of this technical debt, that's really what I'm talking about. Maybe we should clean it up. Maybe we should start from scratch here and there. And maybe we should start openly saying about what we believe is obsolete right now and start kind of pruning it aggressively.

Brian Dawson:

Thank you for those words. It's funny, incidentally, aligns very much with recent conversations that keep coming up about Wardley maps. And I think that there's some overlap there. But no, great insight. And I will cap that off as a call to each of us as individuals in the industry to where/how can we take time to stop and reduce complexity, simplify so we can power ourselves into the future. And with that, Ev, I very much appreciate the time. I'm inspired by your thinking. I'm inspired by what you guys are doing with Teleport. And I hope some of our listeners are as well.

Ev Kontsevoy:

Well, thank you for having me, Brian, really enjoyed the conversation.

Brian Dawson:

Thank you. So, to the listeners of DevOps Radio, thanks for joining us today. Please be sure to tune in to our next podcast where we'll be having some other phenomenal guest like Ev, talking about all things state of DevOps. And with that, over and out.

Speaker 3:

All right.

Brian Dawson:

... [inaudible 00:00:00] and hit... Okay. Five, four, three, two, one. Hello, welcome to another episode of DevOps Radio. I'm Brian Dawson. And today I'll be your host and you'll join me in talking to Ev Kontsevoy, the CEO and co-founder of Teleport, and a man with a pretty impressive view on the current state and the future state of software development and delivery. Ev, hello, how are you doing?

Ev Kontsevoy:

Hey, Brian. Thanks for having me. I'm doing well.

Brian Dawson:

Yeah. Well, thanks for joining us. Our guests don't know, but you and I had had a chance in recent weeks to have an offline conversation together and I thought it would be awesome to get you on to share some of those thoughts with our guests. To get you introduced and get you started, of course, I gave a very simple introduction. Maybe you can give our listeners an overview of your role at Teleport or generally, what are you doing today?

Ev Kontsevoy:

Sure, sounds good. I'm a software engineer by training. There's a big difference between what I'm doing today and what I like doing today, but that's probably what is true with all of us. I love technology. I like building things, but at the same time, in my current role, I'm a co-founder and CEO of a successful and very quickly growing company. I am in charge of making sure that we continue to succeed, and I'm in charge of growing our executive team, in charge of getting our message out there. And it is very much enjoyable. So yes, I wish I had more time in front of a keyboard and the Vim editor, but it is what it is.

Brian Dawson:

Well, that is funny because I tell the story of... So for years, even though, I wasn't spending the majority of my time at a keyboard, I would always introduce myself as a software engineer or a software professional. It was a couple of years ago where I opened up a new laptop and I realized I didn't have an IDE installed. I hadn't open an IDE in way too long. I realized I couldn't call myself a software engineer anymore necessarily, but that is still where my heart's at.

Brian Dawson:

I tell my team, as we get into technical issues and building out environments, that I'm jealous. They don't believe me, but I miss the time. Now that said, you talked about an successful venture and helping ensure that you guys continue to grow. That of course, is Teleport. Obviously, we're not here to advertise Teleport, but help us understand what Teleport is and what problem you're solving there.

Ev Kontsevoy:

Teleport gives engineers, developers, DevOps engineers unified access to all kinds of cloud resources, that includes SSH, access to servers, access to databases like Postgres or MySQL, access to Kubernetes clusters, internal web applications. Anything you run behind NAT, Teleport connects you to all of these things using obviously, industry-best security practices, namely certificates and capabilities like single sign-on. So, it gives your command-line tools superpowers. It allows you to connect to everything you need to connect and keeps your security and compliance people at your company quite happy.

Brian Dawson:

Would you say that there's the bit of Teleport, well, I mean, obviously the name implies kind of what you covered, but it flattens the world.

Ev Kontsevoy:

Well, there is a reason it's called Teleport, right? Because if you...

Brian Dawson:

Right.

Ev Kontsevoy:

I'm very much inspired by the future of computing, the vision that has been presented to us in sci-fi literature like Neuromancer and Snow Crash. The idea of us using one giant multitenant computer, and they're all logging into it. It's basically what you see in the Matrix, right? The nice thing about being in the Matrix is you plug this thing in the back of your head, and you're in. And it's completely seamless, and security is actually quite amazing, right? Because whatever regular people can do, agents can do even more.

Ev Kontsevoy:

We felt that today's computing world, like the way we interact with all these different cloud computers, is extremely annoying, that you have to deal with different login mechanism, and role-based access control is different everywhere. There are like proxy servers and jump hosts. So, think of Teleport as the cord from the Matrix. You just login one time and then all of your command-line tools suddenly know how to connect to something. Yes, Teleport creates the illusion that the rest of your company's computers are in the same room with you.

Brian Dawson:

Yeah, that is phenomenal. On discussion for later is, when we reference sci-fi and that how we have to watch out for AI and ethics since we centralize all the computing, but I know in Teleport realm, that's conversation for later. Before we move on though, and dig more into that vision, I do want to get into your career and your path to where you're at, as we usually do here on DevOps Radio.

Brian Dawson:

So, as you said, you're an engineer. That's what you enjoy; spending time in Vim. But you've now founded in a running a successful company. What was the path between the guy that loves sitting down at a terminal or down at computer and coding to the guy that is creating a company that realizes the Matrix?

Ev Kontsevoy:

Whoa! I don't know where you want me to start. I just did... Mm-hmm (affirmative)?

Brian Dawson:

Well, anywhere. But we're curious, let's start with this: how did you get into computer science? How did you end up programming?

Ev Kontsevoy:

Oh, that's an interesting question. I was born in what used to be called Soviet Union. And as many of us, and probably including you, so I got exposed to computers through gaming, like really early when I was a teenager. And it became kind of clear that I want to do that. But in Russia at the time, they would say something like, you become computer scientists only after you fail at becoming a real scientist.

Ev Kontsevoy:

So with that kind of background, I decided that I needed to go get a degree in math, because it's the closest thing to computer science. Also, because Russia has a lot of fantastic universities that teach that. That's basically how I've gotten into it. Because prior to discovering computers, I felt that I wanted to be a aerospace engineer, because come on, flying things, isn't that exciting, right?

Ev Kontsevoy:

In terms of how you transition from getting into software, and then eventually, to co-found companies, I think you'd have to get lucky a few times. In my case, my first real job, like in terms of job I really wanted was I was a software engineer at a company called National Instruments. National Instruments was building virtual tools in a computer, that you would use data acquisition board plus some software to measure anything you want. So, that was basically one big giant idea behind the company. And I loved it. Having other engineers from all types of industries, from companies like Lockheed Martin, man, use what you do to build things that fly... It was just incredibly exciting.

Ev Kontsevoy:

And then, when I decided to explore what the rest of the world looks like outside of National Instruments, I was extremely not impressed with software that we make for people. It's like, really, I just realized that for me to be happy, I need to be building software for other engineers. Because I can put myself in their shoes. It's just something incredibly fulfilling, something satisfying about building software for engineers.

Brian Dawson:

That's interesting.

Ev Kontsevoy:

Right now, if you listen to me say that, it doesn't sound overly unusual like, "Yeah, sure. Go get a job. Build software for engineers." I'm talking about the year 2005/2006. It wasn't really clear what kind of job do I need to have? I got an offer from Microsoft at the time to go work, and they had a team called Developer Tools. So it's like working on CLR runtime or something like that. That meant I had to relocate to Seattle, right? I was in Austin, and I loved being in Texas. So that's really kind of my path, how I become became an entrepreneur because I needed a job. So I had to... I needed to eat. Yeah. Yeah. I got to-

Brian Dawson:

Okay. And you didn't want to... You needed a job and you... So I got a couple of things. One is you found empowerment in people using your solutions. You identified gaps in those solutions. You needed a job because you needed to eat and you didn't want to move. And those things came together to say, "I need to go out and create my own thing. I see a problem. I need to go solve it." Is that fair?

Ev Kontsevoy:

... Yeah, it paints me in a very positive light, like I was carefully scheming and planning all of that. No, no. It was actually an accident. So what happened? First of all, I didn't want to move to Seattle, I wanted to move to East Coast. That's really why I decided against to go into Microsoft. I moved to New York City. And around that same time, I went to an event like a developer meetup or something, where Jeff Bezos was talking to developers.

Ev Kontsevoy:

At the time, he was already a famous billionaire with these kind of giant personal bodyguards following him everywhere. But Amazon wasn't cool. It was not the kind of company where you wanted to run or host your application. It was a bookstore, launched EC2. I don't think it was called AWS at the time.

Ev Kontsevoy:

Bezos went on stage, and he did this presentation like Computing as a Utility. And at the end of that presentation, developers were asking him skeptical questions about, "Well, your cloud kind of sucks because of this and that," and they would kind of... Performance issues, or whatever. But someone asked him, "How come you cannot get any email in or out of your cloud?" Because without that... Because for spam-filtering purposes, I think they were blocking SMTP ports, or something.

Ev Kontsevoy:

I was sitting there in the audience, and that's really what gave me idea that I got to start a company. Because before, I thought if I want to build software for engineers, my mind was kind of locked in this mode like, "I need to sell libraries, software libraries" because that's really what software for engineers used to be. But that is like a dead market. Everything's open source now. So I would-

Brian Dawson:

Statically-linked SDKs, right? [crosstalk 00:11:50].

Ev Kontsevoy:

... Yeah, yeah, like some kind of SDK. That was kind of sad, so I felt like, "There is no way you're going to start this company." But then this whole exchange between Bezos and this random engineer made me realize, "Hey, I could just have an API and charge money for it. All these people who are going to start using Amazon's cloud, they will need the email capabilities." That's how the first company was born called Mailgun. So it was just me being in the same room with these two people exchanging questions. It's not like I was particularly prepared for that.

Brian Dawson:

Right, right. Particularly prolific and you had a grand vision. But no, that is actually a really interesting story. A couple of things to comment on there, one to squeeze in. You and I had a conversation. And I think I told you ultimately, I want to go start something on my own And I'm looking to build up the muscle. I don't remember exactly what you said. But your story encapsulates a bit of the response, which kind of came across as you just do it. It's not about... You didn't wait to build up a muscle to understand how to do it. You saw an opportunity, you saw a problem, and you went out and chased it, right?

Ev Kontsevoy:

Yeah, pretty much. I don't believe you can predict what kind of muscle you need to grow in order to do something big. Just like Nike says, right? Just do it.

Brian Dawson:

Just do it. But the other interesting thing that I just caught there, that correlates back to a conversation I was having with a gentleman from Dell Capital Investments. And he was marveled at how API... It's like, if you asked me 10 years ago if APIs as a service would be one of the largest growing segments in kind of the DevOps or software landscape, he would have never believed it.

Brian Dawson:

And if Tyler Jewell happens to listen to this, I know this not exactly what he said. But we were kind of marveling at that. Now, when we look at Mailgun, kind of this view you had and selling API access versus SDKs is actually turning it into one of the fastest-growth business models in our space now. There was-

Ev Kontsevoy:

Yeah, I [crosstalk 00:14:02] Twilio for that. Twilio, I think, was the first company that really, really fixed investors' minds. Before Twilio, I don't think we had a successful example of an entrepreneur raising a ton of money on a API idea. Maybe I'm wrong, but this is how it felt at the time. When I saw Twilio funding announcement, like series A, and then B, and so on, so forth, it felt like it was the first company that made investors believe, so Jeff has done a phenomenal job there explaining to them.

Ev Kontsevoy:

I failed. I went to some investor meetings and I said, "Hey, I'm going to sell you an API." I'm not going to name that gentleman, but I remember one of them just kind of, in a patronizing tone. He's like, "Ev, you don't understand. API, it's not a product. First, you build a product. And then second, you put an API in front of it." So, okay, whatever.

Brian Dawson:

... That's funny. Well, now, you can just silently send him some market reports with nothing else. So now I mean, look, that credits you with however you want to look at it. And not here to stroke your ego, there was a level of even though it sounds like you didn't know how to express it, or necessarily sell it, a level of vision you had there, that's manifested itself.

Brian Dawson:

Now, what I would like to ask, shifting a bit, is software engineer went out, made a company happen, had a reasonable... You exited with that, moved on to Rackspace, which gave you a new view of the world. And that ultimately led into Teleport. Actually, before I go into the next question that I have listed to ask you, tell me a bit about that transition. What did you see at Rackspace that led you to this view of future computing that's oriented around the Matrix and ultimately Teleport?

Ev Kontsevoy:

Well, maybe it wasn't so much about Rackspace. But with Mailgun, using that company technology as an example. So we started with a few bare metal servers, at SoftLayer. That was a hosting company in Texas that got acquired later by IBM. And then, the Mailgun team and Mailgun technology stack was basically following this cloud revolution, like 2009, 2010. '11, '12, '13, so on and so forth. So as all these different cloud providers were making new and new capabilities, we had to adopt that.

Ev Kontsevoy:

Just being there and witnessing how certain innovations making our lives better, and certain innovations make our lives more miserable, made me develop certain strong views on cloud computing. And then, when we got acquired by Rackspace, that gave me access to a lot of people who were in my shoes. I'm talking about engineering leaders at companies that use a lot of cloud computing, not necessarily Rackspace customers. So I met and talked to people at GitHub and companies like that where they were just sharing their pain points with me. And that basically kind of shaped my belief that the cloud computing soon it will be replaced with something dramatically better. That's kind of where my mind exists right now in the future.

Brian Dawson:

Okay. So, there's... But I have to say, somewhat controversially, I think I had shared this with you before. But it's, again, some wisdom to your approach. And that's that look, there's so much with kind of this DevOps movement and digital transformation and the motion to cloud, that you have a bunch of people out there speaking at a very high level, right? At a very macro level in terms of what it means.

Brian Dawson:

But I think sometimes we forget about kind of the tactical, on-the-ground bits and pieces that compose this movement to the future. And I say all this because when I first came across Teleport, and I'm like, "Yeah, well, SSH, that's not sexy, right?" And not to offend you, right? I mean, it's SSH. But there's a reality that this path to the future requires, that the tools we use, the processes we perform day-to-day, which may not be sexy enough to make it into a keynote presentation, actually, are the core of that movement to the future. Is that fair? And am I shocking you in saying discussions about NAT and SSH are not sexy?

Ev Kontsevoy:

Look, I'm used to it because at Mailgun people used to tell me, "Ev, why are you doing email in 2009? Email is not sexy anymore." I guess that's what I do. I just like my not-sexy things. But I have a way of making SSH sound sexy. Imagine if someone asks you, Brian, someone nontechnical, someone in your family who is, I don't know, into again, some completely different field, and they ask you to explain what this thing is.

Ev Kontsevoy:

So then, what I would tell them is that, look, you're interacting with the Internet, and everything that's on the Internet, using this thing called browser that's in front of your nose, and it has a protocol that it speaks, and it has this user interface. And there is like probably $1 trillion worth of value that being delivered to people who use browsers all the time.

Ev Kontsevoy:

But then, did you know that there is completely different side of it? So the Internet is split between people who are consuming it, and people who are making it happen. And those makers, those engineers in the shadows that make that possible, what do you think they use to get all that code out there to get all those configurations and get all the data in? They have completely different tool, and it's this secret browser you never heard about, and it runs on SSH, most of the time. So now suddenly, he's like, "Ooh, my God, it's almost like a dark matter of the Internet." Ha! Ha!

Brian Dawson:

Yeah, no, no, that is good. Right, yeah, there's all this background that people don't know. I talk about my kids a lot on the podcast. My son's an economics student. And he has to do some market analysis on SaaS. It's funny that you say that because we kind of spent hours... What I realized is he was asking, "Okay, what is AWS? What is the cloud, really? What is a VM? What is open source? What is Linux?" I realized that there is so much there that is hidden from the lay person, and that I didn't come up with a concise way to explain it. So you did it well. I will call back though, that you said, "I have a sexy way to explain SSH." I didn't know if the voice was part of it. No, that's a good explanation.

Brian Dawson:

All right. Well, let me move you to another question. So you are now a CEO and co-founder of two technology companies, both can be considered successful. What are some of the key lessons you learned. And we have a split of leaders, founders, practitioners, a widespread audience. But for that group, either those that are currently leading today, or those that aspire to one day, what are some things that you learned that you can share?

Ev Kontsevoy:

I can talk about maybe two things, and one of them is maybe more tactical and another one is maybe more macro. On a tactical level, and it's a mistake we've done when we started Mailgun. In the early days, we were overly influenced by the financial crisis. Because if you were already in the workforce, for those of you who are too young to remember that, it felt like there was no money in the economy. Investors were not writing checks, customers were not spending as freely. I'm talking like year 2009, maybe early 2010. So it made this kind of impression on us that we need to be extremely frugal, and we need to work super hard.

Ev Kontsevoy:

And ultimately, we basically worked ourselves into this situation where we felt extremely weak. This was one of the reasons why we said yes to an acquisition offer. In hindsight, it was a mistake. So when you starting a company, and you're starting a business, you must not treat that as a sprint. It has to feel more like a marathon or it actually has to be, not even a marathon, it needs to feel enjoyable, and it needs to be sustainable.

Ev Kontsevoy:

And so, think of a company as a way for you to build a pleasant reality around yourself. It allows your day-to-day life to be better, and your company needs to be a vehicle, how you're making your personal life also better. Because if it's not true, first it will be like a slight pain, and then eventually it will be agony. So, that was a tactical thing.

Brian Dawson:

Yeah, that's powerful. Before you go to number two, let me just ask you to clarify a word. When you guys worked yourself into a position where you felt weak, what do you mean by that? I'm assuming people were burnt out [crosstalk 00:23:32].

Ev Kontsevoy:

Uh-huh (affirmative). Actually, physically weak. I gained some weight. I started to develop some joint pains. This is what happens when you work crazy hours, when you don't have any time off. When you take every failure extremely personally, when you... I remember once I got an alert on my phone, that our database was dropping connections. It meant that someone's email was not getting delivered. And I remember feeling literally paralyzed waist down for a few minutes. But this just kind of sinking sense of panic. Don't do that to yourself. At the end of the day, it's just a job, so you have to manage your kind of psychology, all these things.

Brian Dawson:

And it sounds like at the end of the day, it's just a job. But if you truly care about it, if you put yourself [inaudible 00:24:26], at the end of the day, if that's the way you run your business, ultimately, you're not likely to succeed.

Ev Kontsevoy:

Exactly.

Brian Dawson:

Fair. Okay. And you had number two. That was awesome insight. And then, you said you had a more of a macro lesson.

Ev Kontsevoy:

I don't think it's a lesson but it's something that I've been thinking about it lately is that it's interesting how you phrased, in the beginning of this conversation, this kind of process of starting a company: you identify a customer need, you identify solution, and then you go and build a company as an answer to that solution. And I used to basically believe that that is indeed the case. Look, there is nothing wrong with it.

Ev Kontsevoy:

Most companies probably are founded this way, but if you want to inject some fun into it, and this is something that's probably happened with Teleport, to some extent, is I think it's more fun to create a company about a really interesting end state, about the future state. You want to change the world to be a certain way. And it could be something absolutely crazy, and being crazy is good.

Ev Kontsevoy:

Let's come up with really insane example. You've heard about this UFO case, right? Nimitz the aircraft carrier saw this Tic-Tac-shaped UFOs and all the pilots saw it. It was in the news. It was in the New York Times. All right then, so how do these UFOs move? People say that they bend the spacetime fabric. So then, you basically say, "You know what, let's start a company that will build a drive that basically, let's invent this propulsion method that UFOs use, something completely crazy."

Ev Kontsevoy:

And then, go and yeah, raise money to do it. Tell investors like, "If you want to be famous for solving this mystery, if we need to move away from burning tons of carbon to shoot one like 10 pounds of payload into orbit, let's go and do that." And by doing that, you will attract a lot of other crazy people too. They will join you to try to build this thing. But you know, the interesting thing? You'll probably fail. It's pretty, very unlikely, let's put it this way, that you will create this-

Brian Dawson:

You've got to learn how to bend the spacetime.

Ev Kontsevoy:

... but you will probably create a lot of other cool stuff in the process. That's really the thing. And not only your own life will be a lot more interesting this way, but I do believe that the kind of quality and the, not even quality, but the impact of things you will end up building will be way higher than anything that you could have done if you were just like trying to solve a specific customer problem. Something like this actually happened with us in Teleport. Teleport was originally started... Well, maybe we can talk about it later, but-

Brian Dawson:

No, go ahead.

Ev Kontsevoy:

... When you asked me about what did I learn in Rackspace? What is it I'm unhappy about? I am unhappy about state of the cloud. Because running software in the cloud today is incredibly complicated. The complexity of computing has gone in absolutely insane. It was for me, in the early years, that was why I abandoned Windows. Because building software for Windows was getting more and more complicated, but it was the same software, you just had to learn more APIs, you had to spend more time reading docs, you have to type a lot more. You just have a lot of typing to make simple things.

Brian Dawson:

For loving... So, Keith, I hate to interrupt because I always talk about this. And I always felt it was a shortcoming of mine. But as we got into object-oriented programming, and it grew in kind of the complexity underneath in computing, I used to get frustrated with the amount of time... To type a couple of lines of code, the number of docs I had to read, right? I had one of those rotating monitors that would sit and I was...

Brian Dawson:

I forgot, maybe it was C# at this time. But literally, to do anything, I had to have a monitor rotated sideways, and I'd spend half my day reading docs. Anyway, you just triggered a memory about some of the pain, but back over to you, right.

Ev Kontsevoy:

It is like some PTSD going on here. Absolutely. And then, I switched to Linux, and it felt simple, and it was awesome. "Hello, World!" started to look like "Hello, World!" again, like these little applications are possible. And then this cloud thing happened. We started to pile layers and layers and layers of abstraction. Like I remember I was at Rackspace, and I was talking to a company and they're asking me, "So what do you think our container strategy needs to be?" And like, "I don't know, what do you want them to be?" And they said, "Well, aren't containers better than VMs?"

Ev Kontsevoy:

I said, "Okay, if they are for your case, sure." And they were kind of happy. They said like, "Oh, it means we're going to replace all our VMs with containers." I laughed, and I was like, "Most likely, no. Most likely you're going to run containers on top of your-

Brian Dawson:

On top of VMs, on top of hardware.

Ev Kontsevoy:

... I just feel like we never tell people, "What are you making obsolete with this thing that you build?" Everything gets piled on top, on top, on top. And that is the craziness that we now have this industry. We call ourselves DevOps. We're DevOps people. Half of engineering budgets right now, it's companies are allocated to DevOps. What do we do?

Ev Kontsevoy:

Well, if you can go look into these technology stacks that we use, we have these layers like, "Okay, there's a Linux sitting on bare metal, that just hypervisor and VM sitting on top; that's another operating system inside. That VM doesn't exist in isolation. You use some kind of API to manage it, like EC2 API or OpenStack, whatever. That's another layer.

Ev Kontsevoy:

So, there is like a container running in it that has another image of an operating system. That container image came from somewhere, oh, there's a Docker registry over there. And all of it is managed by Kubernetes. That's another layer. So then, there is probably Grafana dashboard, and then there's MongoDB. And then there is NGINX. And there is Redis cache. And I'm not even talking about the actual application, but this stack keeps going up and up and up.

Ev Kontsevoy:

And now, think about it. Every layer is listening on a socket. Every layer has its own configuration file. Every layer uses some kind of encryption. Every layer has its own idea about what role-based access control is. And now, so you count all of this endpoints, all of these sockets, config files, and then you have to have people on your team with expertise on every single thing. That is insane.

Ev Kontsevoy:

The thing is you have to have it even when your application is relatively small. It's not like you can gradually kind of build up this thing. So it's insane now. You look at startups, like five-person startup, they just graduated from Y Combinator or whatever and they are already looking for a DevOps hire. That is just crazy. Because DevOps people basically run software manually.

Brian Dawson:

Well, yeah, and we have had some debates about [ESO 00:31:51], so hold your thought, I'm going to try not to take over too much. First, it's funny because you just captured. And I told you about this conversation I had with my son. It was exactly that. I mean, he's trying to wrap... I'm like, "Son, it's infinite layers, just take my simple explanation."

Ev Kontsevoy:

Turtles all the way down.

Brian Dawson:

But yeah, I mean, look we are trying to roll out a simple Django app or PHP app on the cloud, it is funny that you raise it. You literally can have a few hundred lines of code, and some configuration files. And it's deploying on this huge glacier of technology. Yeah, so that is... Forgot where else I was going to go. But yeah, that's really actually powerful, the way you call that out. Now, that explains the word, clunky that you used. Anyway, that's [crosstalk 00:32:39].

Ev Kontsevoy:

This is kind of like where I was going with this is to the realm of crazy ideas. So we decided that the world actually wants Heroku experience, it's pretty obvious. Like remember when Heroku showed up, it was like, "Oh, my God!" That it was like first true serverless platform that you could use without having to deal with all these layers. However, people want Heroku for their own use case. They want Heroku for their own scale. They want Heroku for their language of choice, whatever.

Ev Kontsevoy:

And that, we decided like, "Why don't we build a company that makes computing feel like the Matrix?" Because Matrix by itself, it's basically like cloud, right? It should be, should definitely be something like that; some kind of data center somewhere, some kind of servers. But when you interact with the Matrix, you don't feel presence of any layers, you get in, and then you basically execute instructions inside of the Matrix.

Ev Kontsevoy:

We felt, "Okay, let's build a company, and we're going to say that we will make your applications run anywhere in the world by themselves without DevOps people." And you might sound, "Well, that's a crazy idea. How can you make arbitrary application run anywhere in the world without DevOps people?" But okay, but what if we try? What if we try to build the stack? Where are we going to end up?

Ev Kontsevoy:

It's not exactly the new propulsion method to [crosstalk 00:34:05], but yeah, so we were not as crazy. It was more enjoyable, I'll tell you that. So if you start with something a little bit more abstract, it makes this kind of search for a new product much, much more interesting and enjoyable. And that's really where Teleport came from.

Brian Dawson:

And so, Teleport became the component problem of that larger propulsion vision, right?

Ev Kontsevoy:

Correct, so with Teleport, we realized that all of these layers of technology, they all have sockets they listen on. There is always security considerations. How do you control who has access to this database, to this SSH server, to this Kubernetes cluster, to this dashboard, whatever? So, if you were to manually configure access to every single thing, even if you use scripting, that is still a ton of work, and you need to have a lot of expertise. But most importantly, you creating a lot of opportunities for human error.

Brian Dawson:

Yeah, it's a lot of risk. It's a lot of risk.

Ev Kontsevoy:

Yeah, you will get owned. And so, what companies do, it's a combination of things they do, they either just cut access completely for their engineers and say, "You know what, no, you don't touch this from now on. You have to fill out a form." So you making people less productive if you do that. Or they will default to some hack like, "Let's have a shared secret, and we're going to use one password," or, "we're going to have this one SSH key that these five people can have." So they basically try to balance complexity with productivity, with usually mediocre outcomes.

Ev Kontsevoy:

And then, we've felt that, in the process of trying to build this Matrix-like experience, we did build these cordless phones, which is what Teleport basically does. It says, "I will natively speak all protocols that you use. I will give you credentials for all endpoints and all protocols using your identity, so you have to authenticate with Google or GitHub, or whatever it is your company use, and then Teleport will configure your command-line environment, so it's connected to everything you need."

Ev Kontsevoy:

But in doing so, not all engineers are super productive because they can access all of these things they want; SSH servers, Kubernetes clusters, databases. But the security people now don't have to configure every single socket, every single endpoint manually. It's all right [inaudible 00:36:30]. So, this is like a story, how we started with a relatively crazy idea: let's try to run applications anywhere in the world. And now, basically ended up with a new class of product, which we believe, called the access plane. You have a control plane, data plane, and Teleport is your access plane.

Brian Dawson:

Awesome. Yeah.

Ev Kontsevoy:

I like that, so if you ask me about these lessons you learned... And so, I definitely enjoy this experience of building something wonderful, but by trying to build something crazy first.

Brian Dawson:

Yeah. Yeah. I love that. And I forgot we were still on that seed topic. We forked on a bunch because I got a lot I want to dig in. Maybe we'll hit as we go further, but yeah, I do want to call out that that lesson both, so first, thank you for sharing those two lessons, right? One is when you're building a company, don't treat it as a sprint, treat it as a marathon or even better, ensure you're building it around something you enjoy doing, or else it compromises your sustainability and possibly puts you in a weakened position.

Brian Dawson:

Lesson two is shoot for the moon. Don't look to solve a point-today problem, but come up with something crazy. You'll fail, but in falling short, you'll land on something that really matters and you'll enjoy it while you're doing it. And I think there's somewhat of a commonality in both of those lessons, that I think if you go for the moonshot, you don't spend your time trying to make up a North Star that inspires people, that makes people want to get up every day and work and innovate to solve the problem. If you start with something crazy, the bright side is you have some inherent or built-in inspiration that motivates people when they show up every day. Is that fair?

Ev Kontsevoy:

Yeah. And I do think you get higher quality people joining you too.

Brian Dawson:

Yeah, all right, so now let's bring it back down a little bit. So some of what we hit there, in multiple points, we've talked about developers on the ground, practitioners, and how you talk to them and you identified pains they were feeling. Can we bring it back down a little bit and revisit what you're hearing from developers about their pains and how you feel you're addressing, or we need to address those things?

Ev Kontsevoy:

Well, are you talking specifically about Teleport or just in general?

Brian Dawson:

I think, in general. In general.

Ev Kontsevoy:

So, one thing that is now interesting, I'm not sure if it's a pain that developers are realizing, but it's definitely a question that we should be starting to ask ourselves. Why are we separating developments like front end and back end? Because if you think about it, so what is a front-end engineering?

Ev Kontsevoy:

So let's say, okay, using a browser. Let's just put mobile development aside, but okay, so using a browser to visit a certain website. As okay, the page is being served to you and certain code executes. So where is the code coming from? The code sits in some kind of server anywhere, regardless of is it JavaScript, or is it some Golang. The only difference between front end and back end, is that code is getting downloaded into a browser, and then it gets executed there.

Ev Kontsevoy:

And then, this one little silly fact, that just something gets downloaded, we basically have sometimes like two completely different recruiters inside of a company. One is, "Well, I'm hiring for front-end roles." And the other goes like, "I'm hiring for back-end roles." So I'm desperately waiting for this difference to go away. I don't think I'm the only one. We've been dreaming of [crosstalk 00:40:32] for a while, so trying to follow this Wasm space to see what happens.

Ev Kontsevoy:

Because I would just prefer as an engineer, I'm building an application, you could put like annotations to certain functions like you, you're going to run on my server side right next to database, right? You, you're going to run on a client, go into the browser. And you, you're going to run on edge and it's like a CDN PoP, right? And then, it's just same application, same programming language, same choice of libraries. I just want that experience badly, and I do notice that other people want it too.

Brian Dawson:

Well, I do know that that's a resurgence, so a bit of your precursor to Teleport, but well, I kind of generically put it at while you talk about the separation between front-end and back-end developers, gosh, there's a number of threads to pull there, I also hear a bit of, "Look, let's have one team" to put it grossly, right? Let's have one team that can write one codebase, taking a systems-thinking approach and deploy it anywhere, deploy it in multiple places.

Brian Dawson:

Let's focus on the logic, not the back-end infrastructure, the back-end deployment model, and I'm loading a lot into the statement, so I'll be curious about your feedback, which then starts to get to an area, not only is it extra coordination and complication, but when you segment teams in that way, when you segment teams by what's the platform they're deploying to, are you dealing with back end? Are you dealing with front end?

Brian Dawson:

You not only get additional complexity following kind of Conway's law, you get a broken user experience. Now, I know I heavily packed that statement, but let's see if you have any comments or thoughts there if I'm interpreting you correctly, and what's your take on this?

Ev Kontsevoy:

Maybe I need to step back and simplify it a little bit.

Brian Dawson:

Okay.

Ev Kontsevoy:

I am desperately craving for more kind of localhost simplicity. So how could we make the process of developing for the cloud similar to developing for localhost, where everything runs on localhost, and everyone is happy?

Brian Dawson:

You're not worried... Right, okay.

Ev Kontsevoy:

It's basically again, it goes back to this kind of same Matrix-like experience. So just imagine, imagine if all these clouds that we have today, if you made them feel like one giant computer, like one computer. And that computer has an operating system. And that operating system provides you with a stable API and certain guarantees. And that API of the operating system is incorporated into the runtime of the language you're using. It's basically kind of similarly like libc and Unix, right?

Brian Dawson:

Yeah.

Ev Kontsevoy:

So it makes everything extremely simple and enjoyable again. Are we going to get there eventually? I hope so. Because then, we would... Like, for example, people sometimes are proud. They would say things work, "We engineer for chaos. We treat our servers like cattle. Our software is fully resilient."

Ev Kontsevoy:

I'm bored when I hear that. I don't want to even think of servers. I don't want to think about... Okay, just taking the localhost analogy, it's like you, you have a laptop. Okay, so if you have a laptop, it has RAM. And let's just say, each stick of RAM is like a server in that cloud computer. So if you say, "Well, I treat my RAM like cattle, certain DIMMs just appear and disappear inside of my computer," would you really want to build software on a computer like that? It's just ridiculous.

Brian Dawson:

Yeah.

Ev Kontsevoy:

You want the operating system to provide you with a certain level of guarantee, so you don't even need to think of which RAM stick is buffer is [crosstalk 00:44:34]. It just needs to go away. I was really hoping that Kubernetes will be like a step towards that future. But it looks like it's going to end up another layer on top of a pile of other layers, and we have to manage that plus a bunch of other stuff. And so, we'll see. We'll see. But it feels like we're reaching the breaking point.

Brian Dawson:

Yeah, yeah. And I do think, yeah, I love the way you summarized it. I think my heavily-packed statement was saying things in multiple similar ways. And yeah, no, I think it makes sense. I agree. I'll be a little bit more radical than I maybe normally would be in saying that that aligns with your statement that not only will we end up in a place where we're not talking about infrastructure, right?

Brian Dawson:

And we're not managing layers and layers of complexity, but we're also going to stop talking about bespoke delivery pipelines and processes. At some point, we need to end up at a place where there's a focus, and this will be buzzwordy, a focus on the logic, a focus on creating value, just allowing software engineers to show up, create logic, and deploy it without worrying about what's in between. But no, I love that vision.

Brian Dawson:

Now, let me ask, Ev, what is the role of open source in this, right? We're talking about the future, the future of computing, we've used the word kind of that Matrix vision. Where's open source building as things move forward? And what role does open source have in the future of software development in your mind?

Ev Kontsevoy:

So, there're basically... I think we would have very different conversations about open source like from a business perspective, what does it mean for an open source company, how do they make money? Or if you're a consumer of software, what opinion you should have about the software being open source versus proprietary, like closed source? But that's on one side. And then, from the other side it's basically the industry: the process of creating software. How is that changing, and what role open source plays?

Brian Dawson:

Yeah, I'd like to talk about both. I'd like to get your comment on both. Let's start with the first one. If I'm understanding your view correctly, they're kind of intertwined, right? They're not wholly exclusive. But more adoption and use of open source, which I think right now that we always have to have the debate about do we get gain? Say, I'm an organization looking to deploy, looking to create a new solution. I have to debate the pros and cons of deploying open source. And oftentimes, there are security concerns. There's indemnification. It seems like this view of a Matrix world has the potential to eliminate those concerns with adoption of open source. Do you have comment on that?

Ev Kontsevoy:

Yes, and I think it's actually directly related to what we were talking about just in the last five minutes. So, we have this software in-house so enormously complex. And the difficulty of running something in production is now higher than difficulty of building it in the first place. Are you following me?

Brian Dawson:

Yeah. Yeah.

Ev Kontsevoy:

The industry is naturally embracing open source because if they say, if the cost of creating something, if it's not a commodity, so the open source is out there, we're going to charge you for what's really hard, and that is running it in production. So this is why you see these companies that have products that are completely open source and they make money on their hosted cloud offerings.

Ev Kontsevoy:

However, it creates this crazy incentives. Like if you were building an open source solution, it's in your best interest making it incredibly hard to run. And people will come to you like, "Sell me the hosted version of what you've built." So, it's interesting, I think about this a lot.

Brian Dawson:

Yeah, yeah.

Ev Kontsevoy:

But going back to kind of security space, the interesting thing you mentioned too, I just now realized that you did ask me about it. Teleport has made the exception to this rule. Teleport is trivial to run yourself. It's a single binary, by the way, it's just a daemon. It's like sshd, but you would run Teleport and it gives you all these other protocols. So why would companies pay us money then?

Ev Kontsevoy:

Well, because in security space, it's one of these, I'm not sure, it's probably an exception to the norm, is that no one ever, in some sort of meeting setting, gets up and says, "From now on, I want to be responsible for security and compliance." If we ever get someone who's like, "I'm the person you're going to blame." Never happens.

Ev Kontsevoy:

It does happen with other things. For example, let's say you want to use open source database like Cassandra, right? But you don't really know how to run it really well. So then you're probably going to go get yourself the DataStax license. And then, you learn, eventually. And when you do, sometimes a certain engineer will get and says, "Yeah, I'm an expert in Cassandra now, having dealt with it a for a year. It's on my resume. We don't need that license anymore. I'll just do it." It just happens. Engineers develop a thing to love for certain technology, and then they feel like they don't need the help. It never happens with security. No one ever says, "I'm in love with FedRAMP Moderate compliance."

Brian Dawson:

So, so right. "I will own FedRAMP [crosstalk 00:50:41]."

Ev Kontsevoy:

Yeah, that's basically what we sell. We sell our expertise that is manifested in code. You download Teleport, you look at its code, and I believe it's beautiful. So, I'm proud of our repository, go take a look at how everything is engineered. And then, people want to pay us money, they want to have relationship with the vendor, who guarantees that their engineer is going to be productive and they will have amazing security and amazing compliance. And so-

Brian Dawson:

You hit on something on that, while we talk about Teleport, and noting that we at DevOps Radio and you as well, Ev, it's not about pitching Teleport, but there's a really interesting view on the world that is exposed to this. So I want to continue to dig it on Teleport a bit. Now, is there... So you have an open-core or open-source component of Teleport? Or what role does open source play? And... Yeah, go ahead, go ahead, sorry.

Ev Kontsevoy:

... Teleport is basically 99% open source that-

Brian Dawson:

Interesting.

Ev Kontsevoy:

... technically, I should probably say we are open core, but it assumes that there is a small core and a bunch of kind of enterprise stuff you could buy. No, no, in reality, the enterprise version, it basically has a few things in it that most small companies don't care about anyway.

Brian Dawson:

Okay.

Ev Kontsevoy:

So yeah, yeah, because in enterprise setting, for example, like Teleport can run in this FIPS mode, where we make a modified binary that doesn't even support certain ciphers and hashing functions that are not listed on like NIST, a recommended list. Anyway, it's stuff that startups for sure don't care about. Enterprises do. Just as I said, for larger companies, by having a commercial relationship with us, they basically could get a peace of mind and some of these scale-related capabilities.

Brian Dawson:

Awesome, awesome. All right. So, I want to shift and you've been pretty open and pretty transparent, so I'm really interested in hearing your answer to this question. On DevOps Radio, we have a tradition in asking people about their dev-oops moment. Not DevOps, but dev O-O-P-S, like dev-oops, oh, smack! Where you faced a challenge in your career.

Brian Dawson:

Oftentimes, people will respond technically, it could be from a leadership position, team-builder position, but a place where, to put it, frankly, you've screwed up, there were consequences, but you took a great lesson away from that. Is there something that you've experienced that you could share as a lesson to our listeners?

Ev Kontsevoy:

Yeah, someone asked me this question just a few days ago. Well, it made me think. So, it was a software bug, which I actually did not find. I eventually learned what it was with the help of another engineer. But so, in the beginning of my career, I was working on this HyperTrend control. It's a UI component that plots electrical signal, it goes up and down, and up and down.

Ev Kontsevoy:

And the deployment target, where it was supposed to run was industrial automation setting, like it's on the production line, for example, right? So naturally, it's 24/7/365, basically all the time, it's on all the time. And to make sure that the control and all this other machinery behind it, because it had a local cache that was reliable, there were no memory leaks.

Ev Kontsevoy:

So I would leave it running like on my work computer basically all the time. If it's a family vacation, I would keep it running like every weekend, just to make sure that I don't introduce any issues in there. And one day, I show up and it's blank, like there is basically no signal. I check everything I could, there was no memory leak, no exhaustion of GDI resources, for those who did Windows programming.

Ev Kontsevoy:

And so, I was chasing that thing for weeks and weeks and weeks. And then, I couldn't even reproduce it, again. I had like a special machine in the corner plotting and it was working just absolutely fine. And then, few months later, the same thing happened, it got blank again. And I cannot claim that I ever solved this issue. But then someone else made an observation like, "Did you notice that the two times when you saw this problem, they were exactly like, I think, six months apart?"

Ev Kontsevoy:

And it probably has something to do with that date-time shifting, when there are basically time jumps by an hour forward or backward. And that was the bug. So when I was plotting it, I was using current time as a part of this formula to compute basically, which pixel needs to be highlighted. And when the time shifted in my logic, it made the control basically, it started drawing off-screen. And so, that was kind of the first couple of years of my career. That was my introduction into time handling for programmers, it's incredibly [crosstalk 00:55:58].

Brian Dawson:

So, I have an idea about kind of a macro lesson or learning from that. But is there anything that when you think about that very technical mistake that you draw from that in approaching other things in your career?

Ev Kontsevoy:

Well, things are more complex than they appear, which makes me again, scared about cloud complexity. So if something as seemingly simple as date could create these issues where you spend months trying to chase why a certain pixel not visible on your screen anymore, so I can't even begin to explain what kind of fears I have by looking at all these stacks that we have in cloud [crosstalk 00:56:45].

Brian Dawson:

Yeah, and one misusage, one, yeah, that is interesting, one kind of innocent, inappropriate function call; sitting down, it's like the pee under the mattress fable, right? It could cascade all the way up to the top of the stack. Yeah, no, I get it. Your rightful in losing sleep at night and sweating at the fear of an end of... No, I'm playing with you.

Brian Dawson:

Now, that is interesting. And I do think it both plays the other way. Like, here's where abstraction can be dangerous, but also at more tactical level where abstraction is important, right? You tied very literal to current time. Yeah, we can probably have a whole episode unwrapping the lessons from that. I do appreciate you sharing that.

Brian Dawson:

My other standard question for you that I'd love to hear is, observing that you are very thoughtful, you think about this space a lot, you're very informed, it makes me that much more curious to hear what your recommendation is for a book, a podcast, a resource, a person to follow, that you would recommend to our audience. And noting this doesn't have to be technical.

Brian Dawson:

It could be something outside of this technical sphere that has informed you and you think it could help others. But do you have some things that you would point me and the audience to? Is there a book that changed your-

Ev Kontsevoy:

Mm-hmm (affirmative). So, how about this, I would maybe suggest a certain trend, or just more general advice.

Because you probably have all kinds of people listening to us right now. So, let's just make an assumption that if you are software engineers, try to get into something that's not software. Because what I find that the most interesting thing happens when let's say you are an expert in the field, I'm just assuming that if you have the energy and natural curiosity, and you love what you do, you will inevitably become better and better and better at software.

Ev Kontsevoy:

You will find your own books to read. You will figure out some podcasts. It's inevitability in your career. Fine. However, to make it even more enjoyable, entertaining, but most importantly, to make you a much more interesting human being, try to venture into completely different field. Like one example, in school, I think we all take class, again, depends on which school you go to. So it's this technique of simulated annealing in software.

Ev Kontsevoy:

It's for certain AI algorithms, the way you jump out of the local maxima for a function and you look for a global maxima is to simulate how metal is melting and cooling. So, that's an idea taken from metallurgy and brought into software. Whoever came up with it knew something about how molten metals, how they cool, right?

So, that what I find most fascinating is when software people get into something else, if you interested and get into optics, learn something about light. Get into psychology, and then see how you're... It makes you like a miniature superhuman because it gives you a domain, where you can apply your software superpower into a completely new field. I find it fascinating. So unfortunately, I don't have much free time these days, but that's what I would absolutely enjoy doing, is to explore something completely different.

Brian Dawson:

Fantastic, unexpected advice, actually. And so, I'll have a couple of comments. But I'll phrase it as go follow a resource that has nothing to do with software and develop expertise there. I also think that's interesting one, because it goes back, because I started my career in console game development, right? Creating virtual worlds effectively, right? Where you were dealing with assembly-level programming, some higher-order C libraries, how to get a chip to do things. But at the end of the day, you were getting a chip to do things to model the real world. So you had to learn physics, you had to learn psychology, you had to learn optics, you had to learn...

Brian Dawson:

And I always have and always will, in line with what you've said, carry the utmost respect, generally speaking, for game developers because it is not just about solving a technical problem. Then why do I think what you share is really powerful? It's because, as we've kind of highlighted with some other things, when you're sitting down and you're coding up a function, or you're invoking a function, the reality is, especially as we move to your Matrix world, Ev, is you're solving higher-order problems. And not only can you get creative inspiration for what you do technically and in software, but it also ties what you're doing in software to higher-order problems that you're solving. Anyway, this is getting really esoteric, but I love that answer, Ev, really resonates with me. Well, that's a... Yeah.

Ev Kontsevoy:

Hey, Brian, if you want a suggestion, I would love to listen to someone who is a lawyer with a software background on your podcast. Because if you think about it, what do we do with programming language? We basically convert from English to a language computer understands. And then, if you'll talk to lawyers, what do they do? They translate things to legalese. That's like a special language that is run by a legal system like by courts and judges. So, that's your computer, they interpret their language like if you've crafted the contract.

Ev Kontsevoy:

So, I'm really curious if you were good software engineer, and then you became a lawyer, what similarities do you see? Can we transform our entire system of making laws and enforcing them into basically a programming exercise? I don't know about it enough. But I just thought about it. It falls into that same... Uh-huh (affirmative)?

Brian Dawson:

No, I love it. I love the suggestion popping up in your mind. And I actually think we will make that happen. I can think about a couple of people that I know or come across that have moved between the CS field and the legal field. So great suggestion, we'll have to remember to credit you when we get the guest on to do that. I'd love to explore.

Brian Dawson:

So, Ev, it's actually been, as I anticipated, a joy talking to you. I know all good things come to an end. But before I let you go, what I would love to hear is your final words or final thoughts for the listeners of this podcast.

Ev Kontsevoy:

Well, I hate to be a broken record, but I would encourage all of us, everyone who works in this industry to think about why we're doing what we're doing and try to recognize that we've built incredible amount of complexity into our systems. We need to start realizing that we accumulated so much of this technical debt, that's really what I'm talking about. Maybe we should clean it up. Maybe we should start from scratch here and there. And maybe we should start openly saying about what we believe is obsolete right now and start kind of pruning it aggressively.

Brian Dawson:

Thank you for those words. It's funny, incidentally, aligns very much with recent conversations that keep coming up about Wardley maps. And I think that there's some overlap there. But no, great insight. And I will cap that off as a call to each of us as individuals in the industry to where/how can we take time to stop and reduce complexity, simplify so we can power ourselves into the future. And with that, Ev, I very much appreciate the time. I'm inspired by your thinking. I'm inspired by what you guys are doing with Teleport. And I hope some of our listeners are as well.

Ev Kontsevoy:

Well, thank you for having me, Brian, really enjoyed the conversation.

Brian Dawson:

Thank you. So, to the listeners of DevOps Radio, thanks for joining us today. Please be sure to tune in to our next podcast where we'll be having some other phenomenal guest like Ev, talking about all things state of DevOps. And with that, over and out.