A number of years ago James Urquhart discovered something interesting about the internet. “It resonated with me that the internet is becoming this massively complex adaptive system,” says James, who is currently global field CTO at VMware. “There was no central enterprise architecture where everybody knew where their pieces fit.”
Since then, the concept of complex adaptive systems has grown in influence, becoming an important branch of science that is changing the way that businesses think about working and interacting with each other. For businesses to thrive today, James says, they need to be able to adapt and respond to events as they change in real-time.
Listen to James on Episode 91 of DevOps Radio
This simple but powerful idea is one of the reasons James was attracted to “event-driven integration” and the related concept of flow architecture. “Flow is the movement of real-time data to all the different parties that could gain value from that data,” James says. “It basically focuses on that movement of information across organizational boundaries.”
The goal of flow is to make it as simple as possible for organizations to subscribe to multiple streams of data. It’s how businesses will increasingly integrate with one another to meet customer needs. James sees a trucking company, for example, subscribing to a weather stream to get the latest weather report in real time, and then meshing that with the latest traffic data flowing in from another stream. “Now I can plan my truck routes and schedules more precisely,” James says. There’s just one problem: Currently it’s hard to integrate events and data streams between organizations. “When you look at subscribing across organizations, there are no standard interfaces or standard protocols to do that today,” James says. “Each one has a whole different streaming process.” For example, if you subscribe to a Twitter stream, “you can’t just take that exact same connection code and point it to a tech stream coming from, say, Twilio.”
Event-Driven Integration Is on Its Way
But that day of seamless integration may come sooner than you think, James predicts. “There are companies already working on new forms of network connectivity that are very ‘publish-and-subscribe aware’ and that focus on connecting to a topic that's distributed across multiple locations.”
In fact, James explores that concept in-depth in his new book, “Flow Architectures: The Future of Streaming and Event-Driven Integration.” In it, he makes the case that the coming arrival of standard protocols and standard publish-and-subscribe interfaces will make it easy for businesses to connect to streams from other companies, nonprofits and governments. What’s more, when businesses or organizations adopt those standards, James says, they’re going to see a marked change in the way they do business with each other. Why? Because the cost of integrating real-time data “will go so low that you can do things that are just impractical to do today,” James explains. But the real driver of the move to this event-driven integration will be better customer experiences; businesses will discover new activity sources to enhance their business and drive new markets. “Real-time inventory is already taking advantage of flow, making it a no-brainer for any business to be able to supply to another business,” he says.
DevOps + Event-Driven Integration Together
Since joining VMware, James has been helping top IT leaders and engineers leverage DevOps and agile approaches to boost their businesses. “It's a big challenge,” he says. “There's a lot of information out there about how to run DevOps teams, but as you get to larger and larger teams that are interdependent on each other, it's a very difficult problem to manage.”
This is where James thinks streaming and event-driven integration can make a big difference, and he’s been exploring the use of flow concepts and technologies to maximize DevOps economies of scale and efficiencies within an enterprise. Looking ahead, James predicts that dev toolchains are going to be increasingly event-driven and composable, enabling software developers to bring in different capabilities as needed.
For example, “you could easily insert compliance automation in the right places in your process and be able to move that around as you learn where it’s most useful,” James says. “You can have a stream that comes into your environment to help with that.” In fact, uncovering the potential value of flow is what his book is really about, James says. “It's about laying out why people will care.”
James Urquhart is a strategic advisor at VMware and author of Flow Architectures: The Future of Streaming and Event-Driven Integration. To hear more of James’ story and the opportunity to win a copy of his book, listen to Episode 91 of DevOps Radio.