Key Takeaways from Continuous Discussions (#c9d9) Episode 43: DevOps & CD for Non-Web Applications

Last week, we hosted another episode of our Continuous Discussion (#c9d9 ) podcast- this time discussing DevOps practices for non-web apps , such as IoT, embedded software and Hardware manufacturing.
Our expert panel included: Mark Dalton co-founder and CEO of AutoDeploy; Stephen Hendrick, ESG industry analyst; Juni Mukherjee, author of “Continuous Delivery Pipeline – Where Does It Choke? ”; J. Paul Reed, build/release engineering, DevOps, and human factors consultant; and our very own Anders Wallgren and Sam Fell.
During the episode, we discussed the differences between web-ware and firm-ware, the modern challenges and consequences of IoT, and the importance of simulators and version control in firmware deployment. Continue reading for best practices and advice from our expert panelists.

Differences between Web-Ware and Firmware

"the importance of quality in the web world is relatively low compared to firmware and hardware" - @SDH_ESG #c9d9 pic.twitter.com/zZbq8DER4U

— CloudBees (@electriccloud) June 14, 2016

Make sure you are integrating the #software and #firmware together so they are progressing at the same time - @samueldfell #c9d9
— CloudBees (@electriccloud) June 14, 2016

Reed on the differences of firmware and web-ware and why we should be taking IoT seriously: “There are the obvious technological differences , like the ways you do development, but one of the biggest things that comes up is the paradigm shift – the difference in mindset around these things. Part of the problem with firmware is organizations go into the mode of delivering software that way. The consequences in the IoT space are so much higher. We are talking about serious business - firmware in cars and potentially people. Is it cool that these devices are connected to the internet?”

Dalton talks about the importance of accounting for firmware in your CD practice: “While you have to be continuously innovating and continuously delivering with your software, you also need to be accounting for this firmware component as well. You need to make sure that as you have your pipeline as part of your continuous integration, there is continuous testing across the platform.”

Some benefits of firmware, per Hendrick : “Firmware is updatable and I think the whole notion of being responsive to change is ultimately a very powerful characteristic of firmware and one that ultimately, along with the performance you get from it, is going to be an important part of solutions as we go forward.”
Mukherjee

stresses the importance of simulators and emulators in firmware: “Let's say you have components being distributed and tested, imagine some of them are software and some of them could be hardware. At this early phase, should we involve the real devices? In the pre-commit phase of the pipeline, maybe we can use emulators, simulators. We actually did this with 3D printers - because it gives us the freedom in design that you will never get if you're constantly going back and forth in the supply chain.”

Challenges with DevOps for Hardware, IoT and Embedded

"You can have a lot of #agile cycles before you put anything on the #hardware " @samueldfell #c9d9 pic.twitter.com/ZqEMAeSoCG
— CloudBees (@electriccloud) June 14, 2016

"We don't talk about #versioncontrol and #realtimerollback enough" ... from @mdalton323 on #c9d9 pic.twitter.com/eMir7YB7mg
— CloudBees (@electriccloud) June 14, 2016

It’s no joke that the IoT has serious consequences. Mukherjee explains just how crucial some consequences are: “How important is it to get right? Let's say, in medical devices vs. the online world? I was in the online advertisement industry and I was doing some behavioral targeting to make sure the correct ad is being served. What happens when it doesn't get a click, nobody dies right? Nothing happens. You lose money. But if it's a medical device, you can't take a single chance.”

Cultural challenges are some of the hardest to overcome in embedded and IoT, says Reed : “The hardware people are the new operations team. They are the people you don’t talk to until the end. They are the people you might make fun of and they make fun of the software engineers. In DevOps we talk about this cultural aspect. One of the greatest challenges is, as we’ve found with ops teams, you have to start talking to the firmware people. How do we make DevOps work in embedded and IoT? Make sure you are communicating with the firmware and hardware teams.”

Simulators can remedy the challenges presented by firmware, and even defy physics! Hendrick explains: “One of the key takeaways is the notion of simulation. It becomes critical when you deal with firmware and hardware, because committing something to build a prototype is very expensive. And if you can build a software simulation of firmware or hardware, depending on what it's supposed to accomplish, you can be way ahead of the game from the standpoint of being able to understand how the firmware of the devices will work in the real world. In fact you can do things with these simulations that you can't do in the real world - you can defy physics.”

Another recurring theme from the discussion? Version control. Dalton explains its importance in firmware: “Real time rollback is absolutely critical, it is something that we have adopted and we implore our customers to adopt because as soon as a bug is out in production and a user finds it, you have to be able to go back to the previous version. It doesn't matter what it takes, you need to have full version control across the spectrum.”

Tips for DevOps for Non-Web Apps

If you've written down a process and get a human to follow it, you can also codify it- #automate ! @samueldfell #c9d9 pic.twitter.com/E0BHrnIE79
— CloudBees (@electriccloud) June 14, 2016

Make sure you are doing a good job at dependency management to ensure #security for #embedded - @jpaulreed #c9d9 pic.twitter.com/tqeYXnJZ0c
— CloudBees (@electriccloud) June 14, 2016

Have one image so one image can have one version to eliminate variability - @JuniTweets | #c9d9 pic.twitter.com/9Hg2ESc2aW
— CloudBees (@electriccloud) June 14, 2016

"As you move from web toward hardware the notion of design becomes more important" --@SDH_ESG #Agile #c9d9 pic.twitter.com/57UfJD3LZL
— CloudBees (@electriccloud) June 14, 2016

"automate everything--build and deploy, have robust communication for #visibility " - #tips from @mdalton323 on #c9d9 #DevOps #software
— CloudBees (@electriccloud) June 14, 2016

Here’s a list of key tips from Dalton : “A couple of key things : have a single source repository, automate everything, build and deploy, make the build self-testing, make sure that you’re testing a clone of a production environment and make sure that you have a robust process communication so that everyone is seeing what is going on across the entire spectrum of delivery.”

What’s the one thing app developers should be focusing on, per Hendrick ? Design! “I think there's an emphasis on design that needs to be taken more seriously. We need to have fewer surprises with regard to the devices that get produced at the end of the day, which will help drive down the cost of change. Good design is absolutely paramount.”

Google is not going to answer your problem here, says Reed : “DevOps is very emergent in the embedded and IoT space. Don’t think you’re going to Google ‘Continuous Delivery’ and take the top ten posts and do that in an embedded world because you are going to get burned and you will think CD sucks. You need to look at and have some conversations with people who have deep CD and CI knowledge and apply that knowledge to the context of the problem.”

Once again, version control and having the ability to roll back to the last known good version is crucial. Mukherjee gives an example: “If the outcome is that your house is burning, so be it, we need to think of that upfront, not when the house is burning. Not when your car has stalled, that's way too late.”****

Watch the full episode here:

Want more Continuous Discussions ?

We hold our #c9d9 ’s every other Tuesday at 10 a.m. PST, which features expert panelists talking about DevOps, Continuous Delivery, Agile and more. Next time on Continuous Discussions: Fostering a Culture of Continuous Improvement .