CI is a critical practice for all development organizations. Years of practice have shown us that smaller code changes, delivered at speed, improve quality. This makes sense, since CI closes the feedback loop to developers, makes Continuous Delivery and Deployment possible and keeps our backlogs small.
Jenkins is the best-known platform for CI/CD. It's been with us in one form or another since 2005 and has grown into a robust platform that can support thousands of jobs and a wide variety of features. It's simple enough to set up as a personal CI/CD server, or can scale up to the enterprise with integrated authorization, cross-platform support, and distributed build agents.
Let's go over three DevOps World 2022 talks covering CI and Jenkins. We'll look at how to use Dagger and Jenkins to simplify local builds, how Broadcom reduced their Jenkins job failures, and then at the future of the Jenkins user interface.
Keep DevOps Local (and in CI!) With Dagger and Jenkins
Using CI is easy. Commit your code often, and build and test it with each commit. But easy isn't the same as simple. In Keep DevOps Local (and in CI!) With Dagger and Jenkins, Rebecca Dittmar, a Software Engineer at Nike, and Jeremy Adams, Head of Ecosystem at Dagger, discussed some common CI issues and how to use Dagger with Jenkins to address them.
Developers are often faced with these issues when they’re using CI:
Code works locally and builds in CI, but fails when they commit and push
Errors in Staging or Production environments that can't be reproduced back on the developer’s system, or even in QA
Slow CI build systems that need 15 minutes or more to complete and build and test
Depleted CI resources that bring your pipelines to a halt because of too many simultaneous jobs
According to Jeremy and Rebecca, these problems are often caused by conflicts between local and CI environments, and Dagger can help address them. Dagger is a cross-platform CI/CD engine. It's programmable and container native. So, you can easily deploy it anywhere while integrating it with your Jenkins system. You get the best of both worlds; local builds with CI consistency. It also integrates with native tooling for most major languages.
Rebecca and Jeremy ran a demonstration of how you can integrate Dagger with Jenkins to create a CI pipeline in minutes. Jeremy demonstrated how Dagger uses caching to speed up repeated builds, and how to pull a build down and run it locally while still maintaining compatibility with the Jenkins environment.
Watch the talk to see the demo in action.
Reducing Jenkins Job Failures by 35% at Broadcom
In the breakout session titled, “How Did We Reduce Jenkins Failures at Broadcom?” Peter Liu, Software Development Manager at Broadcom, shared a story about how his team worked to make major reductions in Jenkins job errors at Broadcom. Peter runs a team of seven engineers that support more than 400 developers distributed over roughly 50 teams. They support dozens of Jenkins systems that run nearly 4,000 jobs each day.
Peter's team deals with a set of problems that looks familiar to any enterprise engineer:
It's hard to identify who's responsible for a failing job
Communications between teams is often difficult, if not impossible
Errors are often routed incorrectly
Most engineers have been trained to ignore errors because there are too many
Broadcom's Jenkins systems would often see as many as 1,000 job failures a day. Peter explained how they used a data-led approach to improve their CI/CD processes and reduce the count by at least 350 per day.
Their process consisted of five steps:
Isolate the reasons for failures
Identify the teams responsible for the issues
Use the economic impact of each failure to prioritize
Automate the solution to reduce human error
Track the impact of their efforts
Broadcom used Propelo's engineering excellence platform to correlate the data across their pipelines. It analyzed the Jenkins failures and uncovered common failure patterns. Peter's team was then able to categorize the problems into different classes.
Infrastructure issues: disk space, connectivity, memory consumption, etc.
Job issues: bad commits, broken build scripts, authentication problems, etc.
Armed with this information, the team was able to create better lines of communication with each team, and target alerts more accurately based on their class.
Check out Peter's presentation for more details on Broadcom’s approach.
Transforming the Jenkins Interface
Jan Faracik and Tim Jacomb, both developers at Kainos, led a breakout session titled, Transformation of the Jenkins User Interface and Where It’s Going Next. They shared where the Jenkins user interface is going and how it will get there. Jan and Tim have both been working on Jenkins for about four years, and as members of the UX group team, they had a lot of knowledge to share, especially for plugin developers interested in implementing the latest user interfaces.
The UX group is focused on modernizing Jenkins, with an emphasis on easier-to-use, more accessible, and having it scale better to different form factors. These efforts include:
Revamping Jenkins forms
Designing a new iconography
Simplifying Jenkins navigation
Jan opened with a brief history of Jenkins, including a screenshot that will look familiar to many long time developers watching this talk.
He discussed how Hudson was forked to become Jenkins, and how the interface improved from there, but the interface has remained consistent for a long time. As the development teams have added new features, the forms and navigation have grown more complicated. It’s time for a change.
Jan shared where Jenkins is going, and any regular Jenkins user would benefit from seeing what he shared. With the end of support for Internet Explorer, Jenkins is able to take advantage of many new web technologies, and the new interface is cleaner and much easier to use. Jenkins is also slated to see support for themes.
Here's an example of a new screen:
Plugin developers will benefit from new interface guidelines, a robust design library, components, and a new color palette with semantic coloring improves usability.
Jan and Tim's breakout session revealed some exciting developments for Jenkins users and developers alike. Check it out now!
CI and Jenkins
We've covered three of the many talks presented this year at DevOps World 2022. This time we focused on CI/CD and Jenkins, and saw how you can use it to make your pipelines more reliable and more robust. We also took a peek at where the platform is headed.
Take a look at the rest of this year's online conference. We hosted some exciting talks from vendors, breakout sessions hosted by industry experts, and presentations from developers that are solving problems just like yours.