Contributed by Ahmed Hosni, Lead Software Engineer, Vistaprint
In order to move fast as an organization, you have to move fast within your organization. As Vistaprint's lead software engineer, my job is to help the company stay agile—no matter how big we get—and to create value streams through software. My team writes the code that runs the business.
I like to keep things simple, and that means removing obstacles to productivity. I don’t want to lose time, energy, and focus dealing with physical infrastructure. This is why I moved our build environment from a set of in-house servers to ephemeral containers on AWS.
In the process, I also streamlined our codebase, standardized our tools and accelerated our release cycle. Hopefully, my story can illustrate how you can do the same.
MORE THAN BUSINESS CARDS
You probably know Vistaprint for our business cards. In reality, Vistaprint enables millions of business owners worldwide to market their business professionally. We provide a wide range of affordable, quality products that can be easily and fully customized to create unique marketing collateral for businesses of any kind.
More importantly, we are a technology company. Everything we do goes through our website. From the moment you pick the colors and the design of your promotional material to the final delivery to your home, office, or one of our shops, everything runs on software.
"We have gone from a three-week release cycle to a one-week release cycle...Now, we can move as fast as our imagination."Ahmed HosniLead Software Engineer
My team doesn’t touch the website. We work on the back-end, and we provide Vistaprint’s software engineers with a paved path, which is a development environment that helps them move faster and achieve shorter times to market.
We also do consultancy work, and we recently helped Vistaprint’s developer team move to an accelerated cloud-based CI/CD environment that streamlined our codebase and accelerated our development cycle.
A MONOLITHIC NIGHTMARE AND A "FRANKENSTEIN" SERVER
Our previous codebase was a monolithic nightmare that generated hundreds of deliverables. It was far too complex. To further complicate matters, we were using Apache Subversion as a centralized mission control system. Developers were stepping on each other’s feet, pumping out conflicting versions of codes.
We also had a "Frankenstein" Jenkins implementation. It was a single server that ran 154 agents with 211 installed plug-ins that conflicted with each other. It was impossible to maintain, and it slowed us down.
The best we could muster was a three-week release cycle, and we were under tremendous pressure to improve upon this. In e-commerce, time to market is crucial, and three weeks to troubleshoot a software conflict or to implement a new functionality is an eternity. Deploying once every three weeks also meant we only got feedback from stakeholders once every three weeks. Our approach wasn’t agile enough for our company’s fast-moving culture or for our customers.
HOMEBREW VS. CLOUDBEES
The biggest issue with our homebrew Jenkins implementation was lack of reliability. Simply put, the features we needed to ensure stability were not available as part of the open-source version of the platform. Chief among our concerns was buggy plug-ins. We could not rely on the Jenkins community to patch these in a timely manner, and so we went looking for enterprise versions of these vital extensions.
The other major stumbling block was the inability to create multiple Jenkins masters. With the open-source version, we could only operate a single master feeding dozens of agents. But with CloudBees CI (formerly CloudBees Core) we can run Managed Masters as a service and a single CloudBees Jenkins Operation Center that acts as a central authority over our entire Jenkins environment.
We started a new project to assemble an opinionated, modular, end-to-end solution for launching production-ready CI/CD environment on top of AWS. Fueled by Mesos/Marathon, we deployed our CloudBees cluster, and then we designed Multibranch Pipelines for our Monolith code base. Infrastructure as code has been successfully used to build pull requests on ephemeral windows containers that runs on Amazon Elastic Container Service (Amazon ECS).
On the other hand, each of our eight software engineering teams now have an individual master. This means that we can update and upgrade individual components of Vistaprint’s e-commerce platform separately, without affecting the whole.
In August of 2018, we deployed a pair of Kubernetes clusters on top of AWS, and then migrated our single CJE cluster to two CloudBees CI's instances running on two different Amazon regions. It was fairly straightforward. We moved credentials from the old stack to the new stack, harmonized plugins, and then we tweaked everything so it ran in Kubernetes’ containerized microservices environment.
CloudBees’ architects helped us move everything over. We are now using state-of-the-art technology to orchestrate containers, optimize scheduling and resource usage, and stabilize our releases.
MOVING TO A ONE-WEEK RELEASE CYCLE
We have gone from a three-week release cycle to a one-week release cycle. Every Tuesday, like clockwork, we push out a new version of our entire codebase. This has had a huge impact on the quality of our work. For starters, reducing the time between releases means our upgrades are even more incremental, and this means less code and fewer features to troubleshoot.
We’re also getting feedback from stakeholders three times faster than we used to. We know what’s working—and what isn’t—and can adapt quickly. The end result is more timely releases that are higher quality. With this reduced timeline, everyone inside and outside the team is excited to contribute to our success.
However, I am most pleased with the awesome developer experience we’ve created with CloudBees. Our homegrown CI/CD platform wasn’t cutting it. Moving our development environment to the cloud means hardware and software conflicts no longer bring us to a halt, and that infrastructure issues no longer slow us. We code and deliver reliable, reproducible builds faster, and if anything goes wrong, we can fall back to a previous state at the push of a button.
CloudBees CI has empowered my team to do more. Our next milestone is a daily build. We can absolutely achieve this from a dependency management and packaging perspective, but only because we’ve moved to the cloud. Once-a-day versions would have been impossible with open-source Jenkins and traditional server architecture, but now it’s in our sights.
The best thing about CloudBees CI is that it has been developed by some of the world’s best Jenkins engineers. The people who built Jenkins as an open-source platform are now working for CloudBees as consultants or as full-time employees. I now have the wisdom of the open source community and the stability of an enterprise version of Jenkins.
THE FREEDOM TO WORK BETTER AND FASTER
Moving to CloudBees CI has freed up a great deal of my time. I am no longer nursing a bad Jenkins stack or troubleshooting a Frankenstein server. When I do encounter an issue, I can write a couple of lines of code, press a couple of buttons to deploy it, and then go straight to configuration management.
Instead of wrangling infrastructure, I now work with my engineers to further break down our monolithic application into microservices. This will enhance our ability to deploy and troubleshoot new functionalities at an accelerated pace, and to recover from disasters with speed and ease.
CloudBees CI has made our CI/CD workflow smoother and faster. My team is more responsive than ever. In the future, I am certain that we will hit a one-day release cycle target. For now, I am happy that we are helping Vistaprint remain the leading e-commerce provider of custom printing and marketing solutions for businesses around the world. With competition breathing down our neck, moving slowly isn’t an option. Now, we can move as fast as our imagination.