Accelerating Android AOSP and Embedded Linux Builds: Introduction
Embedded Linux is everywhere today. Look around and you can probably see a half dozen or more devices running some Linux-based operating system. Everybody knows about Android on mobile devices, of course. But have you ever thought about the software inside your LG TV? How about your Samsung washing machine? Or your Garmin smartwatch? What about the infotainment system in your GM, Ford, Audi, Volvo or Mazda vehicle? All of these feature Android or another variant of Embedded Linux at the heart of the system. If you're in the business of making software for these or any of the thousands of other devices running embedded software you probably already know the role Linux plays, and you may be familiar with the pain that it brings in the form of long build / test / release cycles. What you may not know is that there's a solution: CloudBees Accelerator. And even if you are a long-time user of CloudBees Accelerator, you may not be aware of recent improvements that specifically target the Android, Yocto Project and Buildroot ecosystems to deliver world-class build performance. In this first of a series of posts, we will discuss accelerating system builds for any of the three most common platforms used for embedded device development - Android/AOSP , Yocto Project and Buildroot . Using the latest 11.0 release of CloudBees Accelerator which ships with support for all three platforms, we will provide a high-level review of the different build architectures while providing some reference acceleration benchmark results. In the coming weeks, we will publish more in-depth articles describing how to use CloudBees Accelerator to accelerate these builds beyond what’s possible using other tools, while outlining strategies to consider for getting the most insights, performance and output from Android and Embedded Linux build environments. A survey by VDC Research in January 2018 showed that nearly 45% of users base their embedded development on open source operating systems like Linux. This is expected to grow by ~53% by 2021. VDC Research, January 2018
The Android and Embedded Linux (Yocto Project / Buildroot) builds
Android has been around for years, maturing into a viable alternative for a vast range of embedded device types. At CloudBees, we serve customers using Android to power their mobile phones, TVs, washing machines, watches, infotainment systems, RFID readers, point-of-sale systems, and more. In 2018 alone, nearly 3100 Google Play certified Android devices were launched, adding to the 16,000 or so certified devices that existed in the market at the start of the year. As staggering as these numbers may be, it’s noteworthy to highlight this is likely a significant underrepresentation of actual devices in existence, given the data only account for those launched by OEMs signed up with Google, thereby paying a fee to get certified.
CloudBees Accelerator 11.0 for Android AOSP
Our customers have relied on CloudBees Accelerator to accelerate their Android builds for years. Ever since the release of CloudBees Accelerator 7.0 in 2013, we have used the Android build as a reference environment to drive innovation in build acceleration and push the technological boundaries of CloudBees Accelerator. Noteworthy capabilities that we have introduced in recent years include:
Parse Avoidance for caching of makefile parse results,
Dependency Optimization for aggressive-but-safe parallelization, and,
Job Caching for reuse of output from compilers, documentation generators and other tools.
Among performance and reliability improvements, CloudBees Accelerator 11.0 introduces support for the Android LineageOS distribution. Using an Android 9 Pie build in our CloudBees Accelerator 11.0 reference environment, the below screenshot from the ElectricInsight build analysis tool for CloudBees Accelerator shows a simulation report where the y-axis represents the number of CloudBees Accelerator agents (~cores), and the x-axis represents time. The blue bars simulate CloudBees Accelerator build speed without enabling any of the Android-optimized capabilities, the green bars are with everything turned on. Simulating Android Pie build times with ElectricInsight For the optimized (green) setup, the report shows a significant acceleration already at low densities of cores/agents. From the report, we can also derive that we will need 132+ build agents with the unoptimized (blue) setup to get anywhere close to optimal acceleration. In future articles, we will go into much more depth on Android build acceleration.
As the term implies, Embedded Linux defines and builds a specialized configuration of Linux, tailor-made for embedded devices. That said, there is a lot of resemblance of Embedded Linux development environment to standard or desktop Linux distributions in how one reference and configure packages to define the system. These packages are then either directly compiled into the kernel or leveraged as 3rd party libraries or components in the system build. We have chosen to focus on the Yocto Project and Buildroot platforms, two popular frameworks for defining and building an Embedded Linux environment. The Yocto Project is an open source collaboration project that helps developers create custom Linux-based systems regardless of the hardware architecture.It has ~75 participating organizations, strategically funded and supported by industry household names like Intel, AMD, Arm, and Facebook. Buildroot is a vendor-neutral open source project with a growing and active developer community, featuring 2300+ supported packages and components. Other Embedded Linux build platforms exists such as e.g. OpenWRT, which we may look at in future articles. The below image describes the high-level architecture of the two Embedded Linux build systems, Yocto Project and Buildroot. While the goal of both these system is to produce a Linux kernel, bootloader and root file system, the tools and processes by these two platforms are quite different. Yocto Project uses bitbake as the meta build tool (originating from the OpenEmbedded ecosystem) and Buildroot leverages standard Linux kernel build tools, kconfig and make. To build each package included in the Linux system, Yocto and Buildroot are using a similar workflow of tasks, e.g. for retrieving sources, patching, configuring and compiling.
CloudBees Accelerator 11.0 for Embedded Linux
Our recent 11.0 release of CloudBees Accelerator improves support for Yocto Project and Buildroot. Specifically for Yocto Project with its bitbake tool, CloudBees Accelerator 11.0 adds support for caching of configure tasks, and includes the bb2anno utility to enable end-to-end bitbake build visualization using ElectricInsight. Below image presents benchmark results from a reference build environment, along with the high-level scope of the CloudBees Accelerator 11.0 support, accelerating configure and compile tasks of bitbake builds and all tasks of Buildroot builds. For our Yocto Project reference build, CloudBees Accelerator is reducing the overall runtime of the configure jobs by 14x due to the configure caching introduced in 11.0, while accelerating the overall build time by 46% or 1.9x. For our Buildroot reference build, CloudBees Accelerator is accelerating the build by 3x, bringing down build duration from ~24m to ~8m. In future articles of this series, we will dive deeper into the specifics of these reference benchmark results and the CloudBees Accelerator features making it all possible. The top 2 Mobile Android device makers in the world are using CloudBees Accelerator to power their private build clouds, accelerating software builds while optimizing infrastructure utilization. Automotive companies such as Visteon and Hyundai Mobis are leveraging CloudBees Accelerator to speed up their long Android and Yocto Project builds, helping meet time-to-market pressures. IoT device makers such as Zebra Technologies depend on CloudBees Accelerator for reducing their development cycle time. If you are looking to solve problems with your Android and Embedded Linux build / test /release process, we have a 30-day free trial that we would like you to try - please don’t hesitate contacting us !