Are Clusters a Dying Technology?

I happened across a blog today that made the claim that accelerating builds by distributing to a cluster of computers is "a dying technology." Instead, they said, you should take advantage of increasing CPU density to enable increasing levels of parallelism in your builds — a single system with eight or more cores is pretty affordable these days. So is it true? Have we reached a critical point in the evolution of computer hardware? Are clusters soon to become obsolete? In a word: no. This notion is rubbish. If the massive push to cloud computing (which is all about clusters) isn't convincing, there are (at least) three concrete reasons why clusters will always beat individual multicore boxes.

Memory and Disk Bandwidth

Sure, Intel and AMD and Sun are jamming more and more cores into each CPU, and enabling you to put more CPU's in each box. Unfortunately, advances in memory and disk bandwidth aren't keeping pace. For example, AMD's latest, greatest Opteron is a 6-core processor. You can put four such processors in a single box for a total of 24 cores. The effective memory bandwidth has been demonstrated at about 41 GB/s , or 1.7 GB/s per core. An older system using two dual core processors gave 12 GB/s, or 3 GB/s per core. To get the same number of cores, we would need six of the older boxes, but we would have memory bandwidth nearly double that of the single-box solution . No doubt memory speeds will continue to improve, but as they say, a rising tide lifts all boats — that is, faster memory will help both the cluster and the single-box system.

The same argument holds for disk bandwidth. A typical modern disk can deliver about 80 MB/s during reads. You may be able to find a really fast disk that can do better than that, but are you confident you'll be able to find one that is twice as fast? Four times as fast? Eight times as fast? By some accounts, we may be at the practical limit of how fast disks can go .

The bottom line is that even if memory and disk bandwidth does plateau, I'll always be able to increase the effective bandwidth per core by growing my cluster, but not by simply slapping a few more cores into a single box.

Cost Efficiency

Let's say you've decided that you don't want to deal with a cluster for build acceleration. You've done some tests and you've found that your build really zings on a 8-core system — the build time is comfortably within the targets your VP of Engineering mandated. Nice work! Now, how many of these systems are you going to buy? I mean, surely you want all of your developers to enjoy the benefits of fast builds, right? You weren't thinking you could buy just one or two of these boxes for the release engineering team, were you?

OK, let's say you've decided to go ahead and buy one of these killer machines for every one of your developers. What kind of utilization do you think you'll get from that hardware? Suppose you have 10 developers, and they each do 5 builds per day, and each build runs for about 15 minutes. That's 75 minutes of usage per box, per day. The rest of the time — 22 hours and 45 minutes of every day — those boxes are completely idle. You're using the hardware only 5 of the time. Still think your VP is going to be happy about that?

Maybe you try to increase utilization by buying one of these systems for every two developers, or every three, and asking them to share. Sounds like a hassle to me — every time I want to run a build, "Hey Mike, you using the build machine?" I'll pass on that idea, thanks.

Now compare that to a cluster managed by CloudBees Accelerator. First of all, we don't need to buy as many cores. We only need enough to support the peak load — let's say that's four simultaneous builds. At eight cores per build, we need a cluster with 32 total cores, versus 80 cores total if we get an 8-core box for each of our 10 developers. We've already saved a bundle just on hardware. Second, Accelerator has powerful cluster sharing facilities that make it trivial for all of your developers to share those cores. Accelerator will allocate a "fair share" to each currently running build. Since everybody is using the shared resource (without the hassles of manually sharing), utilization goes up to about 13.

Scalability

The last and best reason to use a cluster instead of a beefy single system is that the cluster is more scalable than the individual boxes. Sure, an 8-core box is pretty affordable these days. But to us, 8-way parallelism in a build is kid's stuff. With emake it's not uncommon to use 16, 24, even 50 cores or more in a build. No matter how many CPU's you put in your single system, it will always be possible to build a cluster with more resources, and with a tool like emake you'll be able to effectively use all that horsepower.

Even more important than that though: using a cluster instead of a single system "future-proofs" your acceleration solution. That 8-core box may be able to deliver a 15 minute build for you now, but you're not going to freeze development, are you? Of course not. Your existing projects are getting bigger, and your adding new projects all the time. For example, the Linux kernel has grown between 2 and 8 in each of the last several versions — a total increase of 25 in a little less than 18 months. More code means longer builds: that 15 minute build is going to creep back up to 20 minutes, then 25, then 30, and on and on. If you've invested in standalone 8-core systems for each of your developers, how will you get back to that 15 minute build target? You can't very well throw a few more CPU's into those boxes. But with a cluster, you can do just that — need more capacity and processing power? Just plug in a couple more boxes and you're good to go.

Conclusion

So, are clusters truly a "dying technology" in the build acceleration space? Only if your build tool is incapable of taking advantage of a cluster. But if you've got the right tools, a cluster is the only sane solution. It's faster, more cost effective, and more scalable than a single box can hope to be.