Title: 7 Ways to Optimize Hudson in Production

Answers by Kohsuke Kawaguchi, Senior Architect at CloudBees.

Q. What about slave-initiated Hudson clusters? Swarm plug-in is too specific and slaves cannot be tuned individually.

A. You can launch slave via Java Web Start. This is one of the ways you can create a slave-initiated Hudson cluster. Once you launch a slave in this way, you can either install this slave as a Windows service from the GUI, or you can run "java -jar slave.jar -jnlpUrl http://.../slave-agent.jnlp" from your script to automate this. I'm curious what your problem was with the swarm plugin, though, as it's designed for this kind of usage.

Q. Is there any way to run Hudson slaves on Java 1.4?

A. No, unfortunately no. A decision was made a few years ago to drop 1.4 support, so that we can develop Hudson faster. Note that you need Java 5 just to run Hudson slaves, and you can continue to use Java 1.4 to build your software. I'm personally not aware of any platforms where the vendor provided Java 1.4 implementation but not 5.

Q. Does any restrictions exists on number of slave machines?

A. There's no real hard limit, and many installations run 100+ slaves on a single cluster. What seems to happen beyond the certain size is that for organizational reasons people start to prefer splitting a large cluster into smaller ones, so that everyone can control their own installation freely.

Q. Can you comment on the stability and reliability of the remoting infrastructure (especially with respect to the CopyArtifact plug-in, in light of issues like [HUDSON-7745])?

A. There appears to be some stability problem in the remoting protocol currently that's affecting some users. I've been trying to reproduce and fix the problem, but so far I've been unable to reproduce the issue. Any help on this issue greatly appreciated.

Q. If using labels instead of build machines, how efficient is the load balancing algorithm to select the least used host from the pool?

A. It uses the consistent hash algorithm, which is the same algorithm the session-aware HTTP load balancer uses. The algorithm itself scales very well, and is highly efficient. It also has a number of nice characteristics, such as locality, even loading, etc.

Q: Any built in support for running hudson slaves in the cloud (Azure/EC2/other)?

A. Yes. There's an EC2 plugin that lets you Amazon EC2 as a on-demand slave pool. This is built on top of an extension point, so a similar implementation is possible for other backends, such as Azure, Rackspace, etc. Jenkins Enterprise by CloudBees offers a similar feature on top of your own VMWare vCenter/ESXi. Perhaps the most comprehensive option to run Hudson in the cloud is our DEV@cloud.

Q. Is there / are you planning a class/webinar that will cover extending hudson (ideally for windows developers :) )?

A. I'd love to do that if there's enough interest! Please send us your thoughts -- info@cloudbees.com -- with suggested topics we should talk about in the future.

Q. When I start a build, where does that work happen in the file system?

A. Hudson assigns a directory to each build (called "workspace"), in a file system accessible to the build machine that's carrying out the build. Typically the workspace is assigned per project basis, but this can change for example if you allow concurrent builds to take place for one project (so that each concurrent build gets its own unique workspace.)

Q. For a Windows server and Windows slave: do you recommend using DCOM or ssh?

A. The only drawback to using DCOM is that it's harder to diagnose problems, when it's not working. When things fail, Windows APIs generally just report an useless error code, and you are left on your own to guess where things are wrong. On the other hand, installing SSH daemon on Windows is tedious, if not difficult. So that's the trade off you should think about.

Q. Is there any news on Mirroring?

A. As you can see in the post on Hudson Labs we've started delivering updates and plugins through mirrors around the world. The number of mirrors are increasing, and I think we have 4 to 5 now. If you'd be willing to be a mirror, please contact us.