Netflix is quite prolific in open source tools for cloud developers - in fact, due to their consumer popularity, and general scale (and visibility) when they speak about cloud - everyone listens.
They also have the coolest looking Github page I have ever seen - LOOK THERE NOW: http://netflix.github.com/
One project I want to briefly highlight is Asgard: Netflix have built an impressive, scalable and fault tolerant platform on Amazon primitives - they swallowed the platform whole - and what they produced is Asgard - a management service for their style of platform. Asgard is a webapp (written in Grails).
With Asgard - you define your application in terms of AMIs (images) - hermetically sealed images that contain your application, with only the configuration varying between instances. Firstly you define an application (initially just meta data):
After this - you define a cluster (which means Austoscale group, rules etc). The actual launching, and instance maintenance uses AWS autoscale at this point.
Of course, the other constructs of AWS are all represented in the Asgard console (such as DBs and other services).
It is quite a bit of work to set all this up yourself directly in Amazon - but what Asgard does is provide a focussed way for a static image deployment platform (which clearly works brilliantly for them). It is really nice that this was open sourced some time ago for anyone to use.
So - what is it for: well, ami/image based platform deployment (Netflix style !) and paring down AWS to the features needed to support this in a proven scalable way.
How to run it? I made a ClickStart
that starts this up in your cloudbees account (which is how I have been exploring it):
Then read the quick start.
This will install it with the right permgen, behind your CloudBees login automatically.
Unless you have been living under a rock - you probably have heard of the “Chaos Monkey” - well SimianArmy
is the project that contains this monkey. This works hand in hand with autoscale groups on AWS to check that things actually work - by terminating servers !
Building these on Buildhive (and gradle)
I have forked and setup both projects and they both build and pass tests successfully (at the time of writing) - this was trivially easy. Buildhive support the “gradle” build tool out of the box (it automatically detected the project type) - so for SimianArmy I told it to run the gradle test “task” (similarly for Asgard - which delegates to grails to run the tests).
(and checkout the build status badges). Pull requests submitted !
If you do work with AWS directly yourself - it is worth looking at Asgard - it does expect you to work a ‘certain way’ - but that is a battle hardened and proven way.
Kudos to Netflix !