Andy Glew on the need for discovered dependencies

Andy Glew's written a piece over on his blog opining about the need for automatic discovery of dependencies and some applications of having that data. Understanding dependencies is a requirement for any (reasonable) build tool, and all build tools allow the specification of dependencies - even shell script: specifying a serial order lets you define fully implicit dependencies! - so if you had a source for complete, accurate dependency information you could generate a makefile, or an ant build.xml, or a shell script with a serial order, from that information, thus allowing a build to use any tool.
One of the challenges to that approach in practice is that most developers have a build tool of choice (make, ant, SCons, Visual Studio, etc) and there's a resistance to using a meta-language to describe all the other bits (compiler options, libraries, variables) which are the other necessary parts of a build system. Getting developers to write in your tool/language is the sticky wicket, since a developer is going to need to change all the little details at some point, and that developer is going to be more comfortable doing it in the tool she's accustomed to. And some tools don't easily map 1:1 - lots of things which work well in SCons are hard/impossible in make, and migrating make to SCons is non-trivial.
In the end, Andy's right on, though. The dependencies are the hard, magic bit - and in large software systems with modular structures and recursive build processes, getting accurate depdendencies is *hard* - so if you can get good deps the tool platform problems can be solved in lots of ways.
Andy has a couple questions in his article I should address, since I work for CloudBees:
"Past conversations, either with Ousterhout or with EC sales folk, lead me (1) unsure as to whether they have done the truly automatic thing for dependencies but (2) sure that they are not "diversity tolerant" - they are yet another tool that requires everyone, the whole project, to use their build tool."

1. Yep, we really have done the truly automatic thing for dependencies - CloudBees Accelerator tracks at the file-system level what files are read and written, collecting the data required to build the dependency graph for a build (and also letting CloudBees Accelerator know when a target was prematurely fired, in order to revert it and re-run once all implicit deps are satisfied). The approach Andy indicates earlier in his article - using strace to trap this info - is a great first pass; strace slows a build down 5x or more, though, and isn't available on Windows. CloudBees Accelerator is built for speed, so we're more elegant about how we do the data collection.

2. CloudBees Accelerator assumes as a commercial requirement that developers are tightly bound to their tools, whether by choice, edict, or (mis)fortune. A commercial product which requires code change will fail , so it's important to us to support specific tools. So we don't actually require everyone to use our tool - we emulate *your* tool in most cases - and compatibility with the original infrastructure is maintained.
   It's an interesting idea, writing a tool-indepdenent build language, or writing a tool which can transition from one implementation (ANT) to another (Make) - and there are commercial companies which have such products available. CloudBees has focused on what we see as the fundamental hard problem - seamlessly discovering dependencies - and uses it to get to what I think Andy's end goal is:
   such a meta-build might make recursive make correct.
   Which is exactly where CloudBees Accelerator ends up: we fix the problems identified by Miller in Recursive Make Considered Harmful by allowing all implicit dependencies to be respected (and some other neat stuff too, like merging sub-makes transparently.)