Smarter dependency-aware evaluator than Hydra

[copumpkin]

As far as I've been able to tell, Hydra does the following to figure out what to build:

1. Evaluate your jobsets
2. Pick a job, anywhere
3. Build it
4. Farm out its subtasks across remote builders

This leads to sometimes me starting a branch in it and seeing haskellPackages.foo being the first job being built. If I check the Steps tab for that job, I see that it's downloading the bootstrap tools, bootstrapping the stdenv, and so on. Given that the bootstrap steps aren't explicitly named jobs, it can be hard to track them down.

More intuitive to me would be:

1. Evaluate jobsets
2. Understand the dependency graph
3. [Optional] Let me explore it somehow
4. Start evaluating based on the graph
5. [Optional] Prioritize builds based on how many downstream builds they enable

Or some variation of that.

[Ericson2314]

I think I agree, but I also think Nix itself is also to blame here.

[expipiplus1]

Hopefully the work I've started on a haskell interface to nix-build can go part of the way to solving this.

[copumpkin]

@Ericson2314 how do you mean? We can easily graph out the build dependencies, so it's not as if the information isn't there

[expipiplus1]

I'm planning to do it by setting up a build hook with NIX_BUILD_HOOK which captures the requirements and forwards them to a smarter builder.

The story is complicated somewhat by IFD (see NixOS/nix/#666)