Declarative format

[domenkozar]

I'd essentially like to turn https://github.com/mayflower/hydra-jobs/blob/master/spec.json into something more .travis.yml alike.

inputs:
  nixpkgs:
    type: git
    value: git://github.com/mayflower/nixpkgs.git

notifications:
  email:
    - [email protected]

Input types:

• git
• mercurial
• boolean
• nix
• string
• previous build
• previous evaluation

Linting:

• error out if section appears two times

[expipiplus1]

Should the src input always be the current repo?

[domenkozar]

We could get rid of src input in the first place since imports can be relative and we'll just evaluate some conventional file like release.nix

[expipiplus1]

That's a good point. I suppose it's only there in the existing declarative jobsets for ease of integration

[domenkozar]

Indeed :)

[expipiplus1]

unless it could be used to do things such as building the branch with master merged into it or something. I'm not sure that nix is the right environment for that though.

[domenkozar]

@expipiplus1 since yaml uses attoparsec https://github.com/snoyberg/yaml/blob/master/yaml.cabal#L58, how much work would we need to make yaml provide friendly error message? Rewrite backend to megaparsec? I'd like to understand the scope here.

[expipiplus1]

Are the error messages so bad at the moment? It's certainly outside Hercules's scope. There are other yaml parsers on hackage including the reference parser.

…

On Sat, Dec 3, 2016, 11:41 PM Domen Kožar ***@***.***> wrote: @expipiplus1 <https://github.com/expipiplus1> since yaml uses attoparsec https://github.com/snoyberg/yaml/blob/master/yaml.cabal#L58, how much work would we need to make yaml provide friendly error message? Rewrite backend to megaparsec? I'd like to understand the scope here. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA0U3B_q5MQuVVbQboa-sr7FOdMsPTWTks5rEf4ugaJpZM4K9FnV> .

[domenkozar]

@expipiplus1 we can still swap out the parser in future, but I'd like to be sure we don't use a parser that won't produce useful messages to users if they make mistakes in declarative config.

[griff]

I really appreciate what you guys are trying to do with this project.

I am curious as to what you mean when you say you want to make the declarative format more Travis CI like because they and other build servers like them have a very strong correlation where 1 branch = 1 jobset while Hydras declarative format takes the form of a special jobsets that creates the other jobsets. So which model are you planning for Hercules?

Personally i like the Travis model and the Hydra model feels bolted on as an afterthought though it is a lot more flexible and more complex.

[domenkozar]

Thanks!

declarative format takes the form of a special jobsets that creates the other jobsets

I'm not sure what you mean by that exactly. Can you provide an example?

[expipiplus1]

@domenkozar he's talking about the declarative jobset functionality that shlevy added to hydra. the .jobsets jobset.

[domenkozar]

@griff Hydra declarative jobsets is a feature that added declarative format with minimal code change. But it wasn't really properly integrated, which we plan to do travis-ci like.

We might need to consider that a project can have multiple jobsets though.

[griff]

@domenkozar I believe you can have different content in .travis.yml in different branches on Travis CI. Doing the same with this project might solve the multiple jobsets per project though you would then be limited to 1 jobset pr. branch.

[spacefrogg]

Although I have no say in this, I want to contribute my thoughts to this discussion. Two distinct points I have to make:

JSON for jobsets

I use hydra declarative jobset feature only recently. The first thing that strikes me is, that you only need JSON ever once per project, namely the one file you use to instantiate the project itself and that points hydra to the nix expression that generates the .jobsets jobset. After that you can handle everything from that nix expression. I wrote a small library (20+ lines) that handles argument testing and json creation, which is trivial as nix has toJSON as a builtin. This feature does not feel like an afterthought to me. BTW an extra src input is not needed for declarative jobsets as it comes as declInput into the nix expression already.

Definition of project

As projects in hydra are the basis for defining releases, it is natural to have multiple jobsets per project of which any one might evolve into a release or multiple releases. In that sense, projects should support "multiple branches". The flexibility is a great gain and comes at virtually no cost in the trivial cases. If another path is pursued in hercules, declarative projects might have other needs, though.