Roadmap to version 3

[andreynering]

This is a general issue to track proposals with small or big breaking changes, that could be released in the next major version.

This list is a work in progress and can change at any time.

Development is happening on the v3 branch.

Cleanup

• Change module name from github.com/go-task/task/v2 to v3
• Remove all code related support of version 1 (Remove all code related support of version 1 #237)
• Remove automatic inclusion of Taskvars.yml and Taskfile_{{OS}}.yml. This was done before we had a generic inclusion of Taskfiles implemented. We should add options to
  • Don't include the file if it doesn't exist
  • Allow interpolation on the file name to make it possible to include a OS specific file
• Finally delete the vendor directory? (Remove the vendor/ directory on v3 once the lasts 2 Go versions have modules enabled by default #214)

Breaking changes

• Added global method: option. Default changed from timestamp to checksum (Add global "method:" option to allow setting a default for all tasks. Change default from "timestamp" to "checksum" #246)
• Refactor vars and templates to reduce the number of bugs and undesired behavior (Refactor vars and templating to fix some bugs for v3 #218)
• Make CLI arguments override everything (Add options to make all variables available as environment variables #181) (EDIT: What about v3: expansion doesn't work as expected #336?)
• Make up-to-date resolution work when only "generates" is given (Make up-to-date resolution work when only "generates" is given #187)
• Don't run the same dependency (with the same args) more than once by default (Multiple tasks with the same dependency should run dependency once! #53, Force tasks that generate the same files to not clobber each other #193)
• Allow forcing specific tasks (How to force task but not its dependencies #189)

New stuff

• Colored output messages (Use colors for some output messages #207)
• Expose .TIMESTAMP and .CHECKSUM to status: (Expose timestamp and checksum to status #216)
• Allow shorter syntax for tasks with default configuration (Allow shorter syntax for tasks with default configuration #194)
• Allow use as a library (Consider use as an external library #121)

Documentation

• Mention changes related to method: (shouldn't also check the main guide)
• Update "Taskfile versions" article

---

There are more issues and feature request that could be added to this release.

[andreynering]

/cc @smyrman @sheerun @jamestenglish @jaxxstorm @Evertras

---

As you may have noticed, I've being very slow on implementing new stuff for this project. So don't expect this release to happen soon. This is mostly to grab feedback.

[Evertras]

Just a big +1 to the 'run a dependency with the same input only once' by default. That's my only pain point right now, so I'm a little biased. :)

[smyrman]

It's a great list, agree to all points; take your time:-)

[edrd-f]

It would be great if there was a way of making variables part of the environment, as suggested on #181.

[stephenprater]

Some things that are on our wish list.

• Allow sh cmds to export their results to the Templater Env or to a environment variable available in subsequent commands.

cmds:
  - sh: uname
    export: UNAME
  - sh: echo $UNAME

• Preconditions (we made a PR for this one)
• Expose {{.CHECKSUM}} and {{.TIMESTAMP}} in status to check remote artifacts (I have this done as well and will make a PR after pre-conditions.)
• Specify a "format string" for the output prefix (to control width / color of prefix output)
• Allocate a TTY to cmds on demand. (For interactive commands)
• Pluggable interpreters besides "sh" like you could do

cmds:
  - python: |
import os
print(os.getcwd())

We'll probably add some or all of these things as it becomes necessary to support our automation tasks.

[andreynering]

Hi @stephenprater,

Expose {{.CHECKSUM}} and {{.TIMESTAMP}} in status to check remote artifacts (I have this done as well and will make a PR after pre-conditions.)

🙂

Allocate a TTY to cmds on demand. (For interactive commands)

We recently had some issues on running interactive commends that should have been fixed after #200. Are you still experiencing issues? If so, let me know.

Pluggable interpreters besides "sh" like you could do

cmds:
  - python: |
import os
print(os.getcwd())

This is an interesting idea. On the other hand, there are reasons why it isn't available today:

1. You can already call it by having sh: python your_script.py
2. We compile a Shell interpreter written in Go with Task to run commands, so results are consistent between platforms (including Windows which doesn't have Bash always available, or it sometimes behave differently). Allowing Python/Ruby/etc scripts would make it more dependent on the environment (people have different versions of these installed, different libs installed, etc).

[stephenprater]

You can already call it by having sh: python your_script.py

This isn't quite the same thing because you need to interpolate your task variables into the command line, plus you have a script that also does project automation so not all of your automation lives in one spot anymore.

This is definitely our lowest priority item - and it's mostly because things like:

service_status=$(nomad deployment status $deployment | grep "^services")
service_complete=$(echo -e "$service_status" | awk '{if($4==$7){print "1"}else{print "0"}}')
if [ $service_complete == "1" ]; then
    echo $green
fi
echo -e "$service_status $(tput sgr0)"

are harder to read (and write!) than the python equivalent.

while True:
    deployment = nomad.client.deployment("adf3ec3")
    if (
        deployment["TaskGroup"]["services"]["DesiredCanaries"]
        == deployment["TaskGroup"]["services"]["HealthyCanaries"]
    ):
        break

print("Services Deployed!")

We tend to use Task as an automation tool and for scripting complex CI/CD workflows more than as a pure build dependency tool, so the scenario where people have different versions of an interpreter installed isn't really a concern for us, as it's handled by the language tooling for Python / Ruby.

One of the things that's left out of my one-sentence description there is that the Taskfile itself would need to specify the interpreter.

So at the top you'd do something like:

interpreters:
    python: /usr/bin/env python
    ruby: bundle exec ruby

Anyway - that's definitely pie-in-the-sky kind of stuff - we haven't even looked seriously at this as a feature yet - it's just that every time our devs have to write sh and then remember it's not bash they complain about it. 😉

[andreynering]

One of the things that's left out of my one-sentence description there is that the Taskfile itself would need to specify the interpreter.

So at the top you'd do something like:

interpreters:
    python: /usr/bin/env python
    ruby: bundle exec ruby

This makes me like it better, since it's generic and customizable. 🙂

Implementation-wise, it can be challenging to make YAML accept python: or ruby: attrs if we don't know beforehand which interpreters the user will implement. Not that we won't find a solution, though.

[stephenprater]

Allocate a TTY to cmds on demand. (For interactive commands)

This is definitely not fixed for me on today's master. Filed an issue - #217.

[andreynering]

Things have been progressing slowly.

From now on, I'll ask PR authors to target non-small new features to the v3 branch (bug fixes should target master).

I'll make preview releases once in a while for people to use the new features and report bugs. This is the first one: v3.0.0-preview1.

[zx8]

Any chance remote includes (#98) can be considered for this?

[andreynering]

Any chance remote includes (#98) can be considered for this?

@zx8 Sorry, but that's on hold and it won't be on v3.

I'm not happy with the complixity it'd brings as it's proposed (regarding caching, versioning, etc).

[brandonkal]

Rather than a syntax like this:

interpreters:
    python: /usr/bin/env python
    ruby: bundle exec ruby

Why not just use shebangs at the top of the tasks like the just tool does?

[andreynering]

@brandonkal That sounds like something worth considering. Thanks for the idea!

[andreynering]

Hey everybody,

We just reached a great milestone for v3 which is refactoring variables, which will now be evaluated in the sequence of declaration. This means that Taskfiles like this will just work ™️ without having to set expansions or similar workarounds:

task/testdata/vars/v3/Taskfile.yml

Lines 1 to 46 in 4b02772

	version: '3'
	vars:
	VAR_A: A
	VAR_B: '{{.VAR_A}}B'
	VAR_C: '{{.VAR_B}}C'
	VAR_1: {sh: echo 1}
	VAR_2: {sh: 'echo "{{.VAR_1}}2"'}
	VAR_3: {sh: 'echo "{{.VAR_2}}3"'}
	tasks:
	default:
	- task: missing-var
	- task: var-order
	- task: dependent-sh
	- task: with-call
	missing-var: echo '{{.NON_EXISTING_VAR}}' > missing-var.txt
	var-order:
	vars:
	VAR_D: '{{.VAR_C}}D'
	VAR_E: '{{.VAR_D}}E'
	VAR_F: '{{.VAR_E}}F'
	cmds:
	- echo '{{.VAR_F}}' > var-order.txt
	dependent-sh:
	vars:
	VAR_4: {sh: 'echo "{{.VAR_3}}4"'}
	VAR_5: {sh: 'echo "{{.VAR_4}}5"'}
	VAR_6: {sh: 'echo "{{.VAR_5}}6"'}
	cmds:
	- echo '{{.VAR_6}}' > dependent-sh.txt
	with-call:
	- task: called-task
	vars:
	ABC123: '{{.VAR_C}}{{.VAR_3}}'
	called-task:
	vars:
	MESSAGE: Hi, {{.ABC123}}!
	cmds:
	- echo "{{.MESSAGE}}" > with-call.txt

This is not yet released, not even to the preview channel, so you need to manually build the v3 for now, but I'd really like feedback on whether it's working as expected and it's bug free. (Remember that to test this you need to change your Taskfile version to 3).

Completing this means we're now close to finally shipping v3. Since this issue is opened for more than a year already, I'll probably postpone some of the bulled points above to v3.1+, specially those who aren't breaking changes, so we can ship this sooner.

Opinions are welcome!

[danquah]

Would definitely prefer a V3 with the current set of (great) features, then subsequent fixes of whatever bugs has snug in.

And maybe then go so far as to ping all the issues that are then scheduled for a V3.1 and evaluate whether they are still relevant? :)

[andreynering]

v3.0.0 is finally released! 🎉

As I said, it was not possible to address everything at once, but this release is already big enough, and I plan to add features in smaller releases from now on...

If the feature you asked for doesn't have a an open issue, please open so it isn't forgotten.

Thanks for everyone that helped in one way or another!

[marco-m]

Just wanted to 🎉 ! Thanks! :-)

[danquah]

Yes, a thousand times 🎉 Thank for all the hard work!

[Nadock]

One of the things that's left out of my one-sentence description there is that the Taskfile itself would need to specify the interpreter.
So at the top you'd do something like:

interpreters:
    python: /usr/bin/env python
    ruby: bundle exec ruby

This makes me like it better, since it's generic and customizable. 🙂

Implementation-wise, it can be challenging to make YAML accept python: or ruby: attrs if we don't know beforehand which interpreters the user will implement. Not that we won't find a solution, though.

Did this interpreters feature make it into v3? I can't see any details in the documentation aside from #243 and I can't see any other reference to it in the repo. It's certainly a feature I'd like to see added to Task.

Thanks for all your hard work either way 🎉

[andreynering]

Hi @Nadock,

Sorry, that just wasn't implemented. Maybe it's worth creating an separated issue about it.

[Nadock]

Hi @Nadock,

Sorry, that just wasn't implemented. Maybe it's worth creating an separated issue about it.

All good, I suspected as much. I have opened #448 to track this 👍