Repository navigation
Add option to 'flatten' included taskfiles #273
Description
It would be great to be able to set an option to avoid having to call the tasks using the task includename:task syntax when including a taskfile.
So this example assumes you have files like:
./Taskfile.yml
./common/Taskfile.yml
The use case for this is when creating a main Taskfile for a repository and importing some common tasks, lets say "list", "clean" and "version", right now I need to then call task common:clean etc to run them.
Instead I would like to be able to do something like this in my main taskfile:
includes:
common: ./common
flatten:true
override: false
Where the 'flatten' property would define that the tasks from 'common' should be available as if they were present in the main taskfile, so "task clean" would actually call "task common:clean". This would probably have to throw a conflict error if the "clean" task exists in both the main taskfile and the 'common' one, meaning that anything from an included taskfile that would be flattened out needs to be unique. Possibly an "override" property could be added instead, to let the included task take precedence over any task with the same name in the main taskfile.
This kind of functionality would allow you to add taskfiles as a submodule to a repository, create a main taskfile for that repo where you include the submodule taskfiles, and still be able to run common tasks with a short syntax without having to point out the sub-taskfile. Or in other words include other taskfiles as transparent "libraries".
Activity
andreynering commentedon Dec 16, 2019
Hi @tommyalatalo, thanks for opening this issue!
I'm still not convinced to add this feature. I think this brings too little benefit to be worth it: mostly allowing shorter task names (i.e.
buildinstead ofcommon:build).I like how namespaced tasks allow you to immediately know where a task if defined (you just have to follow the import).
Am I missing something on other ways this would be useful?
tommyalatalo commentedon Dec 16, 2019
I think there is a valid use case here.
If I want to do multiple includes from a main taskfile, and those includes can vary, lets say like this:
Then the main taskfile could benefit from the "flatten" and "override" function in that it would allow you to set default build and tests tasks in the main file, but you can include another taskfile which would override them as desired, depending on the content of the repository you're working in. That is to say if I work in a go repo I just import the "go" taskfile and set it to flatten and override, then it would automatically take over the targets in the main taskfile.
Another use for this is to clean up the
--listoutput.When including multiple taskfiles from the main you could end up with something like this:
This would result in duplicate output of commands in
--listlike:So you see in the above case I would be able to override all the non-"go" targets to keep the list output clean, since this list can get very long if including multiple other taskfiles. This duplicate list output can be confusing to someone who doesn't know as much about the importing of other taskfiles, and is simply cluttering. The main point of this would be to keep the standard targets "build", "test" and "clean" in this case always the same, so that you don't have to manually edit them to point to the "go" targets or whatever else you're importing.
I think this relates somewhat to #268 since if you do an override of a task, it would still be beneficial to see whether that task points to another task or if its still the default, so adding an "Alias" column to the
--listoutput would provide that information in a good way.ghostsquad commentedon Feb 20, 2022
I kind of disagree that flattening should exist, as there's a couple things that I think might be confusing, how does a user reasonably figure out what tasks refer to what, especially since you can reference
super.<task>using:tasknamein an included file:I don't think it would be very clear to an end user, if flatting was a thing, to figure out what's actually going on. The more I get into Golang development, the more I'm realizing that readability counts quite a bit. This is a case that would likely significantly hurt readability.
Gowiem commentedon Jul 19, 2023
@pd93 thanks for sharing the dup and closing out my ticket #1271. I was actually just reading through this issue right after filing mine. Should've searched better before writing the whole thing up.
Anyway, I wanted to share that I think there is value in this and why. Particularly in the case that a lot of orgs are using Task (#770), which is where there is a centralized repo of tasks that get pulled into other repos as shared tasks and there aren't really any tasks in the consuming repos. In that use-case, we're creating a namespace hierarchy that is unnecessary and creates tedium. I get the argument against that everything should be explicit, but there is value in terseness as well.
marverix commentedon Jul 26, 2023
Hi, I just was looking for Gradle alternative in my organization, which currently is using Gradle as a task manager. Currently, the lack of the "flattening" possibility in Task is a blocker for me to use it as a Gradle alternative. The use case is very similar to my previous speakers. As a CI/CD tech lead, I want to provide one file, which would be a Convention that we use in the company. For example, I can declare a Taskfile:
PipelineConvention.yamlAnd with that, I could do a git submodule in any repo of the organization and create Taskfile:
This would give me the possibility to create a Jenkins Pipeline that will always just run for example
task stage-init. In my opinion, this is a must-have feature to allow bigger organizations standardize tasks across all repos.As I said, in Gradle it's easily achievable (and I'm thinking about Gradle mainly as a task manager, not the builder - because I don't care about Java, and it wasn't mine decision to use Gradle), because you can overwrite any task - even subtask.
So at least if you don't want to implement
flatten, then maybe at least allow to overwrite included (namespaced) task from the taskfile that includes. Eg.Br,
Marek
vmaerten commentedon Oct 16, 2023
We have the same use case as @marverix in my company
Chi-teck commentedon Apr 3, 2024
My use case:
There is a number of tasks included to the Taskfile that is stored in Git repository.
I need to be able to include additional tasks from local git ignored Task file.
So the resulting list of tasks would look like follows
For consistency it's highly desired that all tasks share same namespace. No matter where they come from.
Chi-teck commentedon Apr 3, 2024
I agree that that 'flatten' behavior should be optional and disabled by default.
elocke commentedon Apr 4, 2024
This feature would be massive benefit to our team!
cenk1cenk2 commentedon May 5, 2024
With the inclusion of the remote task files, I would argue that this would be a great addition instead of specifying an URL for each task command and just including some templates at a root level for similar repositories, even though it increases the abstraction a bit, and the user has to pay attention to their task names.
tomharrisonjr commentedon May 21, 2024
My use-case for this is simple: I have common commands across projects that insulate devs from dealing with docker commands, so
task uprunsdocker compose up -dthendocker compose logs <container>, orcleanthat runsdocker system prune -afand so on.They're all the same across projects, so a natural case for includes, so I avoid repetition and centralize logic.
Maybe this is a different use-case than namespaces or aliases as they exist today.
marverix commentedon May 22, 2024
Dear authors and/or maintainers of the repo, can we know what is the cause of the reluctance towards this feature? I have created a PR as a working PoC, but no one even looked at it. Now there is a ton of conflicts.
#1281
andreynering commentedon May 22, 2024
Hi @marverix,
I'm sorry that your PR got unreviewed. The fact is that more issues and PRs are opened than we're able to keep up. We're volunteers and have limited time to dedicate to the project.
I'm not against the idea of flattened includes. The codebase got quite a bit of refactor in the past months, which explains the conflicts in your PR. It may be necessary to start from the ground up.
pd93 commentedon May 22, 2024
Just wanted to add that besides the time constraints. One of the reasons I've personally been holding off on this is because of the large refactors being made. It would have added complexity to the changes that were being made if we'd started introducing large changes like this at the same time.
It's worth noting that there are still some big changes coming to AST merging in the future (which is where this feature would currently go). As always, we can't make any promises about when we will get time to look at this, but it has not gone unnoticed that this is one of the most upvoted feature requests.
marverix commentedon May 23, 2024
Thanks for the reply and you hard work!
trulede commentedon May 31, 2024
In some cases, this kind of solution may be helpful. Its based on one of the techniques we use ... you have to figure out the
shbit for yourself, but you get the idea - it is actually possible.The other interesting thing might be the
aliasoption applied to an imported task. How exactly that is handled. Could be a neater solution, upon import (i.e. the includes statement) to alias the imported commands into the importer namespace.But its very difficult when you have more mature/complex taskfiles. Which is why I suggest to try the more explicit approach above.
Also, one can consider a bootstrapping approach, where a first Taskfile "compiles" a second Taskfile and then dispatches task commands to that second Taskfile (via the task command). What "compiles" means is up to you, in our case, we clone a git repo that has a common Taskfile collection for our tools, and then programmatically build another Taskfile with our higher level task/commands.
So the developer only ever has to type task X and X runs .... task takes care of the work in setting that up.
And before you say, too hard ... no its not! Very easy to write some go code to emit a Taskfile using either the AST or even simple templating.
fetinin commentedon Jun 21, 2024
I’ve made the following work-around to send all commands to namespace of included taskfile, until this issue is fixed. Hope it will help someone.
vmaerten commentedon Aug 26, 2024
It has been merged, it'll be available in the next release 🚀