Aggregator tracker

[ValerianRey]

This issue tracks some candidate aggregators that could maybe be added to TorchJD. Feel free to suggest other methods, or to add more information about the methods in the table. If you're interested in implementing one, please tell it so that we can investigate the method in a bit more details (in particular, we need to verify that it can be implemented as an aggregator), and so that we don't work concurrently on the same thing.

Status	Name	Ref	Stateful	Existing implementations	Special Remarks
🟡 In progress (#676)	MoCo (Multi-Objective Gradient Correction)	Mitigating gradient bias in multi-objective learning (ICML 2023 oral, 95 citations)	Yes (Jacobian EMA)	LibMTL
🔵 To investigate	DB-MTL (Dual Balancing Multi-Task Learning)	Dual-balancing for multi-task learning (Neural Networks n°195, 9 citations)	Yes (Jacobian EMA)	LibMTL (official)	The paper combines 3 ideas: applying a log to the losses (not something we need TorchJD for), normalizing the gradients (this should be a Normalizer rather than a part of the aggregator) and using the EMA of the gradients. This last idea is also common with some other methods, so maybe we could implement it as a wrapper or something. So in the end I don't think there is anything to implement specifically for DB-MTL.
🟣 To Do	M-ConFIG	ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks (ICLR 2025 spotlight, 57 citations)	Yes	official	Not actually implementable as an aggregator or as a weighting (tum-pbs/ConFIG#19), but we could add a usage example in ConFIG's docstring saying how to do M-ConFIG, even if that involves creating an entire class in this usage example.
🔵 To investigate	Excess-MTL	Robust Multi-Task Learning with Excess Risks (ICML 2024, 35 citations)	Yes	official, LibMTL (both are identical)	https://claude.ai/share/7a51ce61-b926-43d0-af12-d6c0b1c59ad5