Repository navigation
Add metrics for worker communications #6892
Description
To better understand worker communications, their performance and required resources, we should track more metrics. Specifically, it would be interesting to track the number of open incoming/outgoing connections and comm_nbytes in Worker.get_metrics. This could help debug issues such as #6208 or coiled/benchmarks#229 (comment) and help take decisions on possible solutions for those (#6208 (comment))
Reacted by Gabe Joseph
Activity
fjetter commentedon Aug 17, 2022
If we go down this road I believe it would be interesting to add
comm_nbytes(or a derivative of some sort) to the dashboard. I suspect that unmanaged memory is often actual communication overhead and it would be nice if we could separate this visually.