You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Measuring energy on a single machine running multiple concurrent GPU processes (Federated Learning simulation) #936
I have been using codecarbon for some time, and I was wondering what is the recommended way to use CodeCarbon to measure energy/emissions on a single machine where multiple GPU processes run in parallel?
Our use case is a federated learning simulation that launches many federated “clients” as separate processes on one GPU.
Right now, we run one EmissionsTracker per process (each simulated client), and this seems to be working reasonably. However, if we track per process, how does CodeCarbon derive the GPU power consumption? Is per-PID attribution supported via NVML, and could that double-count energy cost when multiple processes share a GPU? How should we think about CPU/RAM attribution when multiple processes are active? Are there docs/examples for multi-process, multi-GPU scenarios we should follow?
Thanks in advance! Any pointers to best practices or examples would be greatly appreciated.
Activity
benoit-cty commentedon Sep 24, 2025
Hello,
Sorry, CodeCarbon can't do per-process with NVML yet. So you have to run CodeCarbon on the whole machine.
If you have multiple GPU, CodeCarbon will monitor all of them.
You could specify which GPU you want to monitor by giving the GPU id, so you could do per-GPU monitoring.
For CPU/RAM attribution, the process mode is supposed to work.