Repository navigation
[Nexthop][misc] Move the hw oper delta instead of copying it - #1906
Draft
deepakgudiseva-nexthop wants to merge 1 commit into
Draft
deepakgudiseva-nexthop wants to merge 1 commit into
deepakgudiseva-nexthop wants to merge 1 commit into
Conversation
At high ECMP width a route replace or withdraw makes the oper delta sw_agent pushes to the HwSwitch large (a replaced 128-way route is ~110-220 KB), and the delta is copied several times on its way there: into the per-matcher cache, when filtering it per switch, and when handing it to the HwSwitch. Peak memory during big route updates scales with those copies. Build a sole consumer's delta without caching it and move it to the HwSwitch, filter per-switch deltas in place, and move rather than copy where the source is reset afterwards.
2 tasks done
Author
|
Setup: 128-way ECMP, 22k routes (~2.8M paths), with fboss_sw_agent memory, GiB:
Together the two PRs cut sw_agent's peak during route updates by about 3.6 GiB (~60%). Steady state stays the same. |
Author
|
/autorebase |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-submission checklist
pip install -r requirements-dev.txt && pre-commit installpre-commit runSummary
At high ECMP width a route replace or withdraw makes the oper delta SwSwitch pushes to the HwSwitch large (a replaced 128-way route is ~110-220 KB), and that delta is copied several times on the way: into the per-matcher cache, when it is filtered per switch, and when it is handed to the HwSwitch. sw_agent's peak memory during large route updates scales with those copies; with a few thousand 128-way routes it can exceed the agent's memory limit.
This builds a sole consumer's delta without caching it and moves it to the HwSwitch, filters per-switch deltas in place, and moves rather than copies where the source is reset afterwards. No behaviour change.
Test Plan
pre-commit run(clang-format 21.1.2) clean on the modified files.