Skip to content

[Nexthop][misc] Move the hw oper delta instead of copying it - #1906

Draft
deepakgudiseva-nexthop wants to merge 1 commit into
meta:mainfrom
nexthop-ai:agent-move-hw-oper-delta
Draft

deepakgudiseva-nexthop wants to merge 1 commit into
meta:mainfrom
nexthop-ai:agent-move-hw-oper-delta

Conversation

@deepakgudiseva-nexthop

Copy link
Copy Markdown

Pre-submission checklist

  • I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running pip install -r requirements-dev.txt && pre-commit install
  • pre-commit run

Summary

At high ECMP width a route replace or withdraw makes the oper delta SwSwitch pushes to the HwSwitch large (a replaced 128-way route is ~110-220 KB), and that delta is copied several times on the way: into the per-matcher cache, when it is filtered per switch, and when it is handed to the HwSwitch. sw_agent's peak memory during large route updates scales with those copies; with a few thousand 128-way routes it can exceed the agent's memory limit.

This builds a sole consumer's delta without caching it and moves it to the HwSwitch, filters per-switch deltas in place, and moves rather than copies where the source is reset afterwards. No behaviour change.

Test Plan

  • Agent unit tests pass with this change in our fork's CI; this commit is the same change applied to main.
  • pre-commit run (clang-format 21.1.2) clean on the modified files.

At high ECMP width a route replace or withdraw makes the oper delta sw_agent
pushes to the HwSwitch large (a replaced 128-way route is ~110-220 KB), and the
delta is copied several times on its way there: into the per-matcher cache,
when filtering it per switch, and when handing it to the HwSwitch. Peak memory
during big route updates scales with those copies.

Build a sole consumer's delta without caching it and move it to the HwSwitch,
filter per-switch deltas in place, and move rather than copy where the source
is reset afterwards.
@meta-cla meta-cla Bot added the CLA Signed label Nov 4, 2026
@deepakgudiseva-nexthop deepakgudiseva-nexthop changed the title [Nexthop] Move the hw oper delta instead of copying it [Nexthop][misc] Move the hw oper delta instead of copying it Nov 4, 2026
@github-actions github-actions Bot added nexthop project:misc FBOSS project: misc labels Nov 4, 2026
@deepakgudiseva-nexthop

Copy link
Copy Markdown
Author

Memory impact (#1906 + #1907)

Setup: 128-way ECMP, 22k routes (~2.8M paths), with enable_nexthop_id_manager, resolve_nexthops_from_id and drop_inline_nexthops_with_id enabled.

fboss_sw_agent memory, GiB:

drop inline only + #1906 + #1907
Peak, route learn 5.8 2.2
Peak, route withdraw 5.6 2.1
Steady state 0.48 0.46

Together the two PRs cut sw_agent's peak during route updates by about 3.6 GiB (~60%). Steady state stays the same.

@deepakgudiseva-nexthop

Copy link
Copy Markdown
Author

/autorebase

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant