Repository navigation
improve slow operations - #351
Merged
Merged
Conversation
A corrupted MRT stream re-synchronises header by header, which emitted one error-level log line per 12 bytes of damaged input. The default, raw, update and route iterators now log the first ParseError of a parser at error level and later ones at debug level, mirroring the once-only Zebra compatibility warning. Assisted-by: Claude Code
Every elem of a RIS Live UPDATE carries its own copy of the message's AS path and communities, so collecting a message that announces many prefixes needs memory proportional to prefixes × path length: a 178 KB JSON message produced ~512 MB of elems. parse_ris_live_message_iter validates the message up front and then yields elems one at a time, making the per-prefix copies lazily; the last announcement takes the shared attributes without a copy. parse_ris_live_message is unchanged and now builds on it. Assisted-by: Claude Code
span_record_warnings walked the whole dissection tree once per warning to find the Nth occurrence of an attribute node. A record carries as many warnings as it has attributes, so a 64 KB BGP UPDATE with ~22K duplicate attributes took 21.7 s to diagnose with dissection. The tree is now indexed by field name in a single pre-order walk, and spans are looked up from that index. Spans are unchanged; the same record now takes 42 ms. Assisted-by: Claude Code
- RIS Live JSON elems now carry origin_asns derived from the AS path, so origin_asn filters match them as they do raw and MRT elems. The captured-frame comparison now includes origin_asns. - The route iterator sends only ParseError through the once-per-parser limiter; other body errors are logged at error level as before and can no longer use up the one error-level parse error line. - With core dumps enabled the first parse error no longer promises further debug-level errors, since iteration stops there. - span_record_warnings skips building the span index for records without warnings. - parse_ris_live_message_iter copies the message only on error paths, pre-sizes the announcement list and collects withdrawals directly. - Replace the circular iter-vs-vec test with hard-coded expected elems and add a test for once-per-parser parse error logging. - CHANGELOG: add the origin_asns fix and a Contributors section. Assisted-by: Claude Code
The raw, update and route iterators each carried a copy of the framing-error match in handle_record_parse_error, so every change to the error policy had to be made in four places. They now call the shared handler. The copies had the same arms and outcomes, so behaviour is unchanged. Assisted-by: Claude Code
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #351 +/- ##
==========================================
+ Coverage 92.06% 92.12% +0.05%
==========================================
Files 96 96
Lines 23401 23534 +133
==========================================
+ Hits 21545 21680 +135
+ Misses 1856 1854 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The optimizations preserve iterator behavior and are covered by focused regression tests.
0 open findings
What changed in this PR
Improves resource usage for high-volume RIS Live parsing, repeated parser errors, and diagnostic warning correlation.
Changes:
- Adds streaming RIS Live element iteration and populates
origin_asns. - Limits repeated parse-error logging per parser.
- Indexes dissection spans once for linear-time warning correlation.
| File | Description |
|---|---|
CHANGELOG.md |
Documents performance improvements and fixes. |
src/parser/mod.rs |
Exports the iterator and tracks parse-error logging. |
src/parser/rislive/mod.rs |
Implements streaming RIS Live parsing. |
src/parser/iters/mod.rs |
Applies limited parse-error logging. |
src/parser/iters/raw.rs |
Reuses shared error handling. |
src/parser/iters/route.rs |
Reuses shared and limited error handling. |
src/parser/iters/update.rs |
Reuses shared error handling. |
src/parser/iters/diagnostic.rs |
Adds indexed warning-span lookup. |
tests/parse_error_logging.rs |
Tests per-parser log suppression. |
tests/rislive_frames.rs |
Extends JSON/raw parity to origin ASNs. |
🧠 Review effort: Balanced
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use less resources for some inputs that consume a lot of resources