[Vamana] Add get_memory_usage() to report allocated bytes#345
Open
yuejiaointel wants to merge 8 commits into
Open
[Vamana] Add get_memory_usage() to report allocated bytes#345yuejiaointel wants to merge 8 commits into
yuejiaointel wants to merge 8 commits into
Conversation
Adds get_memory_usage() returning the total bytes allocated by a Vamana index (graph storage + vector data + metadata), so an integrator can report and bound SVS memory consumption. Both the static VamanaIndex and the dynamic MutableVamanaIndex are covered, and the method is plumbed through the orchestrator layers so it is callable on svs::Vamana and svs::DynamicVamana. Accounting is capacity-based (the bytes the containers have reserved, not just live elements) so that block over-allocation is reflected: - graph_bytes / data_bytes: capacity() * element_size() of the graph and dataset backing storage (falls back to size() for datasets that do not expose capacity()). - metadata_bytes (dynamic only): slot-status vector, entry-point list, and an estimate of the external/internal ID translation maps. A VamanaMemoryUsage struct and get_memory_breakdown() expose the per- component split; get_memory_usage() returns its total(). Adds unit tests for the static and dynamic indices at both the core-index and orchestrator levels.
Plumbs the index get_memory_usage() through the runtime binding API so it is callable on the runtime VamanaIndex / DynamicVamanaIndex structs (and the inherited LeanVec variants), mirroring the existing blocksize_bytes() plumbing. - Declares the virtual on the base runtime VamanaIndex so both the static and dynamic structs expose it. - Concrete overrides in VamanaIndexImpl and DynamicVamanaIndexImpl forward to the wrapped orchestrator's get_memory_usage(); returns a plain size_t (not Status-wrapped), matching blocksize_bytes(). Adds runtime unit tests for the static and dynamic indices.
Contributor
Author
|
current failing for runtime test, it should be gone after this change goes into nightly |
rfsaliev
reviewed
Jul 10, 2026
|
|
||
| size_t blocksize_bytes() const noexcept override { return impl_->blocksize_bytes(); } | ||
|
|
||
| size_t get_memory_usage() const noexcept override { return impl_->get_memory_usage(); } |
Member
There was a problem hiding this comment.
the statement impl_->get_memory_usage() should be wrapped to runtime_error_wrapper.
P.S. existing blocksize_bytes() has the same issue.
Contributor
Author
There was a problem hiding this comment.
get_memory_usage() returns size_t, not Status, so it can't be wrapped (same as blocksize_bytes()). The wrapped entry point is now get_memory_breakdown(); get_memory_usage() is just its total.
Applies review feedback on the memory-usage API: - Move the dataset_allocated_bytes helper from the Vamana index header to svs/core/data.h (svs::data::detail), since it is a generic dataset utility rather than index-specific. - Make get_memory_breakdown() the single source of truth at the core index and orchestrator levels; remove the redundant core/orchestrator get_memory_usage() (it was just breakdown.total()). - Keep the convenience total in the C++ runtime bindings only, where the integration needs a single number; the runtime get_memory_usage() now computes it from get_memory_breakdown().total(). Core and orchestrator tests updated to use get_memory_breakdown().total().
|
Tick the box to add this pull request to the merge queue (same as
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
get_memory_usage()returning the total number of bytes a Vamana index has allocated — graph storage + vector data + metadata — so an integrator can accurately report and bound SVS memory consumption.Both the static
VamanaIndexand the dynamicMutableVamanaIndexare covered, and the method is plumbed through the orchestrator layers (VamanaInterfacevirtual →VamanaImploverride →Vamana/DynamicVamana) so it is callable onsvs::Vamanaandsvs::DynamicVamana.Why
Integrators (e.g. memory-bounded module hosts) need to account for memory SVS allocates via
mmap/blocked allocators, which bypass the host'smallocaccounting. The existingblocksize_bytes()reports only the initial block size and is neither an upper nor lower bound on the real footprint.get_memory_usage()reports the true allocated total across all blocks plus metadata.What
Accounting is capacity-based (the bytes the containers have reserved, not just live elements) so that block over-allocation is reflected:
graph_bytesgraph_.get_data().capacity() * element_size()data_bytesdata_.capacity() * data_.element_size()metadata_bytes(dynamic only)A
VamanaMemoryUsage { graph_bytes, data_bytes, metadata_bytes, total() }struct andget_memory_breakdown()expose the per-component split;get_memory_usage()returnsget_memory_breakdown().total().Notes:
detail::dataset_allocated_bytes()helper uses capacity-based accounting when the dataset exposescapacity()(flat/blockedSimpleData), and falls back to live element count otherwise (e.g.SQDataset). No public accessors or signatures were changed.Tests
New unit tests at both the core-index and orchestrator levels for the static and dynamic indices:
tests/svs/index/vamana/index.cpp,tests/svs/index/vamana/dynamic_index.cpptests/svs/orchestrators/vamana.cpp,tests/svs/orchestrators/dynamic_vamana.cppAssertions: usage
> 0for a built index, breakdown components sum to the total, andgraph_bytes/data_bytesare non-zero.[managers]and the touched index tags pass with no regressions.