Skip to content

[SVS][PoC] 2-steps vectors update in SVSTiered#988

Open
rfsaliev wants to merge 6 commits into
mainfrom
rfsaliev/two-step-add-points
Open

[SVS][PoC] 2-steps vectors update in SVSTiered#988
rfsaliev wants to merge 6 commits into
mainfrom
rfsaliev/two-step-add-points

Conversation

@rfsaliev

@rfsaliev rfsaliev commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Proof-of-concept for 2-steps SVS index update

This PR is the PoC to demonstrate potential Tiered SVS index update performance improvements by using 2-steps index update capability implemented in intel/ScalableVectorSearch#348.

Which issues this PR fixes

  1. #...
  2. MOD...

Main objects this PR modified

  1. ...
  2. ...

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Note

Medium Risk
Tiered update locking and new SVS nightly binaries affect index correctness and build reproducibility; PoC status means the two-phase API may still evolve.

Overview
PoC for a two-phase SVS backend update so tiered flat→backend moves can do heavy work under a shared lock and only commit under an exclusive lock, matching upstream add_points_compute_changes / add_points_commit.

SVSIndexBase gains makeAddVectorsChanges and applyAddVectorsChanges (with SVSChangesHandler holding preprocessed vectors and computed changes). TieredSVSIndex::updateSVSIndex uses that path when the backend already exists instead of calling addVectors in one critical section.

CMake switches prebuilt SVS URLs from v0.3.1 to nightly *-pr330.tar.gz artifacts and drops the glibc 2.26 GCC binary path; the minimum glibc message for GCC is now 2.28.

For non-multi indices, topKQuery uses the single-query MutableVamanaIndex scratch search path instead of batch QueryResult search to cut overhead on one query.

Benchmarks add TopKSearchDuringUpdate to stress parallel TopK while a background tiered update runs.

Reviewed by Cursor Bugbot for commit 97173eb. Bugbot is set up for automated code reviews on this repo. Configure here.

@rfsaliev rfsaliev requested a review from alonre24 July 8, 2026 12:36
@rfsaliev rfsaliev marked this pull request as ready for review July 8, 2026 12:37
Comment thread src/VecSim/algorithms/svs/svs_tiered.h
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.80328% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.08%. Comparing base (dd6879e) to head (97173eb).

Files with missing lines Patch % Lines
src/VecSim/algorithms/svs/svs.h 91.22% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #988      +/-   ##
==========================================
- Coverage   97.12%   97.08%   -0.05%     
==========================================
  Files         141      141              
  Lines        8245     8291      +46     
==========================================
+ Hits         8008     8049      +41     
- Misses        237      242       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/VecSim/algorithms/svs/svs.h Outdated
Comment thread src/VecSim/algorithms/svs/svs_tiered.h Outdated
@rfsaliev rfsaliev force-pushed the rfsaliev/two-step-add-points branch from 2d9911a to 1a50b2a Compare July 14, 2026 11:42

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1a50b2a. Configure here.

rep->results.push_back(
VecSimQueryResult{impl_->translate_internal_id(neighbor.id()),
toVecSimDistance(neighbor.distance())});
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TopK ignores requested k

High Severity

For non-multi SVS indices, topKQuery now uses the scratch single-query search path with VamanaSearchParameters from joinSearchParams, but only grows the scratch buffer to k. It does not align the search window with k. With the default search window of 10, a request for more than 10 neighbors can return far fewer results than before the batch QueryResult path.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 1a50b2a. Configure here.

Comment thread src/VecSim/algorithms/svs/svs_tiered.h Outdated
…ufferLimit or trainingThreshold"

This reverts commit 1a50b2a.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant