[SVS][PoC] 2-steps vectors update in SVSTiered#988
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #988 +/- ##
==========================================
- Coverage 97.12% 97.08% -0.05%
==========================================
Files 141 141
Lines 8245 8291 +46
==========================================
+ Hits 8008 8049 +41
- Misses 237 242 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…it or trainingThreshold
2d9911a to
1a50b2a
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 1a50b2a. Configure here.
| rep->results.push_back( | ||
| VecSimQueryResult{impl_->translate_internal_id(neighbor.id()), | ||
| toVecSimDistance(neighbor.distance())}); | ||
| } |
There was a problem hiding this comment.
TopK ignores requested k
High Severity
For non-multi SVS indices, topKQuery now uses the scratch single-query search path with VamanaSearchParameters from joinSearchParams, but only grows the scratch buffer to k. It does not align the search window with k. With the default search window of 10, a request for more than 10 neighbors can return far fewer results than before the batch QueryResult path.
Reviewed by Cursor Bugbot for commit 1a50b2a. Configure here.
…ufferLimit or trainingThreshold" This reverts commit 1a50b2a.


Proof-of-concept for 2-steps SVS index update
This PR is the PoC to demonstrate potential Tiered SVS index update performance improvements by using 2-steps index update capability implemented in intel/ScalableVectorSearch#348.
Which issues this PR fixes
Main objects this PR modified
Mark if applicable
Note
Medium Risk
Tiered update locking and new SVS nightly binaries affect index correctness and build reproducibility; PoC status means the two-phase API may still evolve.
Overview
PoC for a two-phase SVS backend update so tiered flat→backend moves can do heavy work under a shared lock and only commit under an exclusive lock, matching upstream
add_points_compute_changes/add_points_commit.SVSIndexBasegainsmakeAddVectorsChangesandapplyAddVectorsChanges(withSVSChangesHandlerholding preprocessed vectors and computed changes).TieredSVSIndex::updateSVSIndexuses that path when the backend already exists instead of callingaddVectorsin one critical section.CMake switches prebuilt SVS URLs from v0.3.1 to nightly
*-pr330.tar.gzartifacts and drops the glibc 2.26 GCC binary path; the minimum glibc message for GCC is now 2.28.For non-multi indices,
topKQueryuses the single-queryMutableVamanaIndexscratch search path instead of batchQueryResultsearch to cut overhead on one query.Benchmarks add
TopKSearchDuringUpdateto stress parallel TopK while a background tiered update runs.Reviewed by Cursor Bugbot for commit 97173eb. Bugbot is set up for automated code reviews on this repo. Configure here.