feat: sqlite-vec vector search #270
Draft
riteshshukla04 wants to merge 7 commits into
Draft
Conversation
Move the library from package/ to packages/react-native-nitro-sqlite so the workspace can host sibling packages (e.g. a future vector extension), updating all build, config, CI, and example references for the new path. Add a React Native Harness on-device test setup in the example app: rn-harness.config.mjs (Android physical-device + iOS sim runners), jest.harness.config.mjs (scoped to *.harness.* files), __tests__/sqlite.harness.ts (13 tests covering execute, transactions, batch, ArrayBuffer/BLOB), plus react-native-harness@1.3.0 and platform packages. Verified on a physical Samsung SM-E146B (Android 15): 13/13 Harness tests pass and the in-app Mocha suite is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add example/__tests__/sqlite-vec.harness.ts: 35 on-device tests covering the full sqlite-vec surface (vec_* scalar/table functions, all four distance metrics, quantization, and vec0 virtual tables — KNN in both query forms, cosine metric, metadata filtering, auxiliary + columns, partition keys, int8/bit vectors, UPDATE/DELETE, and error cases). These run through the existing execute() API and are RED until sqlite-vec is statically linked: verified on the Samsung SM-E146B as 35/35 failing with 'no such module: vec0' / 'no such function: vec_*', confirming the SQL reaches SQLite and only the extension is missing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add react-native-nitro-sqlite-vec: a companion package vendoring the sqlite-vec v0.1.9 amalgamation. When enabled (Android: nitroSqliteVec=true), its native sources compile INTO the core react-native-nitro-sqlite library's single sqlite3 build and register via sqlite3_auto_extension — no second sqlite3, no runtime extension loading (faster than expo-sqlite's loadExtension path). Vector search runs through the core's existing execute() API (vec0 + vec_* functions); thin typed helpers (vecVersion/createVectorTable/knnSearch) are included. Core changes: android CMakeLists/build.gradle flag-gated compilation of the companion's sqlite-vec.c (-DSQLITE_CORE -DSQLITE_VEC_STATIC); operations.cpp registers the extension once in sqliteOpenDb (#ifdef NITRO_SQLITE_VEC); bindStatement now binds whole-number JS values as INTEGER (vec0 rowid/pk/partition require INTEGER; SQLite still coerces to REAL for REAL columns); sqliteExecuteBatch fixes a double-rollback that masked the real batch error. Verified on a physical Samsung SM-E146B (Android 15): sqlite.harness.ts (13) + sqlite-vec.harness.ts (35) = 48/48 pass. Registration is a single usearch-ready seam for future ANN backends. iOS podspec wiring is next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… the core's single sqlite3 react-native-nitro-sqlite-vec ships its own podspec (RNNitroSqliteVec.podspec) + react-native.config.js (iOS autolink; Android skipped — the core's CMake compiles these sources there). On iOS the companion compiles sqlite-vec with SQLITE_CORE/SQLITE_VEC_STATIC and static-links into the core's single sqlite3; USER_HEADER_SEARCH_PATHS pins our sqlite3.h ahead of any other (e.g. expo-sqlite's exsqlite3) when both coexist. Core RNNitroSQLite.podspec defines NITRO_SQLITE_VEC + adds the companion header path when ENV NITRO_SQLITE_VEC=1. operations.cpp uses an angle-bracket include so registerVectorExtensions.hpp resolves via -I on both CMake (Android) and CocoaPods (iOS). Validated: Harness 48/48 on a physical Samsung SM-E146B (Android) and iPhone 17 Pro simulator (iOS). Flag verified both ways (off: 13 core tests pass, 0 vec symbols; on: 48/48), core works with the companion uninstalled, and errors clearly if the flag is on but the package is missing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…c char Condense every multi-line comment in the sqlite-vec work to a single line, and remove a stray '.' that had crept into RNNitroSqliteVec.podspec (would break pod install; ruby -c now OK). Performance reviewed against Nitro's perf guide — already on the fastest path: static link + sqlite3_auto_extension (no runtime load), queries via the existing C++/JSI execute(), BLOB vectors returned as zero-copy ArrayBuffer, no new dynamic types or Swift/Kotlin bridging. No code change needed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an opt-in true-int64 parameter path so callers can pass integers beyond 2^53 without precision loss, using Nitro's Int64 (a branded bigint). The existing JS-`number` heuristic is kept and unchanged: whole numbers still bind as INTEGER (required by sqlite-vec's vec0 rowid/pk/partition columns, which reject REAL), large/fractional ones as REAL. - types.ts: split params from results — add SQLiteParamValue (SQLiteValue | Int64); SQLiteQueryParams and both ExecuteQuery / ExecuteAsyncQuery now accept it. Results (SQLiteValue) stay number. - nitrogen: regenerated; execute() param variant gains int64_t, getResults() variant unchanged. - types.hpp: add SQLiteParamValue variant matching the generated order. - operations.cpp: bindStatement handles int64_t directly (exact bind, no double round-trip); extract the number heuristic into bindJsNumber with numeric_limits<int64_t>-derived bounds (no magic literals). - harness: 2 added tests — a 2^53+1 bigint stored exactly, and a whole number still binding as INTEGER. Verified on-device (Samsung SM-E146B): 50/50 harness (15 sqlite + 35 vec). Removing the number heuristic was empirically shown to break 12 vec tests with vec0's "Only integers are allows for primary key values". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
riteshshukla04
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sqlite VEC + Harness + MONOREPO