Remove NodeStore abstraction, use SurrealDB directly #470

Context

Epic #461 successfully implemented SurrealStore but left the migration incomplete. NodeService still has dual dependencies:

• store: Arc<dyn NodeStore> - Trait abstraction (partially used)
• db: Arc<DatabaseService> - Direct Turso/SQLite access (still heavily used)

Discovery: 12+ methods in NodeService still call self.db.connect_with_timeout() directly. Embedding services also depend on DatabaseService.

Goal

Complete the migration Epic #461 started by:

1. Removing the NodeStore abstraction layer
2. Migrating all remaining DatabaseService usage to SurrealStore
3. Making SurrealDB the only backend

Revised Scope

Keep (Reuse from #461)

• packages/core/src/db/surreal_store.rs - Full SurrealDB implementation (670 lines)
• SurrealDB schema design and documentation
• All SurrealStore unit tests

Delete (Abstraction Overhead)

• ✅ packages/core/src/db/node_store.rs - Trait definition
• ✅ packages/core/src/db/turso_store.rs - Wrapper around DatabaseService
• ✅ packages/core/src/db/database.rs - Old DatabaseService (Turso/SQLite)
• ✅ packages/core/src/db/ab_testing.rs - A/B testing framework
• ✅ packages/core/src/db/metrics.rs - Metrics collection
• ✅ packages/core/src/db/ab_tests.rs - Integration tests comparing backends

Migrate (Complete Epic #461's Partial Work)

• packages/core/src/services/node_service.rs
  • Remove db: Arc<DatabaseService> field
  • Change store: Arc<dyn NodeStore> to store: Arc<SurrealStore>
  • Replace all self.db.* calls with self.store.* methods
  • Remove trait-based indirection (direct method calls)
• packages/core/src/services/embedding_service.rs
  • Replace Arc<DatabaseService> with Arc<SurrealStore>
  • Update constructor and internal logic
• packages/core/src/services/embedding_processor.rs
  • Replace DatabaseService dependency
• packages/core/src/db/mod.rs
  • ✅ Export SurrealStore directly
  • ✅ Remove NodeStore trait exports
• packages/core/src/lib.rs
  • ✅ Export SurrealStore instead of DatabaseService
• All service initialization code (desktop-app)
  • Replace DatabaseService creation with SurrealStore
  • Update dependency injection

Acceptance Criteria

• DatabaseService completely removed from codebase
• NodeStore trait abstraction removed
• TursoStore wrapper removed
• A/B testing infrastructure removed
• NodeService uses SurrealStore directly (no trait, no dual db field)
• NodeService has zero self.db references
• Embedding services use SurrealStore
• All initialization code uses SurrealDB
• All tests pass (no regressions)
• Code compiles without warnings
• Documentation updated (no references to Turso/abstraction/DatabaseService)

Testing

• Run full test suite: bun run test:all
• Run Rust tests: cargo test --package nodespace-core
• Verify SurrealDB initialization works
• Confirm all NodeService operations work with SurrealStore
• No trait dispatch overhead (direct method calls)

Estimated Effort

3-4 hours (revised from original 2-3 hours estimate)

Breakdown:

• 1 hour: Complete NodeService migration (remove self.db field, migrate all calls)
• 1 hour: Update embedding services
• 30 min: Fix initialization code in desktop-app
• 30 min: Clean up test files
• 1 hour: Testing and bug fixes

Dependencies

• Epic SurrealDB Migration - Hybrid Architecture (Desktop-Only) #461 (merged to main) - Provides SurrealStore implementation
• SurrealStore implementation fully functional

Strategic Rationale

Why SurrealDB:

1. Built-in version history (planned feature)
2. RBAC for collaborative sync (planned feature)
3. Better performance characteristics (proven in PoC SurrealDB Feasibility PoC - Quick Evaluation #460)
4. Simpler to maintain one backend pre-launch

Why remove abstraction:

1. Zero users = no migration needed
2. Won't switch back to Turso
3. Epic SurrealDB Migration - Hybrid Architecture (Desktop-Only) #461's migration was incomplete - finish what was started
4. ~2,000 lines of abstraction code we'll never use
5. Direct access to SurrealDB-specific features needed
6. Maintaining half-migrated code is worse than completing the migration

Why complete now:

• Tech debt compounds - half-migrated code is hardest to maintain
• Zero users means perfect time for breaking changes
• Need direct SurrealDB access for future features (versioning, RBAC)

Related:

• Epic SurrealDB Migration - Hybrid Architecture (Desktop-Only) #461: SurrealDB Migration (merged, but migration incomplete)
• Issue SurrealDB Feasibility PoC - Quick Evaluation #460: SurrealDB PoC (performance validated)

Status: In Progress - Abstraction layer deleted, NodeService migration underway