[essentiaMarco]

Summary

Fixes #320.

Owner-scoped LanceDB rows did not include application and project scope in their primary key, so identical logical identifiers could overwrite one another across projects.

• Use deterministic, length-prefixed internal storage keys over the complete application, project, owner, and logical-entry scope.
• Apply the scoped identity to episodes, atomic facts, foresights, agent cases, agent skills, and user profiles.
• Keep historical public response ID shapes separate from internal LanceDB keys.
• Carry project scope through dependent profile, agent-skill, and reflection paths.
• Add a generation-2 storage marker and cross-process lifecycle locks so legacy, incomplete, or concurrently rebuilt projections fail closed.
• Serialize first-time marker, connection, schema, and index initialization across processes.
• Require rebuild completion with no pending, retryable-failed, or permanently-failed cascade rows before publishing READY(2).
• Document the rebuild, interruption recovery, locking, scope validation, and migration requirements.

Area

• Architecture method
• Benchmark
• Use case
• Documentation
• Developer experience
• CI, build, or release

Compatibility and migration

Public response ID shapes and the existing mixed-case validation for sender and owner identifiers remain unchanged.

This change introduces a stricter contract for app_id and project_id:

• Length: 1 to 128 characters.
• Allowed characters: lowercase ASCII letters, digits, underscore, dot, hyphen, @, and +.
• Rejected values include period-only traversal tokens, trailing dots, Windows device names, and runtime-owned or default-directory aliases.
• The literal default remains valid. default_app and default_project are reserved filesystem aliases.

Before upgrading an existing memory root, both its source directories and retained SQLite scope values must be audited.

• A root with conforming scope names can be rebuilt with everos cascade rebuild --yes after stopping the server and other mutating cascade commands.
• This release does not provide a general in-place migration for nonconforming legacy scope names.
• A nonconforming root must either be imported into a new root under accepted scope names or use a separately reviewed deployment-specific migration that updates source paths and matching SQLite columns, path references, and serialized scope values together.
• cascade rebuild reconstructs LanceDB from Markdown but intentionally preserves SQLite state, so it does not perform the legacy scope-name migration.
• After a root has been rebuilt for generation 2, an older EverOS binary must not be run against that root.

Verification

make ci
make docs-check
uv run pytest -q tests/e2e/test_get_endpoint_e2e.py tests/e2e/test_search_endpoint_e2e.py
git diff --check

All commands passed on macOS with Python 3.12.13:

• make ci: lint passed; 2,238 unit tests and 191 integration tests passed; 7 integration tests were deselected by the repository configuration; source and wheel packages built; wheel smoke installation passed.
• Endpoint E2E: 57 passed and 15 were deselected by the repository configuration.
• Documentation, contributor-document, issue-template YAML, and whitespace checks passed.

The real-LanceDB matrix covers all six owner-scoped schemas for cross-project coexistence, replay idempotency, isolated updates, injectivity, and public response-ID compatibility where exposed. Separate cascade, marker, API/CLI lifecycle, and multiprocess lock tests cover end-to-end indexing, migration gates, failed or interrupted rebuilds, serialized first boot, cancellation-safe cleanup, and rebuild exclusion.

Credentialed live-provider tests were not run as part of this verification.

Checklist

• I kept the change scoped to the relevant area.
• I am opening this from a separate branch, not pushing directly to main.
• I updated docs, examples, or setup notes when behavior changed.
• I added or updated tests when the change affects behavior.
• I did not commit secrets, environment files, dependency folders, or build artifacts. The repository-tracked docs/openapi.json snapshot was intentionally regenerated because the API contract changed.
• Active relative links in Markdown files resolve.

Notes for Reviewers

A markerless root initializes generation 2 automatically only when it contains neither extracted-memory source Markdown nor LanceDB artifacts. Existing source or projection data without a current marker fails closed.

If rebuild fails or is interrupted after publishing REBUILDING, normal startup and mutating cascade commands remain blocked. Correct the underlying problem and rerun everos cascade rebuild --yes with the server stopped.

By submitting this pull request, I agree that my contribution is licensed under
the Apache License 2.0.

[gloryfromca]

Thanks for this, and for separating internal storage keys from the public response id shape — that distinction carried into the fix that landed.

#320 is now fixed on main via #481: every business table declares PRIMARY_KEY = ("app_id", "project_id", "id"), with the index layer merging on the declared key. For the migration concern this PR addresses with a generation marker and projection locks, main instead runs a one-time reconcile pass on first start after upgrading, recorded in .index_reconcile_version (#481, completed in #487).

The record models this PR edits under infra/persistence/lancedb/tables/ have since moved to infra/persistence/records/, so rebasing would mean rewriting most of the diff against a fix that is already in. Closing as superseded — thank you for the thorough write-up.