Skip to content

feat: run Pathway on Cyndrbase (replace Convex and cloudflared) - #326

Draft
coreybain wants to merge 35 commits into
mainfrom
feat/cyndrbase
Draft

coreybain wants to merge 35 commits into
mainfrom
feat/cyndrbase

Conversation

@coreybain

@coreybain coreybain commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

What Changed

Pathway runs on Cyndrbase instead of Convex, and Pathway Connect uses Cyndrbase Connect instead of cloudflared. This merges two branches: feat/cyndrbase (Convex replacement) and feat/cyndrbase-connect (tunnel replacement).

Backend and data

  • packages/backend/convex imports cyndrbase/server and cyndrbase/values and deploys with cyndr. The convex runtime dependency and convex dev are gone.
  • Clients keep the Apache-2.0 convex/browser client, talking Convex wire protocol to Cyndrbase.

Dev runner

  • Starts a worktree-local Postgres and cyndrd, plus cyndr dev, which redeploys on change.
  • Web reaches the engine through a same-origin /cyndrbase Vite proxy. The proxy allowlists client routes only, and .pathway/** is denied.

Pathway Connect

  • Removes cloudflared download, install and run.
  • The server runs the connector bundled in @cyndrbase/connect.
  • The relay provisions one endpoint per environment with one stable, sealed connector token, through connect.v1.EndpointService.
  • Provision adopts an endpoint that already holds the hostname. Only unlink removes endpoints.
  • Allocation writes are fenced on the allocation _id.

CI and release

  • Relay deploys, desktop/web releases and iOS releases skip with a notice until CYNDRBASE_DEPLOYMENT_URL / RELAY_CYNDRBASE_DEPLOYMENT_URL exist. There is no Convex fallback.
  • The relay deploy fails if any Connect setting is missing.
  • iOS is retargeted to PATHWAY_CYNDRBASE_URL.

Server

  • Realtime auth refetches answer null when no token can be minted, so a logout no longer crashes the server.

Why

Pathway moves off Convex and Cloudflare onto Cyndrbase, its own Convex-compatible backend and tunnel service.

Each part was built by Opus 5.5 and audited by GPT-6.1 Sol until clean. The last audit of the merged branch found no defects.

Verification:

  • 659 focused tests pass.
  • Local Cyndrbase checks pass 10/10: deploy, 15 crons, HTTP routes, relay tokens, live updates.
  • Loopback Connect end-to-end passes, including DPoP over HTTPS through the tunnel.

⚠️ Draft: not mergeable yet

  • Install will fail in CI. Cyndrbase packages are link:ed to a local checkout and need publishing to npm first.
  • Base is stale. The branch is 35 commits behind main and needs a rebase.
  • Cutover blockers:
    • hosted Cyndrbase and Connect edge (DNS/TLS);
    • production CYNDRBASE_* vars/secrets, including CYNDRBASE_CONNECT_TOKEN_KEY;
    • Clerk dev-instance verification;
    • Convex data migration;
    • uploadfile deploy.
  • Cutover order is in infra/relay/README.md ("Cutover from Convex").

Depends on SpiritDevs/cyndrbase#49 (stacked on quick tunnels) and SpiritDevs/uploadfile#1 / #2.

Checklist

  • This PR is small and focused (no: this is a platform migration; draft for review and safekeeping)
  • I explained what changed and why
  • I included before/after screenshots for any UI changes (no UI changes)
  • I included a video for animation/interaction changes

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Corey Baines and others added 30 commits October 8, 2026 22:08
packages/backend/convex now imports cyndrbase/server and cyndrbase/values,
and `cyndr codegen` regenerates _generated against cyndrbase (the checked-in
api.d.ts also predated http.ts). The convex runtime dependency and the Convex
CLI scripts are gone; @cyndrbase/cli provides `cyndr`.

The unit tests still run on convex-test, which executes Convex's npm
builders: Cyndrbase has no in-process harness yet. A test-only alias
(vite.config.ts, tsconfig.test.json) maps cyndrbase/* to the convex modules,
so `convex` stays only as a devDependency for that harness. Production code
typechecks against cyndrbase alone.

http.ts casts the stored Blob because apps/server, which now typechecks the
module through api.d.ts, has no DOM Blob.slice().

The cyndrbase packages are linked from a local checkout until they publish.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
apps/server, apps/web and infra/relay build their references with
cyndrbase/server; they share Convex's Symbol.for("functionName") key, so the
convex/browser client still calls them. convex/values stays where code
handles client errors: convex/browser throws its own ConvexError class, and
instanceof only matches that class.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`cyndr deploy` regenerates _generated on every run, so formatting it left the
tree dirty after each deploy. Lint already ignores it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
PATHWAY_CONVEX_URL becomes PATHWAY_CYNDRBASE_URL (and VITE_/__PATHWAY_BUILD_
variants); the relay reads CYNDRBASE_DEPLOYMENT_URL. Desktop artifact builds
deploy with `cyndr deploy` using CYNDRBASE_URL, CYNDRBASE_DEPLOYMENT and
CYNDRBASE_DEPLOY_KEY, still refusing a deployment that is not the one the app
bakes in. The release and relay workflows read the matching Actions
variables and secret. Wire identifiers (pathway-convex audience, token typ
values, /v1/environment/convex-token) are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…oudflared

Pathway Connect downloaded, verified, installed and supervised cloudflared,
and the relay provisioned a Cloudflare tunnel, ingress config and CNAME per
environment through the Cloudflare API (and deleted the tunnel on shutdown
because Cloudflare bills per tunnel).

Now the relay provisions one persistent Cyndrbase Connect endpoint per
environment through Connect's EndpointService and rotates an
endpoint-scoped connector token on every provision, revoking the previous
one before recording the new one so a failed provision never leaves an
untracked live token. Unlink revokes the token and removes the endpoint.
The server runs the connector bundled in @cyndrbase/connect, with the same
restart/backoff supervision, and readiness comes from the connector's
registration receipt.

- Delete the cloudflared download/install/lock code, the
  cloud.installRelayClient RPC, the CLI install prompt, the dead
  release-on-shutdown path and its relay endpoint, and the Cloudflare
  Tunnel and DNS bindings. cloud.getRelayClientStatus stays (same wire
  shape) for shipped iOS clients and now reports the bundled connector.
- The runtime config the relay hands the server is now
  {environmentId, edgeUrl, endpointId, connectorToken, origin}.
- "cloudflare_tunnel" stays as the providerKind wire literal: Convex
  validates it and shipped clients send it.
- Desktop builds unpack @cyndrbase/connect/bin from app.asar on every
  platform, and the server runs it from app.asar.unpacked.
- Relay reads CYNDRBASE_CONNECT_API_URL/EDGE_URL/ADMIN_KEY; without them
  managed tunnels fail as not configured.

@cyndrbase/connect is linked from a local checkout until it is published.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Modes that run the server now start a Postgres cluster under
<checkout>/.pathway/cyndrbase (never a shared developer database), a
loopback `cyndrd --local-deploy` engine, and deploy packages/backend with
`cyndr`, then hand the server and web the deployment URL. The deployment's
Clerk and relay issuers derive from the same public config the clients use;
without Clerk or a relay, cloud sync stays off and nothing starts. An
explicit PATHWAY_CYNDRBASE_URL still targets that deployment instead.

Data survives restarts. The engine keeps code in memory, so every start
redeploys. Each start writes .pathway/cyndrbase/cyndr.env so `cyndr run`,
`cyndr env` and psql can reach the running backend.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The smoke hooks called the internal smoke:* functions through
`npx convex run`, which no longer exists here. They now run the backend's
own `cyndr` (never an npx registry lookup), pinned with
CYNDRBASE_DEPLOYMENT and cross-checked against the deployment URL's first
hostname label. The live run reads PATHWAY_CYNDRBASE_* settings.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… change

The engine releases its lease on shutdown and only hands it to a higher
epoch, so a second `vp run dev` on the same .pathway/cyndrbase failed to
start. Each start now passes the current time as its epoch. The one-shot
deploy becomes `cyndr dev`, which keeps redeploying while backend sources
(convex/ and anything it imports) change, like `convex dev` did.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Pathway's backend now deploys to Cyndrbase, but the living docs still
named Convex as the service, documented `npx convex` commands, and listed
the old CONVEX_URL and CONVEX_DEPLOY_KEY settings.

Architecture docs and runbooks now name Cyndrbase. The relay README
bootstraps a deployment with `cyndr env set` and `cyndr deploy`, the
Actions configuration lists the CYNDRBASE_* variables and secret, and
AGENTS.md explains the checkout-local backend that `vp run dev` starts.
The glossary gains a Cyndrbase entry, which also lists the Convex-named
wire identifiers that keep their names. Historical records, iOS docs and
the iOS release variable still say Convex.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
apps/server typechecks http.ts without DOM types; the narrowed Blob made
the sliced body unassignable there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ecycle

An audit of the Cyndrbase Connect migration found these gaps:

- The runtime config change broke older servers and stored configs. It
  again carries providerKind and connectorToken, with edgeUrl/endpointId
  optional. Relays send "pathway_relay", which older servers decode and
  report as unsupported instead of starting cloudflared. A stored or
  relay-issued cloudflared config now reports needs_reprovision. At startup,
  links not managed by the CLI fetch a fresh config from a new
  environment-authenticated relay endpoint,
  POST /v1/environments/:id/managed-endpoint. That endpoint only serves the
  credential of the caller's own managed link.
- Token rotation now issues, records, then revokes the replaced tokens. A
  failed record keeps the old token live and tracked, and the retry reuses
  the new token. A failed revoke still returns the new token, and the old
  one stays recorded for the next provision or unlink.
- Applying a relay config waits for the connector's registration. A
  rejected token or an unreachable edge (15 s) now fails the link instead
  of reporting running.
- Exit code 1 from @cyndrbase/connect (credential rejected) is terminal, so
  a revoked token no longer respawns forever.
- The connector exposes only 127.0.0.1 on the authorized listener port.
  The runtime config no longer carries an origin.
- Remote Cyndrbase Connect API and edge URLs must use https/wss.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With CYNDRBASE_FILES_CONFIG set, uploads commit back to, and getUrl links
point at, the binding's content_origin, which cyndrd serves on its own
listener. The engine listened on a random port, so storage would have
failed and stored icon URLs would have broken on restart. It now listens
on the binding's origin, and a binding without a usable one is refused
before anything starts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The production environment only has the old CONVEX_URL and
CONVEX_DEPLOY_KEY, so the relay deploy on main pushes and the scheduled
release would build against an empty Cyndrbase URL. Each now checks the
production CYNDRBASE_DEPLOYMENT_URL first and, when it is unset, skips the
Cyndrbase-targeting jobs with a notice. Nothing falls back to Convex.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The iOS app reads PATHWAY_CYNDRBASE_URL like desktop and web: the
configure script, xcconfig, Info.plist key and TestFlight checks use the
new name, and the TestFlight workflow takes CYNDRBASE_DEPLOYMENT_URL,
skipping with a notice until it exists. ConvexMobile keeps speaking the
Convex wire protocol, which Cyndrbase serves, so only the URL changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dev runner handed web the engine's loopback URL, which Vite baked into
the bundle, so a browser on another machine or behind a tunnel called its
own loopback. Single-origin dev now proxies /cyndrbase (HTTP and the sync
WebSocket) to a loopback engine and bakes that path, which web resolves
against the page origin. Hosted and desktop URLs are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`cyndr env set NAME VALUE` put RESEND_API_KEY and UPLOADTHING_TOKEN in the
CLI's argv. The dev runner now pipes every value on stdin (`cyndr env set
NAME [--secret]`), which needs the matching cyndr change in Cyndrbase #50.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Connect grants and registrations live in the relay's backend, but installed
clients write them to the backend their build baked in, so the relay can
only move to Cyndrbase after the data and the clients have.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The /cyndrbase dev proxy forwarded every engine path, including the
DeployService, EnvironmentService, bundle upload, data browser and upload
commit routes, and Vite's /@fs/ served .pathway/cyndrbase/deploy-key and
cyndr.env. Anyone reaching a shared dev origin (LAN or tunnel) could take
the key, change env or deploy code. The proxy now forwards only the Convex
sync socket, function calls and file downloads, and the dev server denies
**/.pathway/** (Pathway homes and the local backend's state) on top of
Vite's default deny list. A test runs real Vite servers against a fake
engine to check both.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Local storage URLs name the engine's loopback content origin, and some are
saved (project icons), so over a tunnel or LAN they pointed at the visitor's
own machine. When web reaches its backend through /cyndrbase, file URLs on a
loopback origin are rewritten to the same path under the page origin, where
the dev proxy forwards /.files/download. Project icons, thread queue and
calendar attachments, and issue attachment URLs go through the helper; any
other URL, and every hosted deployment, is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Releases and the relay deploy both waited on CYNDRBASE_DEPLOYMENT_URL, so
letting clients ship against Cyndrbase also moved the relay on the next
main push, contrary to the cutover order. The relay deploy now reads and
gates on its own RELAY_CYNDRBASE_DEPLOYMENT_URL, and the relay README's
cutover steps use the two variables in order.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Provisioning now only issues a token and records it as pending with a
compare-and-set on the allocation, so a lost response or failed commit
strands no connector. The environment confirms a token once its connector
registered; the relay then promotes it and revokes the tokens it replaced.
Overlapping rotations converge on one pending token, a stale confirm is a
no-op, and a revoke outage refuses new tokens past a small cap.

The runtime waits for registration when reapplying an unregistered
connector, and stops a connector whose apply was interrupted.

Adds an additive Convex mutation, recordManagedEndpointDnsIfUnchanged,
for the compare-and-set.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The /cyndrbase allowlist matched a prefix, so a raw request such as
/cyndrbase/.files/download/../../cyndrbase.platform.v1.EnvironmentService/SetEnvVar
(or its %2e%2e form) passed and reached the engine's admin RPCs after
dot-segment resolution, over HTTP and WebSocket upgrades alike. Every
/cyndrbase/ request now goes through one check on the raw request target,
which Vite runs for both, and anything else answers 404. Route segments are
spelled only from characters that decoding and dot-segment resolution leave
alone, so `..`, `%2e`, `%2f` and `//` can never match and the checked path
is the one the engine receives.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The loopback file URL rewrite ran for any path-relative backend base. It now
applies only in a dev build whose backend is the local engine at /cyndrbase.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vite checks server.fs.deny against the requested path and then serves
through symlinks, so an alias to .env or .pathway inside the repo would be
served. Closing that would mean copying Vite's alias, public-dir and /@fs
resolution into a middleware; the dev notes say not to create such links.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Replace confirm-based rotation with one connector token per allocation,
as cloudflared had one token per tunnel. The first provision mints it and
stores it sealed with CYNDRBASE_CONNECT_TOKEN_KEY; later provisions return
it, and only unlink revokes it.

The token slot (the allocation's dnsRecordId column) is written only by a
compare-and-set on its value inside one Convex mutation. A provision marks
the slot before it sweeps the endpoint's unstored tokens and mints, and
stores only over its own mark, so a lost mint response leaves at most one
extra live token and a sweep never revokes a token that can still be
stored. Unlink marks the slot for teardown, which no provision can undo;
a retry or the next provision finishes it.

Removes the confirm endpoint, connectorTokenId, pending/current/retired
state, rotation idempotency keys, the server's confirm call and rollback,
and recordManagedEndpointDnsIfUnchanged. Adds the Convex mutations
swapManagedEndpointTokenSlot and removeManagedEndpointAllocationWithTokenSlot.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…jected tokens

Every allocation write after reserve now names the allocation's Convex _id,
so a stale unlink can no longer revoke, remove, or delete the allocation a
relink created, and a stale provision cannot write into it.

Mints for an allocation's endpoint share one idempotency key, so retried,
racing and delayed mints get the token the edge minted first. The sweep
moves after the store and revokes only tokens other than the stored one,
which retires the Minting marker; every provision runs it, so a token
minted after the edge forgets the key is revoked by the next provision.

When the edge rejects a running connector's token, the server asks the
relay once for the link's current config. A different token is applied;
the same token stops with "relink to restore remote access".

Deletes the unused Convex mutations claimManagedEndpointDeprovision,
removeClaimedManagedEndpointAllocation, recordManagedEndpointDns and
claimManagedEndpointRelease, plus removeManagedEndpointAllocationWithTokenSlot.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Corey Baines and others added 5 commits October 9, 2026 01:53
…ink finish

Removes the server's recovery from a rejected connector token: recovery
could race an unlink and re-create what it had removed. A rejection is
terminal again and logs that a relink is needed.

Only the provision that mints and stores the endpoint's token lists the
edge's tokens to revoke strays, so reusing a stored token makes no list
calls. A mint delayed past the edge's idempotency receipt is now revoked
only at unlink; the hosted edge's durable receipts close that gap.

A provision that finishes an interrupted unlink tears down only the
allocation it observed, never a relink's newer one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…o an unlink

If a provision creates an endpoint and an unlink finishes before it is
recorded, the provision's own cleanup may fail, leaving an endpoint that
holds the environment's hostname with no allocation. Every relink then
failed with already_exists. The relay owns its hostnames, so when
CreateEndpoint conflicts and the allocation does not record the endpoint
holding the hostname, provision removes it (which revokes its tokens on
the edge) and creates a fresh one. An endpoint the allocation already
records, as a concurrent provision's, is kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…an be minted

The worker wakeup, orchestrator control and sync head subscriptions handed
the realtime client a fetcher that rejected when the service token provider
failed. After `pathway connect logout` unlinks the environment the provider
fails `unauthorized`, and the client's refetch after the server confirms a
token runs detached, so the rejection was unhandled and the server exited.

All three now share makeConvexServiceTokenFetcher, which logs the failure
and answers null, as the thread queue worker already did; the client then
reports lost authentication instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…oves

Provision no longer removes endpoints. Removing the endpoint that held a
hostname let a stale provision delete a relink's live endpoint, and let
two provisions recovering the same orphan delete each other's. Now a
provision whose CreateEndpoint hits already_exists adopts the endpoint
holding its hostname and records it with the allocation-fenced write,
which reports whether it held: a stale provision stops there, before it
mints anything. The mint path's sweep revokes an adopted orphan's leftover
token. The losing provision's endpoint removal is gone too.

Removal plans now go stale when an endpoint gains a token, so the relay
plans afresh on every call instead of replaying a stale outcome under a
target-derived key; a retry after the change landed still gets NotFound.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Brings Pathway Connect on Cyndrbase Connect, which replaces cloudflared, onto
the Cyndrbase backend port, along with the six main commits (#318, #319) that
the connect branch was based on.

Conflict resolved:
- infra/relay/README.md: kept the Cyndrbase deployment text and "cloud sync",
  dropped "managed endpoint resources" (the stack no longer provisions
  Cloudflare tunnels), and kept the Pathway Connect paragraph.

Also in this merge:
- deploy-relay.yml keeps the RELAY_CYNDRBASE_DEPLOYMENT_URL gate and now fails
  the production deploy when a Connect setting is missing; the relay README
  and .env.example say so.
- relayPersistence.ts merged textually: the cyndrbase/values import plus the
  allocation-fenced managed-endpoint mutations. cyndr codegen leaves
  convex/_generated unchanged.
- The lockfile unions the cyndrbase, @cyndrbase/cli and @cyndrbase/connect
  links; vp install --frozen-lockfile passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant