fix(core): bound the model field at the observation boundary, and make the privacy guards prove it

CHANGELOG.md

@@ -5,6 +5,17 @@
 ### Added (CLI)
 - `codeburn sync push --attribution` (opt-in): sends git attribution spans — the session→commit correlation from `codeburn yield` (`codeburn.session.attribution` and `codeburn.commit` span types with normalized repo remote, commit SHAs, merged/reverted state, and PR links). Nothing new is sent without the flag; local-only repos and Windows filesystem paths are never emitted as repo identities, and sessions whose project path no longer resolves never inherit the push-time working directory's repo. See docs/sync/README.md "Git attribution".
 
+### Changed (@codeburn/core — breaking; version bump deferred to the next release, which must take at least a minor under 0.x)
+- **The `model` field is bounded at the observation boundary.** `model` (and the optional `pricingModel`) on a `CallObservation` must now match the identifier charset `[A-Za-z0-9._:/@-]`, max 128 chars, in both the zod schema and the published `schemas/observation-0.2.0.json`. Every provider's `toObservations` normalizes through `normalizeModelIdentifier`, so a provider display name — Antigravity's `"Gemini 3.5 Flash (High)"`, Warp's and Devin's display strings — surfaces as `unknown` in an emitted observation rather than rejecting the whole envelope. **CLI output is unaffected:** normalization happens only at the observation boundary, so terminal, dashboard, menubar, and desktop numbers and model labels are byte-identical to before. The bound is a tightening of `observation-0.2.0` in place; an ARCHIVED pre-hardening 0.2.0 envelope whose `model` held a display name now fails validation against that same version string and must be re-normalized before re-validating.
+- **The diagnostics API changed shape.** `DIAGNOSTIC_DETAIL_MAX` is deleted, `sanitizeDetail(value)` becomes `sanitizeDetail(value, key)` and returns a 16-hex keyed fingerprint instead of a truncated message, `DiagnosticDetail` now accepts only that fingerprint, `RecordOutcome.diagnostics` may no longer carry a `detail`, and `isolateRecords` takes an optional `privacyKey`. A diagnostic now carries a record index, a controlled code, and a keyed fingerprint — never content. The previous rule (strip path separators, cap at 200 chars) let any slash-free path, command fragment, prompt line, or API key through verbatim. Callers with no key omit the field entirely; core never degrades to an unkeyed digest.
+- **`decodeCopilot` throws on an empty `privacyKey`** when the record is a JetBrains store. That arm derives its per-turn dedup key with `createHmac(privacyKey, replyText)`, and `createHmac` accepts `''` silently — returning a digest anyone can recompute from the reply text, which is the exact weakness the HMAC replaced. It now refuses, matching every other keyed primitive in the package (`fingerprint.ts`, `sanitizeDetail`). A host calling `decodeCopilot` with `context.privacyKey: ''` used to get a (weak) result and now gets an error; supply a real per-install key.
+- **`SOURCE_REF_KEYED_PARSE_VERSION` is no longer exported** from the CLI's session-cache module. It was a marker token that `computeEnvFingerprint` substring-matched to decide which providers re-parse when the host privacy key changes, and that selector silently excluded copilot — whose dedup keys are equally key-derived but whose parse version names no source ref. It is replaced by the explicit `KEY_DERIVED_PROVIDERS` set.
+
+### Fixed (privacy)
+- **Dedup keys no longer carry raw host paths.** `dedupKey` ships on the observation envelope, and the codebuff, zerostack, pi, omp, grok and LingTai TUI decoders were folding the absolute source path (a chat directory, a session file) straight into it. They now fold a keyed HMAC fingerprint of that path (`sourceRefFingerprint`, new `source` domain), and the CLI bridge threads the real per-install privacy key into the rich decode instead of an empty one. (#933, thanks @ozymandiashh)
+- **Copilot's JetBrains dedup key is keyed.** The per-turn digest was an unkeyed SHA-256 of the assistant's reply text — dictionary-attackable for short replies ("OK", "Done.") — and it crosses into the envelope and the CLI ledger. It is now an HMAC under the host privacy key. (#935, thanks @ozymandiashh)
+- Both changes alter dedup-key VALUES, so the affected providers carry a parse-version bump (`source-ref-fingerprint-v1`; copilot `…-dedup-key-hmac-v1`) that forces one re-parse and drops the old-shape keys from the warm session cache instead of re-ingesting the same records under two shapes. The six fingerprint providers additionally fold a digest of the privacy key into their env fingerprint, so a lost or rotated key also forces a clean re-parse. No totals change: no call is added, dropped, or repriced, and the daily cache needs no bump.
+
 ### Changed (CLI)
 - **`codeburn sync` re-keys device, span, and trace identity.** Device, span, and trace ids are now domain-separated HMAC digests under the per-install privacy key rather than bare hashes. This is a one-time identity re-key: a sync backend sees a new device identity after upgrade, and spans pushed before the upgrade won't correlate with spans pushed after it. `sync push` also now hard-aborts — instead of degrading to an ephemeral, unstable key — when the config dir is unwritable or the on-disk privacy-key file is corrupt.
 

packages/cli/src/providers/bridge.ts

@@ -1,5 +1,6 @@
 import type { DecodeContext } from '@codeburn/core'
 
+import { getHostPrivacyKey } from '../privacy-key.js'
 import type { Provider, ProbeRoot, SessionSource, SessionParser, ParsedProviderCall } from './types.js'
 
 // ── The dual-registry bridge ────────────────────────────────────────────────
@@ -81,10 +82,31 @@ export function createBridgedProvider<TRich>(spec: BridgedProviderSpec<TRich>):
         async *parse(): AsyncGenerator<ParsedProviderCall> {
           const records = await spec.readRecords(source)
           if (records === null) return
-          // The CLI holds the rich decode only; minimization / fingerprinting
-          // happens on the sync path, so an empty privacy key is correct here
-          // (the rich decoder never consumes it), matching claude/codex.
-          const context: DecodeContext = { privacyKey: '', providerId: spec.name, sourceRef: source.path }
+          // The host privacy key, threaded into the rich decode (D1). An empty
+          // key was correct when the bridge was written — minimization and
+          // fingerprinting happened later, on the sync path, so the rich
+          // decoder never consumed it. That is no longer true: the rich
+          // decoders now derive dedup keys (sourceRefFingerprint, copilot's
+          // JetBrains per-turn HMAC) and diagnostic details from the key, and
+          // dedupKey ships on the observation envelope. core's keyed
+          // primitives THROW on an empty key rather than degrade to an unkeyed
+          // digest, so the bridge has to supply the real one.
+          // getHostPrivacyKey() is per-install stable (persisted, like the
+          // optimize detectors use), so dedup keys stay stable across runs and
+          // the session-cache dedup semantics are unchanged.
+          //
+          // It degrades to a FRESH per-process key whenever the config dir
+          // (~/.config/codeburn) is unwritable OR the key file exists but is
+          // corrupt/unreadable — and that says nothing about whether the cache
+          // can persist: the cache lives in ~/.cache/codeburn (or
+          // CODEBURN_CACHE_DIR), a different directory, separately overridable.
+          // A writable cache plus an unreadable key file therefore means a new
+          // key every process. computeEnvFingerprint is what stops that from
+          // inflating totals: it folds a digest of the key into the fingerprint
+          // for every provider in KEY_DERIVED_PROVIDERS, so a rotated, lost, or
+          // per-process key rebuilds the section instead of re-ingesting the
+          // same records under keys the cache has never seen.
+          const context: DecodeContext = { privacyKey: getHostPrivacyKey(), providerId: spec.name, sourceRef: source.path }
           const { calls } = spec.decode({ records, context, seenKeys })
           for (const rich of calls) {
             yield spec.toProviderCall(rich)

packages/cli/src/session-cache.ts

@@ -4,6 +4,7 @@ import { createHash, randomBytes } from 'crypto'
 import { join } from 'path'
 import { homedir } from 'os'
 
+import { getHostPrivacyKey } from './privacy-key.js'
 import type { ToolCall } from './types.js'
 
 // ── Types ──────────────────────────────────────────────────────────────
@@ -234,6 +235,41 @@ export const DURABLE_PROVIDER_NAMES: ReadonlySet<string> = new Set(['copilot'])
 // needs no suffix: the cli-shutdown-cost-v1 bump below already forces its one
 // re-parse, which lands the flag too, and durable orphans now survive
 // fingerprint changes (the carry-forward in getOrCreateProviderSection).
+// Dedup-key hygiene (#933/#935): six providers now thread a FINGERPRINT of the
+// source path into their dedup keys instead of the raw path, and copilot's
+// JetBrains per-turn digest went from an unkeyed sha256 to a keyed HMAC. The
+// session cache seeds its dedup sets from the CACHED keys, so a pre-fix cache
+// keeps the old-shape keys and the same records re-ingest under the new shape
+// (double-count; and for the six, the raw path stays on disk forever). Each
+// entry/suffix below changes the provider's env fingerprint, which forces the
+// one-time re-parse that drops the old-shape keys.
+//
+// A parse version cannot see the OTHER input those keys depend on: the random
+// per-install privacy key. Rotate it, lose the file, or land on the ephemeral
+// fallback and the freshly derived keys stop matching the cached ones, so every
+// record re-ingests as new — silently, and for copilot (the sole durable
+// provider, whose union-merge never deletes) permanently. Every provider whose
+// dedup keys are derived from that key is listed HERE, explicitly, and
+// computeEnvFingerprint folds a digest of the key in for exactly this set.
+//
+// Listed explicitly rather than sniffed out of the parse-version string: the
+// first version of this used `parseVersion.includes('source-ref-fingerprint-v1')`
+// and silently missed copilot, whose keys are just as key-derived but whose
+// parse version says `dedup-key-hmac-v1`. A provider is in this set because of
+// what its decoder DOES, which no substring of a version label can know.
+// Add a provider here whenever its dedup key starts depending on the privacy
+// key — the rotation test in session-cache.test.ts covers one member of each
+// kind (source-ref and durable/copilot).
+export const KEY_DERIVED_PROVIDERS: ReadonlySet<string> = new Set([
+  'codebuff',
+  'zerostack',
+  'pi',
+  'omp',
+  'grok',
+  'lingtai-tui',
+  'copilot',
+])
+
 export const PROVIDER_PARSE_VERSIONS: Record<string, string> = {
   // rich-session-capture-v1: parse-time capture of per-turn gitBranch, per-call
   // LOC deltas / interruptions / userModified / toolErrors, and session-level
@@ -252,10 +288,14 @@ export const PROVIDER_PARSE_VERSIONS: Record<string, string> = {
   codex: 'mcp-attribution-v2-est-cost-rich-capture-v1-cross-provider-pr-v1',
   cursor: 'composer-anchored-crediting-v1-est-cost',
   'cursor-agent': 'workspaceless-transcript-v1',
-  copilot: 'cli-shutdown-cost-v1-skills',
-  grok: 'estimated-cost-v1',
+  copilot: 'cli-shutdown-cost-v1-skills-dedup-key-hmac-v1',
+  codebuff: 'source-ref-fingerprint-v1',
+  zerostack: 'source-ref-fingerprint-v1',
+  pi: 'source-ref-fingerprint-v1',
+  omp: 'source-ref-fingerprint-v1',
+  grok: 'estimated-cost-v1-source-ref-fingerprint-v1',
   hermes: 'reasoning-output-accounting-v1-est-cost',
-  'lingtai-tui': 'token-ledger-registry-activity-v3',
+  'lingtai-tui': 'token-ledger-registry-activity-v3-source-ref-fingerprint-v1',
   'ibm-bob': 'worktree-project-grouping-v1',
   kiro: 'ide-parsing-v1-est-cost',
   opencode: 'session-model-v1',
@@ -293,6 +333,15 @@ export function computeEnvFingerprint(provider: string): string {
   const parts = vars.map(v => `${v}=${process.env[v] ?? ''}`)
   const parseVersion = PROVIDER_PARSE_VERSIONS[provider]
   if (parseVersion) parts.push(`parser=${parseVersion}`)
+  // Providers whose dedup keys are HMAC-derived from the per-install privacy
+  // key must re-parse when that key changes; a digest of the key (never the key
+  // itself) rides the fingerprint for exactly those. Deliberately NOT nested
+  // under `if (parseVersion)`: the key dependency comes from the decoder, not
+  // from having a parse-version entry, and a key-derived provider that ever
+  // loses its entry must still re-parse on rotation.
+  if (KEY_DERIVED_PROVIDERS.has(provider)) {
+    parts.push(`privacy-key=${createHash('sha256').update(getHostPrivacyKey()).digest('hex').slice(0, 16)}`)
+  }
   return createHash('sha256').update(parts.join('\0')).digest('hex').slice(0, 16)
 }