Repository navigation
fix(copilot): prefer lastEventTimestamp for stampless shutdown calls - #1054
Conversation
Two journals for one session both emitted :1 and the shared seenKeys set dropped the second file's first leg. Identity is now (session, model, shutdown timestamp). Bump the Copilot parse fingerprint so present sources re-parse; durable orphans stay.
Extra High HOLD on #1054: timestamp-only keys still collided across journals, and the provider-wide rollup-ts-v1 bump dropped present OTel sources, erasing conversations already pruned from the DB. Identity is now (session, model, timestamp, journal basename). Legacy :n keys migrate on that JSONL file only.
|
Reviewed in depth against current The collision being fixed is unreachable through discovery. Only CLI jsonl sources emit Meanwhile the new key regresses a reachable case. Two more blockers:
And with #946 (either landing order): #946 reorders the stamp fallback to What does hold up (fixture-verified): legacy keys with real per-leg stamps migrate cleanly and stably; orphaned legacy rollups survive untouched; Recommendation:
Also noted for whoever lands anything here: |
Maintainer review on #1054: two-journal collision is unreachable through discovery. Keying by timestamp+journal collapsed a 3-leg stampless journal onto one row, and the :n strip re-sent ledger keys. Keep occurrence keys. Take lastEventTimestamp before sessionStartTime for the call timestamp only. No migration.
ozymandiashh
left a comment
There was a problem hiding this comment.
Reviewed the current head and effective merge. The fallback changes only the timestamp while preserving the :n identity keys, so cache and sync continuity remain intact. The realistic stampless JSONL repro retained all calls/tokens; 106/106 targeted tests, typecheck, and all CI checks pass. No blocking issue found.
Resolves the conflicts getagentseal#946 accumulated while it was in validation. Eight files conflicted; the session-store accounting is unchanged. src/daily-cache.ts — version collision. This PR minted 25 when main was at 24; getagentseal#1056 (`codex-auto-review` pricing) then spent 25 on main. The bump moves to 26/MIN 26 and daily-cache.v26.json, with main's full comment ladder kept as the foundation and this PR's paragraph rewritten to name 26 and record the collision. PENDING_REDERIVE_PROVIDERS and the B1 migration semantics from b6481c1 carry over intact, retargeted at 26. src/models.ts, src/parser.ts, src/audit-report.ts, src/models-report.ts — getagentseal#1075/getagentseal#1078 replaced the per-site "reasoning is already inside output" tests with billableOutputTokens() and REASONING_INCLUDED_IN_OUTPUT. This PR had added copilot to that case at three sites independently. Union: all three sites take main's helper call verbatim, and copilot joins claude and codex in the set — same accounting this PR shipped, now through main's single source of truth. It also reaches parser.ts activeGeneratedTokens (a fourth site, from getagentseal#1079), which is the same correction: a copilot supplementary call carries reasoning with output 0, so counting it as generated repeats the per-turn output. The audit legend already said so on this side. src/providers/copilot.ts — comment-only. getagentseal#1054's lastEventTimestamp-first shutdown fallback was derived from this branch, so the code was already identical on both sides: the `shutdownTimestamp` expression and the `copilot:<sid>:shutdown:<model>:<n>` key are byte-for-byte main's. Both rationales are kept (leg-collapse on date, and residual anchoring). src/session-cache.ts — PROVIDER_PARSE_VERSIONS.copilot takes this PR's `-session-store-v3` suffix; main's getagentseal#1051 note about why a fingerprint change is expensive is kept above it. Codex keeps main's getagentseal#1092 suffix chain untouched. src/main.ts — getagentseal#1067 deleted the unreachable live dailyMap fallback that this PR had taught behavioral weight. Main's deletion wins; the now-unused isBehavioralTurn import goes with it. tests/parser.test.ts — import union. Also: scripts/upgrade-path/run.mjs NEW_DAILY_CACHE -> daily-cache.v26.json, CODEBURN_COPILOT_SESSION_STORE_DB added to the getagentseal#1064 env-isolation CLEARED list, and the CHANGELOG entry's stale "v21" corrected to v26. Verified: tsc clean; 3132 tests pass across 223 files; test:locks 26/26; verify:upgrade PASSED, re-deriving daily-cache.v26.json and holding durable copilot history across the bump. getagentseal#1054's regression ("keeps three stampless shutdown legs as :n keys with lastEventTimestamp") passes on the merged tree. Real-corpus A/B against origin/main over 2026-07-01..2026-08-22: codex, grok, kimicode and opencode byte-identical in export, audit and models; claude drifts only monotonically with run order (a live session writing transcripts, confirmed by interleaving four runs). This machine has no copilot data, so the copilot recovery semantics rest on the suites and the upgrade-path corpus.
Problem
A Copilot CLI journal with several
session.shutdownlegs that carrysessionStartTimebut no per-leg timestamp billed every leg onto one date. Keying those rollups by timestamp (or timestamp + journal basename) collapsed the same journal onto one row.Closes nothing. #1051's two-journal collision is unreachable through discovery (
discoverJsonlSessionsonly yields<sid>/events.jsonl).Root cause
Shutdown timestamp fallback was
event.timestamp || sessionStartTime || lastEventTimestamp.sessionStartTimeis identical for every leg of a resumed session. Putting that stamp in the dedup key made three stampless legs share one key. A:n→ journal-key migration then stripped cached occurrence keys and re-sent them under a new name (ledger double-count).Change
copilot:<sid>:shutdown:<model>:<n>.event.timestamp || lastEventTimestamp || sessionStartTime.:nstrip, and the two-journal fixtures.User impact
Stampless multi-leg CLI journals keep every leg's input/cache tokens and land on the last stamped event's time, not session start. Already-synced
:nkeys are unchanged.Preservation
No LiteLLM rates. No store/OTel/JetBrains/transcript path changes. No
CACHE_VERSION/ fingerprint / ledger rewrite. Per-turn keys unchanged. #946 / #868 / #916 / #998 untouched.Testing
tests/providers/copilot.test.ts+tests/parser.test.ts):1/:2/:3, timestamps = lastEventTimestamp notsessionStartTime, token sum 35002026-07-15T07:54:00.274Zvs last event)tsc --noEmitclean