Skip to content

fix(runtime): preserve context history for prompt caching - #6177

Merged
xxhZs merged 2 commits into
mainfrom
fix/durable-runtime-context-cache
Oct 20, 2026
Merged

xxhZs merged 2 commits into
mainfrom
fix/durable-runtime-context-cache

Conversation

@xxhZs

@xxhZs xxhZs commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Time and Inbox context used to be replaced at the end of each model request. Updating the clock therefore changed an existing request prefix, reducing cache reuse even when the conversation had only grown.

Persist named runtime context as internal events and append a new value only when it changes. Historical values remain unchanged, retries reuse the same snapshot, and replay preserves event order. Split the matter clock from its Inbox so observation-only revision changes do not resend the Inbox. Context removed by compaction is refreshed afterward; internal context remains hidden from the chat transcript.

Validation on current main:

  • 700 targeted tests passed: runtime history, context, retries, replay, compaction and recovery (662), core event decoding (29), Host composition/steering (7), and MatterController (2).
  • Core, runtime and runtime-host builds; UI and desktop type checks passed. Staged lint, formatting, license headers and diff checks passed.
  • Prior isolated real-model validation used GPT-6.1 Sol at medium effort with the modified runtime and actual MatterStore/Controller. A running Inbox amendment and a steering message both reached the next model request. Four calls retained the full previous input prefix; cache hits were 0%, 88.9%, 96.7%, and 96.6%. One earlier repeat still had a cache miss despite an unchanged prefix, so this does not guarantee provider cache availability or resolve every timeout.

Real-model tools were read-only synthetic fixtures outside the repository, not a production business-workflow test. An earlier broader Host test process did not exit naturally and is not counted as passing; the relevant targeted Host tests completed successfully.

Review follow-up:

  • Adapt Host environment, Agent Graph/Swarm, and cold WorkHub recovery fixtures to persisted runtime context history. Production behavior and original assertions remain unchanged.
  • The five review-reported regressions pass; runtime-host type checking and changed-file formatting checks pass. Final Host dist suite: 2,830 passed, 13 skipped, 1 failed. The remaining failure is prompt suggestions ask Kimi K3 for its lowest effort instead of its default thinking (missing thinking: {type: adaptive}); it passed in an earlier isolated run but failed in the full suite, so the full suite is not reported as passing. Missing executor/filesystem-worker build artifacts were corrected before this final run. Separately, assistant suites passed 83 memory-network and 24 initiative tests; Matters passed 79/80, with the queued-wake timeout passing on isolated rerun. These additional failures were not hidden or relaxed. No paid-model tests or production deployment were performed for this follow-up.

Generative-tool attribution: OpenAI Codex made substantive implementation, test, and documentation contributions to this PR. The follow-up commit includes Generated-by: OpenAI Codex; preserve this trailer in the final squash commit.

@github-actions github-actions Bot added the effort/L Under 1000 readable lines label Oct 19, 2026

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review notice: This comment was posted by an automated review agent. It is not an independent human review and does not replace one.

I reviewed all 18 changed files at dc4cba9dd80101ff7a4e5de65f5e38f36f865895, including durable context events, model replay, history compaction and overflow recovery, and the assistant/Matter context sources. The change appends named context updates after existing history and resends current values when compaction removes their previous snapshots.

P2 — Update the Host integration fixtures for persisted runtime contexts (packages/runtime/src/runtime-context.ts:45). CI run 39285344761 fails with four Host test failures and one cancellation. I reproduced all five affected cases unchanged; substituting only the parent implementation of AiSdkTurn makes all five pass. The existing runtimeEnvironment() helper still expects the unwrapped environment prefix, and the Agent Graph/Swarm provider scenarios fail to exclude the wrapped environment messages. Updating those test helpers alone makes the four corresponding cases pass without changing production code. The cold WorkHub recovery scenario also treats a historical topic-association instruction as current. Please adapt these fixtures to the new context history and rerun the full Host suite. These results establish a PR-caused CI regression; they do not establish a real-provider reasoning failure.

Validation: the normal clean root test build, all workspace type checks, lint, formatting, ASF headers, and the protocol epoch guard passed. Eleven original Core/Runtime test files passed 578 tests, and the Matter controller passed two. Two additional controlled compaction cases covered immediate and slow event consumers, removal of old context snapshots, reinsertion of current values, and preservation of the user anchor. Actual SDK serialization of four captured runtime prompts passed 12 content-prefix comparisons across OpenAI Responses, OpenAI Chat, Anthropic and Google; private context markers did not reach the provider payloads. These checks do not measure live cache hit rates, billing, or paid-provider behavior.

P3 — Clarify generative-tool attribution. The PR describes model-assisted validation but does not explicitly state whether generative tools made a substantive contribution. Please add that declaration; if they produced substantive content, preserve the required Generated-by: trailer in the final commit.

The protocol guard reports no protocol change against main at epoch 225. A trial merge with main b0ed5707ff3dc645c66253ed93e50936e3f9d45d is conflict-free; I did not test that merged tree. The reviewed head remains open and unchanged, and its test check is failing. The CI regression needs resolution before this is ready to merge.

}
const changes: { name: string; text: string }[] = [];
for (const context of contexts) {
const text = `Runtime context (${context.name}; host metadata, not a user request; later updates supersede earlier values):\n${context.text}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Update the Host integration fixtures for the new wrapped context history. The existing runtimeEnvironment() helper only recognizes the old plain prefix, while the Agent Graph/Swarm provider fixtures now mistake wrapped environment snapshots for user requests. Exact-head CI 39285344761 and five unchanged local Host cases fail (four failures, one cancellation); swapping only AiSdkTurn back to its parent makes all five pass, and adapting only the four corresponding fixture helpers makes those four pass with current production code. The cold WorkHub scenario also needs to distinguish historical topic instructions. Please update the fixtures and rerun the full Host suite.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review notice: This comment was posted by an automated review agent. It is not an independent human review and does not replace one.

I reviewed dc4cba9dd80101ff7a4e5de65f5e38f36f865895..07d61519e16e1293962726c7c9d87e033b08be16. This is one appended commit, with no rebase: only two Host test-fixture files change, and the production implementation is unchanged. runtimeEnvironment() now unwraps and reads the latest persisted environment snapshot; the Agent Graph/Swarm fixture excludes wrapped environment metadata; the cold WorkHub provider fixture correlates association calls and results after the current user message rather than repeatedly reacting to an instruction retained in history (packages/runtime-host/src/__tests__/execution-model-composition.test.ts:6254, :6265, packages/runtime-host/src/__tests__/fixtures/agent-graph-provider-scenario.ts:336). The existing execution, workspace and recovery assertions remain in place.

The previous P2 is fixed; no new P0–P2 findings. All five previously failing Host cases now pass: cold WorkHub recovery, canonical execution, Swarm activity, Agent Graph supervision and the durable child tool ceiling. Restoring only the two old test modules through a loader, while retaining current production code, reproduces the previous result: four failures and one cancellation. This verifies that the follow-up repairs the fixture assumptions rather than changing production behavior to satisfy them. The earlier contribution-disclosure P3 is also addressed by the PR description and the follow-up commit's Codex attribution.

Another 159 tests passed across runtime context history/replay, model history, mid-turn compaction, event mapping and core event decoding. The normal clean root test build, all workspace type checks, lint, formatting, ASF headers, protocol epoch guard and diff checks passed. Exact-head CI run 39385140776 completed successfully with a successful test check. Its result supersedes the older failing run cited in the previous review.

No protocol files changed in this follow-up; the guard reports epoch 225. A trial merge with current main e66a655e4f262c68205bbda6e76613311db2b3c2 had no conflicts, but that merged tree was not built or tested. I did not repeat the author's paid-model cache-hit measurements, run the entire repository suite locally, or verify production billing/cache availability. Within this incremental scope, I found no remaining correctness blocker; the merge decision remains with the maintainers.

@xxhZs
xxhZs merged commit dc4515c into main Oct 20, 2026
1 check passed
@xxhZs
xxhZs deleted the fix/durable-runtime-context-cache branch October 20, 2026 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/L Under 1000 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants