feat(runtime): automatic context compaction policy #577

Context

#575 wires manual /compact (TUI + desktop) to an LLM history summarizer, so a manual compact now asks the model to write a sectioned summary of the conversation instead of producing a deterministic excerpt fold. The runtime already had the summarize extension point (PersistHistoryCompactBlocksDeps.summarize); that PR just connects it for CLI and desktop.

This issue covers the remaining decisions #575 deliberately left out — all of which concern the automatic compaction path (trigger when the context is about to fill up) and its policy.

Problem

The automatic path exists mechanically (the runtime folds history on send when the context budget trips), but today it is effectively off by default:

• buildHistoryCompactPolicy returns undefined unless MAKA_CONTEXT_HISTORY_COMPACT=ON;
• even when enabled, mode defaults to lookup (read-only: reuse blocks a prior manual /compact produced, do not generate new ones);
• the thresholds (highWaterRatio, tailEstimatedTokens, minRecentTurns, ...) have no built-in defaults — they are entirely env-driven.

So in practice compaction only fires when a user types /compact. For long sessions and unattended agents that is not enough.

Background: where the tokens actually are

Two real session pools were measured to ground the policy (per-turn size ≈ chars/4):

Two takeaways:

1. Desktop daily sessions are short and small — they rarely fill the window, so they rarely trip compaction at all.
2. Autonomous sessions are also short (1–2 turns) but each turn is huge, because one turn runs dozens of tool calls whose outputs accumulate. The autonomous token problem is intra-turn (tool-output bloat), which is what active-tool-result-prune / active-full-compact already address — not the cross-turn history compaction this issue is about.

Cross-turn compaction serves the long multi-turn session case, which is real but under-represented in current data. The policy still needs sensible defaults so that when such sessions occur, compaction behaves well.

Proposal

Four decisions, each with rationale.

1. Auto-trigger: default on, write mode

• Enable automatic compaction by default (drop the MAKA_CONTEXT_HISTORY_COMPACT=ON gate, or default it on).
• Default mode to read_write (produce new summary blocks, not just reuse manual ones).
• Trigger threshold: compact when estimated context reaches contextWindow − reserve, with reserve = 16K tokens (room for a long response plus a few tool calls).

An absolute reserve instead of a ratio: the room the model needs to keep working is roughly constant regardless of window size. A ratio undersizes small windows (32K × 20% = 6.4K, too little) and oversizes large ones (200K × 20% = 40K, wasteful). An absolute reserve also adapts naturally — small windows trip earlier, large windows later. pi uses the same scheme (contextWindow − reserveTokens, reserve 16384).

Auto-trigger is not primarily aimed at autonomous (whose problem is intra-turn). It is for long multi-turn sessions where nobody is watching. A half-auto "ask the user before compacting" prompt is explicitly rejected: in unattended / long-running work it would stall the agent waiting for a confirmation that never arrives. When auto-compaction does run, it surfaces a non-blocking notice (e.g. "context compacted") — the user is informed, not blocked.

2. Summarizer model: reuse the session's model

• Summarize with the same model the session already uses.
• Do not add a summarizerModel config now.

The summarizer's input is exactly the history the model has already seen, so prompt-cache hits make the extra call cheaper than it looks. Switching to a cheaper model would discard that cache and re-process the whole history. A cheaper summarizer is a real optimization for cost-sensitive autonomous runs, but it is premature until data shows the cost matters.

3. Tail policy: turn-first with a token cap

• Default to keeping the most recent 3 turns verbatim (turns stay whole, so tool-call / result pairs are never split).
• Tail cap: 16K tokens.
• If the last 3 turns exceed the cap, drop the oldest turn and recompute, until it fits or only one turn remains.
• If a single turn still exceeds the cap, truncate to the most recent single step (one tool-call / result pair), kept whole. Never go below one step.

Turn-first beats token-first because cutting mid-turn can sever a tool call from its result and confuse the model. The cap only exists to bound the autonomous case where one turn is enormous (p90 37K); for normal turns it never trips.

4. semantic-compact: leave as-is

semantic-compact (single-turn, LLM, structured task-state JSON) stays, unchanged in name and behavior. It addresses the intra-turn case that cross-turn compaction does not, which the data shows is exactly the autonomous bottleneck. It is orthogonal to the cross-turn summary; renaming or merging it is out of scope.

Non-goals

• Changing active-tool-result-prune / active-full-compact (the intra-turn machinery).
• A summarizerModel config.
• A half-auto "ask before compact" prompt.
• Renaming or merging semantic-compact.
• Tailoring compaction specifically for benchmark runs.

Open questions

1. Is default-on auto-compaction acceptable for all surfaces (desktop, TUI, headless), or should headless / benchmark opt out?
2. Is 16K the right reserve, or should it scale with the model's max output?

References

• feat: wire manual /compact to LLM history summarizer #575 — manual /compact LLM summarizer (landed)
• pi: packages/agent/src/harness/compaction/compaction.ts (absolute reserve + token tail)
• opencode: packages/opencode/src/session/compaction.ts (turn tail + truncation)
• claude-code (reverse-engineered): three-tier compaction — MicroCompact for intra-turn, API summary for cross-turn