Add DeepSeek cost-runtime policy for Runtime v2/AiSdkFlow #19

Summary

Maka's Runtime v2 architecture already has the right place to add DeepSeek cost optimization: the AgentFlow / AiSdkFlow split. We should integrate DeepSeek as a cost-aware AiSdkFlow policy selected by model/provider, not replace Maka's runtime or introduce a separate agent framework.

This issue proposes a CostRuntime layer made of focused modules:

• flow policy selection for cost-sensitive models
• stable prefix shape and prefix-change diagnostics
• normalized DeepSeek/OpenAI-compatible usage and pricing
• provider adapter hardening for reasoning, finish reasons, usage fields, and retry semantics
• canonical tool schemas and tool surface economy
• context, memory, and history economy
• validation tests and live DeepSeek probes

The goal is to make DeepSeek cheap for ordinary follow-up turns by keeping provider-visible prefix bytes stable, pricing cached and fresh input correctly, and making cache misses explainable.

Product insight

Runtime v2's AgentFlow / AiSdkFlow split is exactly where per-model or per-provider flows can live. DeepSeek should be a policy applied at that seam:

• RuntimeRunner continues to run a selected AgentFlow.
• The default AiSdkFlow remains the AI SDK implementation.
• A flow resolver can choose normal AiSdkFlow or cost-aware AiSdkFlow configuration for DeepSeek/cost-sensitive models.
• Provider-specific wire behavior remains in ModelAdapter / provider factory code.

This keeps the architecture incremental and avoids turning cost optimization into a sidecar, separate backend, or replacement framework.

Proposed architecture

Add a small resolver near RuntimeKernel.startTurn():

AgentFlowResolver.resolve({
  sessionId,
  backend,
  connection,
  providerType,
  modelId,
  runtimePolicy,
}): AgentFlow

Default behavior returns the existing AiSdkFlow. For DeepSeek or catalog-marked cost-sensitive models, return AiSdkFlow with a CostRuntimePolicy, or a small CostAwareAiSdkFlow adapter while provider-visible request assembly still lives in AiSdkBackend.

Add CostRuntime as modules, not one monolith:

• CostFlowPolicy: selects cost-aware behavior and records policy version.
• PrefixShape: hashes provider-visible system/tool/history/provider-option shape and reports local change reasons.
• PromptComposer: splits durable prefix from volatile turn-tail material.
• NormalizedProviderUsage: preserves cache-hit input, cache-miss input, output, reasoning, finish reason, and total tokens.
• CostPricing: prices fresh input, cached input, cache writes, and output separately.
• ProviderAdapterHardening: keeps DeepSeek reasoning/usage/retry semantics at the provider boundary.
• ToolSchemaRegistry: canonicalizes and sorts tool schemas before streamText.
• CapabilityEconomy: avoids paying every turn for optional or disabled tool families.
• HistoryMemoryEconomy: prunes/archives stale tool outputs and uses indexes/retrieval instead of prompt stuffing.
• CostObservability: exposes cached vs fresh input, reasoning/output, cost, and prefix-change reason in runtime events and telemetry.

Why this matters

DeepSeek prompt caching is useful only when the start of the provider request is byte-stable across turns. Maka currently rebuilds the system prompt and tool map each call, includes volatile prompt fragments such as date/git/cwd, advertises a broad tool surface, and can replay old tool outputs into history. Some of this is correct product behavior, but today it is not tracked as cost-shaping request state.

We need to make the provider-visible shape explicit:

• Stable prompt and tool bytes should remain stable.
• Intentional resets should be visible.
• Accidental churn should be testable.
• Provider usage should distinguish fresh input from cached input.
• Pricing should not double-charge cache-hit prompt tokens.

Required usage contract

Normalize provider usage at the adapter boundary:

interface NormalizedProviderUsage {
  inputTokens: number;
  outputTokens: number;
  totalTokens: number;
  cacheHitInputTokens: number;
  cacheMissInputTokens: number;
  cacheWriteInputTokens: number;
  reasoningTokens: number;
  finishReason?: string;
  stopReason?: string;
  usageSource: 'ai-sdk' | 'providerMetadata' | 'derived';
}

Rules:

• DeepSeek prompt_cache_hit_tokens => cacheHitInputTokens.
• DeepSeek prompt_cache_miss_tokens => cacheMissInputTokens.
• OpenAI-compatible nested cached-token fields => cacheHitInputTokens.
• If misses are absent, derive max(0, inputTokens - cacheHitInputTokens - cacheWriteInputTokens).
• If cache data is absent, treat all input as fresh input.
• Preserve raw finish reason separately from Maka's mapped stop reason.

Pricing must use fresh input for full input price:

fresh input  = cacheMissInputTokens * input rate
cache read   = cacheHitInputTokens * cache-read rate
cache write  = cacheWriteInputTokens * cache-write rate
output       = outputTokens * output rate

Prefix diagnostics

Add a PrefixShape object computed before model invocation:

interface PrefixShape {
  version: 1;
  providerType: string;
  connectionSlug?: string;
  modelId: string;
  systemHash: string;
  toolSchemaHash: string;
  toolNames: string[];
  activeToolsHash: string;
  providerOptionsHash: string;
  memoryIndexHash: string | null;
  workspaceInstructionsHash: string | null;
  historyRewriteVersion: number;
  prefixHash: string;
}

Attach previous/current prefix hashes and a reason to usage events and telemetry:

• first_turn
• system_prompt_changed
• tool_schema_changed
• memory_index_changed
• workspace_instructions_changed
• provider_options_changed
• model_or_provider_changed
• history_projection_changed
• history_rewrite
• provider_cache_expired_or_unknown
• unknown

This does not need to prove DeepSeek's internal cache key. It needs to explain local request-shape churn next to provider-reported cache hit/miss tokens.

Tool surface economy

Tool schemas are paid prompt surface. Add a ToolSchemaRegistry that:

• canonicalizes provider-visible schemas
• emits tools in stable sorted order
• hashes tool schemas and active tool names
• estimates schema token/byte cost
• records schema change reasons

Then reduce default schema surface:

• Keep core local tools available by default.
• Do not advertise disabled or credentialless tools as executable schemas.
• Move optional families such as WebSearch, Office, Rive, Explore, skills body loading, MCP, and future connectors behind an explicit capability profile or connector.
• When a capability is enabled, mark the next request as an intentional tool-schema cache reset.

Context, memory, and history economy

Add a budgeter before provider message materialization:

• Preserve recent tail turns.
• Archive stale large tool outputs.
• Replace old terminal/web/explore/Office/Rive outputs with deterministic placeholders.
• Keep artifact refs and byte counts for recovery.
• Retrieve full archived output only on demand.
• Run deterministic prune/archive before LLM summarization.
• Treat compaction as a named cache-reset event with historyRewriteVersion.

Memory and workspace instructions should move toward indexes:

• Durable prefix contains compact memory/workspace indexes and hashes.
• Full memory/workspace bodies are loaded when relevant.
• Mid-session memory writes become one-turn notices or controlled index updates, not silent system-prompt churn.

Phased implementation

1. Add flow resolver seam and cost-aware policy selection.
2. Normalize usage and fix pricing semantics.
3. Add prefix diagnostics around the current request builder.
4. Canonicalize tool schemas and add capability economy.
5. Split durable prefix from volatile turn-tail prompt material.
6. Harden DeepSeek provider adapter behavior.
7. Add history/memory budgeter, prune-before-compact, and retrieval.
8. Expose cached/fresh/reasoning/cost/prefix reason in UI and telemetry.
9. Add env-gated live DeepSeek probes.

Acceptance criteria

• DeepSeek/cost-sensitive models can select cost-aware AiSdkFlow policy without changing non-DeepSeek behavior.
• Usage normalization preserves cache hits, cache misses, reasoning tokens, output tokens, total tokens, and finish reason.
• Cost calculation uses cache-miss/fresh input for full input price and does not double-charge cached tokens.
• Runtime/session/telemetry usage surfaces can carry cache miss, reasoning, finish reason, cost breakdown, and prefix reason.
• Prefix diagnostics record stable hashes and targeted change reasons.
• Tool schemas are canonicalized and hashed in stable order.
• Disabled optional tools are not advertised as executable schemas.
• History compaction/pruning increments a rewrite version and records a cache-reset reason.
• Tests cover DeepSeek usage fixtures, pricing, prefix hashing, tool schema canonicalization, history pruning, telemetry persistence, and UI projection.
• Live DeepSeek probes are available behind credentials/env flags.

Non-goals

• Do not replace Runtime v2 or AiSdkFlow.
• Do not introduce a separate DeepSeek-only backend unless the provider requires a different stepping engine.
• Do not port another framework's planner/subagent stack for this issue.
• Do not hard-code DeepSeek pricing as permanent truth.
• Do not claim ROI until live provider probes validate field mapping and cache behavior.

validation_plan

Unit tests

• model-adapter usage normalization:

  • DeepSeek top-level prompt_cache_hit_tokens and prompt_cache_miss_tokens.
  • OpenAI-compatible nested cached-token fields.
  • reasoning token extraction.
  • raw finish reason preservation.
  • derived cache miss fallback and negative clamp.

• telemetry cost:

  • cached input is charged at cache-read rate only.
  • fresh/cache-miss input is charged at full input rate.
  • output is charged at output rate.
  • cache-write remains separate for providers that support it.

• prefix shape:

  • identical provider-visible shape gives identical prefixHash.
  • system prompt, tool schema, provider options, model/provider, memory index, workspace index, replay gate, and history rewrite changes produce targeted reasons.

• tool schema registry:

  • registration order does not affect canonical hash.
  • equivalent schema objects serialize identically.
  • optional tool add/remove changes hash with tool_schema_changed.
  • invalid/repair tool inclusion rule is deterministic.

• prompt composer:

  • volatile date/git/cwd facts do not silently mutate durable prefix.
  • memory/workspace/skills indexes are stable and separately hashed.
  • memory writes create a controlled one-turn notice or index update.

• history economy:

  • stale large tool outputs are archived and replaced with placeholders.
  • recent paired tool calls/results remain valid.
  • history rewrite version increments on model-visible rewrite.
  • archived details are recoverable by ref.

Integration tests

• AiSdkBackend emits and persists token usage with cache hit/read, cache miss/fresh, reasoning, finish reason, and prefix reason.
• recordLlmCall receives normalized usage, cost breakdown, finish reason, and prefix diagnostics.
• RuntimeEvent replay modes produce distinct prefix replay gates.
• Model/provider switch between turns records a prefix reset reason.
• Optional capability enablement records a tool-schema reset on the next request.

UI and storage tests

• Telemetry rows round-trip cache miss, reasoning, finish reason, prefix reason, and cost breakdown.
• Usage summaries distinguish cache hit, cache miss, cache write, output, and reasoning.
• Chat/session usage chips can display fresh input vs cached input without confusing cache miss with cache creation.

Live DeepSeek probes

Add env-gated probes that run only with credentials:

• Stable two-turn prompt should produce cache-hit tokens on the second turn if provider cache behavior applies.
• Tool schema change should reduce or reset cache hits and produce local tool_schema_changed.
• System prompt change should produce local system_prompt_changed.
• History compaction/rewrite should produce local history_rewrite.
• DeepSeek usage fields should map into normalized usage.
• Early retry behavior should be validated only before visible output.

open_questions

• How much provider-visible request assembly should move from AiSdkBackend into AiSdkFlow as part of this work, versus passing cost policy down first and refactoring later?
• Where should PrefixShape be persisted long-term: runtime events, LLM telemetry rows, run trace data, or a dedicated side table?
• Should the first implementation expose optional capability economy to users, or keep it as an internal policy profile until the UI is ready?
• What is the minimum useful durable prefix split for v1: hash-only diagnostics around the current prompt, or an immediate PromptComposer refactor?
• Which tool families are core by default for Maka, and which should become optional connectors?
• How should memory/workspace instruction indexing preserve user expectations when full bodies are no longer injected every turn?
• What token estimator is accurate enough for tool schema and prompt fragment budgets before provider usage returns?
• What DeepSeek cache TTL and cache-key fields should live validation assume, if any?
• How should pricing defaults be refreshed or overridden so built-in rates do not become stale?