Move the inference engines into nlp-engine: the interface, the local bridge and the remote client, with one prompt dump #3744

Overview

The code that sends a conversation to a model lives in two packages. The local engine is in nlp-engine. The engine interface, the bridge to the local engine, and the client for remote OpenAI-compatible endpoints are in agent, beside the chat loop. The crate boundaries now say all of it belongs in nlp-engine. This issue moves it, with no change in behaviour.

Problem Statement

• "How a conversation reaches a model" is one concern with two homes. The remote client went into agent because it needs an HTTP library and nlp-engine had none.
• The prompt dump was written twice, once per package, for the same reason.
• The interface both engines implement (ChatInferenceEngine) is defined in agent, so nlp-engine cannot name it.

Proposed Solution

Move into nlp-engine:

• ChatInferenceEngine and the request and response types it uses (InferenceRequest, StreamingChunk, ChatMessage, ToolDefinition, InferenceUsage, InferenceError, ChatModelSpec), from agent_types.rs;
• the local bridge, local_agent/inference.rs;
• the remote client and its discovery, local_agent/openai_compat_inference.rs and openai_compat_discovery.rs;
• the two prompt dumps, merged into one that tags the engine.

Stay in agent: the loop, routing and the routing probe, tools, prompt templates, response processing. model_manager.rs stays unless it can move without nlp-engine importing from the workspace: it chooses and holds the engine from settings.

nlp-engine gains the HTTP client dependency and remains a workspace leaf.

Documentation

• crate-boundaries.md - nlp-engine and agent responsibilities, "Local and remote engines live together", the dependency DAG and decision tree rule 7
• ADR-034 - the provider modes the two engines serve
• prompt-dump.md - the dump's record format, which the merged dump keeps or the page is updated
• Updates: development/prompt-dump.md if the record format changes; any doc that names the moved files

Architecture Impact

Implements the boundary now stated in architecture/crate-boundaries.md. No ADR: the page is the record of the decision.

Acceptance Criteria

• agent contains no inference engine: no type that implements sending a conversation to a model, local or remote
• nlp-engine imports nothing from the workspace
• One prompt dump serves both engines: one module, one writer and one sequence counter, so records from a local turn and a remote turn form a single numbered timeline in one file; the per-engine copies and their separate counters are deleted
• A record has the same shape whichever engine wrote it, tagged with the engine; what differs is only what was put on the wire (the rendered prompt text for the local engine, the request body for a remote one)
• bun run prompt:dump reads a file that mixes local and remote turns and shows them in order
• Each type moved has one definition; nothing is re-exported from its old location to keep old imports working
• No behaviour changes: the agent tests, the routing and decision evals on the scenarios touched, and a manual local turn and remote turn give the same results as before
• model_manager.rs is moved or left, with the reason written in the pull request
• Code passes bun run quality:fix and bun run test:changed

Technical Specifications

Reference Files

• packages/agent/src/agent_types.rs - the interface and its types (about 76 uses of agent_types:: in agent)
• packages/agent/src/local_agent/inference.rs, openai_compat_inference.rs, openai_compat_discovery.rs, openai_compat_prompt_dump.rs, model_manager.rs
• packages/nlp-engine/src/chat/mod.rs, prompt_dump.rs, types.rs
• packages/agent/Cargo.toml, packages/nlp-engine/Cargo.toml
• packages/agent/tests/it/ - the live OpenAI-compatible tests, which move with the client

Implementation Details

• A mechanical move: do it in one pull request with no other change, so the diff is reviewable as a move.
• Today both dumps already write to the same file, under the same setting, with the same record kinds (prompt, response). They are two copies of the code with two counters, so a seq repeats across engines and only the engine tag tells records apart. One writer fixes that.
• nlp-engine builds llama.cpp behind a feature. Check that the remote client compiles with that feature off, so a build with no local engine can still reach a remote model.
• Do this when no other open work is changing agent_loop.rs or the engine files, to avoid a rebase across a large move.

Related Issues

• Reconsider the nlp-engine/agent crate boundary for LLM connectivity code #1978 (the decision this carries out)

Non-Goals

• Any change to routing, the loop, tools or prompts
• A new kind of remote provider