diff --git a/CLAUDE.md b/CLAUDE.md index ec54be578..5a90a67fc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -340,7 +340,7 @@ operator-facing TUI over the session API, and the component the root README lead with. See [`cmd/agentop/README.md`](cmd/agentop/README.md) for flags and keybindings. **Plugins are all opt-in.** Each one lives in a `cmd/*/plugins_.go` file -gated by `//go:build include_plugin_` (15 such files in +gated by `//go:build include_plugin_` (16 such files in `cmd/cortex/`); `main.go` imports no plugin package directly, so a build with no `-tags` registers no plugins at all and rejects every config it is handed. Tag sets come from [`scripts/profile-tags`](scripts/profile-tags/), one profile per @@ -581,6 +581,7 @@ Every event on `/v1/sessions/{id}` and `/v1/events` carries: open and a close saying 200. An opaque CONNECT that could not be dialed reports the same situation one layer down as `tunnelReason: "dial-failed"` with `error.kind: "dial_failed"`. +- `requestedHost` — present only when a plugin redirected the request (`pctx.Redirect`): the host the client asked for. `host` is where the request actually went, and usage, the cost ledger and pricing all follow `host`. agentop's detail pane shows both on a `redirected:` line. - `tunnel`, `tunnelReason`, `bytesUp`, `bytesDown` — an opaque CONNECT (or transparent-redirect) tunnel records two rows sharing a `requestId`: the open (`phase: "request"`, `tunnelReason` saying why the bytes stayed opaque) and, when the tunnel ends, the close (`phase: "response"`). The close carries the CONNECT's own `statusCode` — 200, or 502 with the dial error in `error` when the destination could not be reached (`tunnelReason: "dial-failed"`) — plus `durationMs` for how long the tunnel stayed open and the bytes it carried each way (up = client to destination). It is not the destination's status: that travels inside the client's end-to-end TLS. A bridged tunnel's open is recorded with its first decrypted request — in that request's session, directly before it, stamped with its time — and records no close, because the request carries its own response; a bridged tunnel that recorded no request gets its open and a close when it ends. Tunnel rows are kept out of `/v1/usage`: a tunnel's lifetime is not a request latency. - `httpMethod`, `httpPath` — the HTTP verb and path, so a request no parser recognized is still identifiable rather than showing only a host. Distinct from the `method` inside `a2a` / `mcp`, which is a protocol method name. On an opaque tunnel `httpMethod` is `CONNECT` and `httpPath` is absent — opaque bytes carry no request line. The path is query-stripped and percent-decoded, so query-borne credentials never reach the timeline, but a secret in a path *segment* (a bot token, a webhook path) does survive on this unauthenticated surface — worth knowing before exporting events off-box. @@ -616,7 +617,7 @@ The authbridge binary watches its config file (`/etc/authbridge/config.yaml`) vi **What reloads:** any plugin list change (add/remove/reorder) and any plugin `config:` subtree edit. -**What doesn't reload (pod restart required):** `mode`, any `listener.*` address, and the session store parameters (`session.ttl`, `session.max_events`, `session.max_sessions`, and the `session.archive` block). +**What doesn't reload (pod restart required):** `mode`, any `listener.*` address, the top-level `mtls` block, and the session store parameters (`session.ttl`, `session.max_events`, `session.max_sessions`, and the `session.archive` block). **Bad YAML stays safe:** if Load/Validate/Build/Start fails, the active pipeline keeps serving and the error is exposed on `/reload/status` with `reloads_failed` incremented. The pod never goes unhealthy from a bad edit. @@ -683,9 +684,21 @@ can talk to this authbridge. Per-caller policy / SPIFFE allowlists are out of scope; the trust bundle IS the policy. Plugins that want per-caller decisions read `pctx.PeerCert` and check the URI SAN. -**Hot-reload boundary:** mTLS config (`mtls.mode`, cert paths) requires a -pod restart to apply, matching the existing rule for `listener.*` -addresses. Plugin-pipeline config keeps its own hot-reload behavior. +**Hot-reload boundary:** mTLS config requires a pod restart to apply, +matching the existing rule for `listener.*` addresses. A reload that adds the +`mtls` block, removes it, or changes `mtls.mode` — the only field the block +carries — is refused rather than half-applied: the active pipeline keeps +serving, `reloads_failed` increments, and `/reload/status` names `mtls.*` +among the fields that need a restart. Legacy `cert_file` / `key_file` / +`bundle_file` keys are dropped at load, so editing them is neither applied nor +refused. The inbound TLS listener and, under `strict`, the forward proxy's +mTLS dialer are built once at boot, so no reload can reach them. The outbound +chain's redirect check (`forwardproxy.Support`) refuses a `WritesDestination` +plugin whenever an `mtls:` block is configured: under `strict` the dialer +would nest TLS inside TLS; under `permissive` there is no dialer, so the +refusal is broader than strictly needed and fails closed. Refusing the reload +keeps that check matched to the mtls config the listeners were built from. +Plugin-pipeline config keeps its own hot-reload behavior. ### envoy-sidecar mTLS diff --git a/cmd/agentop/README.md b/cmd/agentop/README.md index c1d44e7af..0b2b2b3ed 100644 --- a/cmd/agentop/README.md +++ b/cmd/agentop/README.md @@ -652,6 +652,98 @@ service status` fails, the line says it could not check rather than guessing. hand, what OpenCode needs to trust Cortex's CA, what Cortex records for it, what was verified, and its known issues. +## Choosing an inference server (`agentop server`) + +With several LiteLLM servers, each with its own URL and key, `agentop server` +chooses which one an agent's **new** sessions use, through the +[`inference-router`](../../docs/plugin-catalog.md#inference-router) plugin. A +session stays on the server it started on, including one already running when its +agent is first routed, until the proxy restarts: a restarted proxy treats every +session it sees as a new one. Routing is opt-in per agent: until an agent is given +a server, its traffic goes where the agent sends it. + +```sh +agentop server add ete https://ete-litellm.example.com # asks for the key; it does not echo +agentop server add glm https://glm-litellm.example.com +agentop server use glm --agent claude-code # new claude-code sessions → glm +agentop server # the servers, and two checks +agentop server reset --agent claude-code # stop routing claude-code +agentop server remove glm # refused while an agent uses it +``` + +With Claude Code's `ANTHROPIC_BASE_URL` at `ete`, the listing after `use` reads: + +``` + ete ete-litellm.example.com uses Claude Code's names + glm glm-litellm.example.com uses Claude Code's names claude-code + + ✓ Claude Code points at ete (~/.claude/settings.json) + ✓ Claude Code asks for Claude's own model names (~/.claude/settings.json) +``` + +- **`add `** reads the key at a prompt that does not echo, or from + stdin with `--key-stdin`, which is refused when stdin is a terminal, where the key + would echo as typed. It is never an argument, so it never reaches shell + history. A name that exists is replaced only after a yes (`--yes` skips the + question), which is how a key is rotated; with no terminal to ask on, nothing is + written. A server on a host another server already has, on any port, is + refused. The first `add` creates the plugin's entry, last in the outbound + pipeline. A key containing `$` is refused: the config loader would read it as an + environment variable. +- **`use --agent `** routes the agent's new sessions to the server, + including an agent that has not run yet. **`reset --agent `** stops + routing it. Either way a session already running stays where it is if it has sent + a request to one of the servers since the proxy started, which is how the router + knows where it is — unless the proxy has since dropped it from memory, which keeps + the most recently used sessions (100 by default), before the router pinned it. +- **`remove `** refuses while an agent is routed to the server, and names + the command that takes the agent off it; it also refuses the last server. A + session that started on a removed server gets a 503 asking for a new session + until the server is added back, or until the proxy restarts, which forgets + which server each session started on. +- **`agentop server`** lists each server's host (its whole URL when it is plain + `http`), its model mapping and the agents routed to it, then checks + `~/.claude/settings.json`: that `ANTHROPIC_BASE_URL` names one of the servers, + without which nothing is routed, and that neither the `model` setting nor a + model variable names a model other than Claude's (see + [Switching inference servers](../../docs/agents/claude-code.md#switching-inference-servers)). + It reads that one file, so it cannot see a project's settings, a file passed + with `claude --settings`, or a variable set in the shell. When the router's entry + has `on_error: observe`, which records what it would route and routes nothing, or + `on_error: off`, which stops it running, the listing says so first, and `use` and + `add` say so too. + +Two things the router needs that `agentop server` does not check. It routes only +requests it can read: a plain `http` request sent through the proxy, or an `https` +one the TLS bridge decrypts, which needs the bridge on and the port the agent +connects to in `tls_bridge.ports`, 443 and 8443 unless set. An agent whose base URL +is `https` on another port, such as `:4000`, reaches the proxy as an opaque +`CONNECT` tunnel and is never routed, although the check above, which matches by +host alone, shows ✓ for it; a server on such a port can still be where routed +requests go. And a server is `scheme://host[:port]` only: a routed request keeps +its own path, so an agent whose base URL has a path needs that same path on every +server. + +Every change is written to `~/.cortex/config.yaml`, keeping its comments except +those on the lines the change replaces or removes, every line of a removed block +among them, and returns once the proxy has reloaded it; nothing restarts. If the +proxy refuses the reload, the command puts the file back as it was, prints the +proxy's error and exits 1, and the proxy keeps the configuration it had. If the +proxy stops answering before it reports the reload, the command puts the file back +the same way, says the proxy stopped answering, and exits 1. A file someone edited +while the proxy was reloading is not put back but left as found, with an error +saying so. If the proxy reports neither a reload nor a refusal within 30 seconds, +the command exits 1 and leaves the file as written. With no proxy running the file is still written, and the change applies at +the next start, to every session from then: a proxy that starts knows where no +session is. Nothing is written unless the result loads as a Cortex config and +passes the router's own rules; what only the running proxy can check, such as +whether its build includes the router, is what a refused reload reports. A change +that changes nothing writes nothing. `--config PATH` points every form at another +config, with the stats address read from it: that is how to try this against a +second Cortex without touching the one every session uses. Flags go after the +action. Exit codes: 0 done, or nothing to do; 1 refused or failed; 2 a usage error, +a malformed name, agent or URL included; 3 a replacement declined. + ## Panes The UI has these panes. `Enter` drills in; `Esc` backs out. diff --git a/cmd/agentop/cmd_server.go b/cmd/agentop/cmd_server.go new file mode 100644 index 000000000..150518c5f --- /dev/null +++ b/cmd/agentop/cmd_server.go @@ -0,0 +1,418 @@ +package main + +import ( + "encoding/json" + "errors" + "flag" + "fmt" + "io" + "io/fs" + "maps" + "net/url" + "os" + "path/filepath" + "slices" + "strings" + "text/tabwriter" + "unicode/utf8" + + "github.com/rossoctl/cortex/core/config" + "github.com/rossoctl/cortex/core/pipeline" + "github.com/rossoctl/cortex/core/plugins/inferencerouter/routerconfig" +) + +// routerName is the inference-router plugin's registered name: the outbound entry +// every `agentop server` form reads and writes. +const routerName = "inference-router" + +const serverUsage = `agentop server — choose the inference server each coding agent's new sessions use + +Usage: + agentop server [--config PATH] + agentop server add [--key-stdin] [--yes] [--config PATH] + agentop server remove [--config PATH] + agentop server use --agent [--config PATH] + agentop server reset --agent [--config PATH] + +With no action, lists the servers and the agents routed to each, then checks +that Claude Code's settings let it be routed. + +A server is a gateway, such as LiteLLM, with its own URL and key. Routing is per +agent and opt-in: until "use" gives an agent a server, its traffic goes wherever +the agent sends it. A session stays on the server it started on, so a change +applies to new sessions only. A proxy restart forgets where each session is, and +treats each one it then sees as new. + +"add" reads the server's API key at a prompt that does not echo, or from stdin +with --key-stdin; it is never an argument. Adding a name that exists asks before +replacing it (--yes skips the question), which is also how a key is rotated. + +Every change is written to ~/.cortex/config.yaml, or to --config PATH, and returns +once the proxy has reloaded it. Nothing restarts. Flags go after the action. +` + +// runServer handles the `server` subcommand. Returns the process exit code. +func runServer(args []string, stdin io.Reader, stdout, stderr io.Writer) int { + action := "" + if len(args) > 0 && !strings.HasPrefix(args[0], "-") { + action, args = args[0], args[1:] + } + switch action { + case "": + return serverList(args, stdout, stderr) + case "help": + fmt.Fprint(stdout, serverUsage) + return 0 + case "add": + return serverAdd(args, stdin, stdout, stderr) + case "remove": + return serverRemove(args, stdout, stderr) + case "use": + return serverUse(args, stdout, stderr) + case "reset": + return serverReset(args, stdout, stderr) + } + fmt.Fprintf(stderr, "agentop server: unknown action %q\n\n", action) + fmt.Fprint(stderr, serverUsage) + return 2 +} + +// serverFlags parses flags and positional arguments in any order, so +// `agentop server use glm --agent claude-code` reads as written; the flag package +// alone stops at the first positional argument and would leave --agent unparsed. +// ok is false when the command should stop, with code its exit status: 0 for a help +// request, which goes to stdout, and 2 for a bad flag. +func serverFlags(flags *flag.FlagSet, args []string, stdout, stderr io.Writer) (pos []string, code int, ok bool) { + flags.SetOutput(stderr) + flags.Usage = func() {} + for { + if err := flags.Parse(args); err != nil { + if errors.Is(err, flag.ErrHelp) { + fmt.Fprint(stdout, serverUsage) + return nil, 0, false + } + fmt.Fprint(stderr, serverUsage) + return nil, 2, false + } + rest := flags.Args() + if len(rest) == 0 { + return pos, 0, true + } + // Parse consumed a "--": everything after it is positional. + if used := len(args) - len(rest); used > 0 && args[used-1] == "--" { + return append(pos, rest...), 0, true + } + pos = append(pos, rest[0]) + args = rest[1:] + } +} + +// serverTarget is the config `agentop server` reads and writes, and the stats URL +// whose /reload/status confirms a write — "" when no proxy answers there. +// +// Not localEditTargets, which it otherwise mirrors: that returns no path at all when +// the proxy is down, and a write with no proxy running is still worth making, since +// it applies at the next start. And --config names a file localEditTargets cannot. +func serverTarget(configPath string) (cfg *config.Config, path, statsURL string, err error) { + if configPath == "" { + cfg, path, err = localCortexConfig() + if errors.Is(err, fs.ErrNotExist) { + return nil, "", "", errors.New("no Cortex config at ~/.cortex/config.yaml; install Cortex with `agentop setup`, or name a config with --config") + } + } else { + path = configPath + cfg, err = config.Load(path) + } + if err != nil { + return nil, "", "", err + } + if statsURL = dialURL(cfg.Stats.StatsAddress); statsURL != "" && !localStatsUp(statsURL) { + statsURL = "" + } + return cfg, path, statsURL, nil +} + +// readRouter is the router entry's config, decoded without the plugin's validation +// so a listing still works on a config the proxy would refuse. present is false +// when the outbound chain has no router entry. +func readRouter(cfg *config.Config) (c routerconfig.Config, present bool, err error) { + for _, e := range cfg.Pipeline.Outbound.Plugins { + if e.Name != routerName { + continue + } + if len(e.Config) > 0 { + if err := json.Unmarshal(e.Config, &c); err != nil { + return routerconfig.Config{}, true, fmt.Errorf("the %s entry's config: %w", routerName, err) + } + } + return c, true, nil + } + return routerconfig.Config{}, false, nil +} + +// routerInactive is why the router entry in cfg routes nothing although it is in +// the chain, or "" when it routes: its on_error. Under observe the router redirects +// nothing and records observe/would_route where it would have routed; under off it +// does not run. Either way a server given to an agent changes no traffic, and a +// command that said the agent's sessions now go there would be wrong. path is the +// config file, for the fix. +func routerInactive(cfg *config.Config, path string) string { + for _, e := range cfg.Pipeline.Outbound.Plugins { + if e.Name != routerName { + continue + } + fix := fmt.Sprintf("Remove on_error from the entry in %s, or set it to enforce, to route.", homeTilde(path)) + switch e.OnError.Resolved() { + case pipeline.ErrorPolicyObserve: + return fmt.Sprintf("The %s entry runs under on_error: observe, so nothing is routed: a request it would "+ + "route is only recorded, as observe/would_route. %s", routerName, fix) + case pipeline.ErrorPolicyOff: + return fmt.Sprintf("The %s entry runs under on_error: off, so it does not run and nothing is routed. %s", routerName, fix) + } + return "" + } + return "" +} + +// agentsOn lists the agents routed to server, sorted. +func agentsOn(c routerconfig.Config, server string) []string { + var out []string + for agent, s := range c.Agents { + if s == server { + out = append(out, agent) + } + } + slices.Sort(out) + return out +} + +func serverList(args []string, stdout, stderr io.Writer) int { + flags := flag.NewFlagSet("server", flag.ContinueOnError) + cfgPath := flags.String("config", "", "Cortex config file (default ~/.cortex/config.yaml)") + pos, code, ok := serverFlags(flags, args, stdout, stderr) + if !ok { + return code + } + if len(pos) > 0 { + fmt.Fprintf(stderr, "agentop server: unexpected argument %q; the action comes first: agentop server [flags]\n", pos[0]) + return 2 + } + cfg, path, _, err := serverTarget(*cfgPath) + if err != nil { + fmt.Fprintf(stderr, "agentop server: %v\n", err) + return 1 + } + c, _, err := readRouter(cfg) + if err != nil { + fmt.Fprintf(stderr, "agentop server: %v\n", err) + return 1 + } + if len(c.Servers) == 0 { + fmt.Fprintln(stdout, "No inference servers yet. Add one:") + fmt.Fprintln(stdout, " agentop server add ") + return 0 + } + + // Every row has all four cells, the agents one empty when no agent is routed + // there: tabwriter aligns a column only across consecutive rows that have it, so + // a short row would restart the alignment below it. The padding that leaves after + // the mapping of such a row is trimmed. + var table strings.Builder + tw := tabwriter.NewWriter(&table, 0, 0, 3, ' ', 0) + for _, name := range slices.Sorted(maps.Keys(c.Servers)) { + s := c.Servers[name] + fmt.Fprintf(tw, " %s\t%s\t%s\t%s\n", name, serverHost(s), mappingText(s), strings.Join(agentsOn(c, name), ", ")) + } + tw.Flush() + for line := range strings.Lines(table.String()) { + fmt.Fprintln(stdout, strings.TrimRight(line, " \n")) + } + fmt.Fprintln(stdout) + + // First: while it holds, the checks below change nothing. + if why := routerInactive(cfg, path); why != "" { + printCheck(stdout, false, why) + } + for _, ck := range claudeCodeChecksAtHome(c) { + printCheck(stdout, ck.ok, ck.text) + } + return 0 +} + +// serverHost is how a server's URL is listed: its host, or the whole URL for plain +// http, so a server whose traffic crosses the network unencrypted says so. +// +// A URL the router refuses shows nothing of itself, not even its host: the likeliest +// reason it is refused is a pasted key, and a key given as a username that contains +// '/', '?' or '#' ends the authority early, so url.Parse takes the key for the host +// (https://sk-.../x@gw.example parses with Host "sk-..."). No parentheses either: +// `server add` already puts this in some. +func serverHost(s routerconfig.Server) string { + ep, err := routerconfig.ParseURL(s.URL) + if err != nil { + return "not a valid URL" + } + if ep.Scheme == "http" { + return ep.URL() + } + return ep.Host +} + +// mappingText is a server's model mapping as one cell. +func mappingText(s routerconfig.Server) string { + switch { + case s.Opus == "" && s.Sonnet == "" && s.Haiku == "": + return "uses Claude Code's names" + case s.Opus == s.Sonnet && s.Sonnet == s.Haiku: + return "all → " + s.Opus + } + or := func(m string) string { + if m == "" { + return "—" + } + return m + } + return fmt.Sprintf("opus → %s · sonnet → %s · haiku → %s", or(s.Opus), or(s.Sonnet), or(s.Haiku)) +} + +// settingsCheck is one line of `agentop server`'s check of Claude Code's settings. +type settingsCheck struct { + ok bool + text string +} + +// claudeCodeModelVars are the env settings that make Claude Code ask for a model by +// a name other than Claude's own; the settings' top-level "model" key is the other +// way. The router maps Claude's names, by family, to a server's; a request already +// carrying a server's name has no family left to map, and Claude Code shapes its +// requests for the model it believes it is using. +var claudeCodeModelVars = []string{ + "ANTHROPIC_MODEL", + "ANTHROPIC_DEFAULT_FABLE_MODEL", + "ANTHROPIC_DEFAULT_OPUS_MODEL", + "ANTHROPIC_DEFAULT_SONNET_MODEL", + "ANTHROPIC_DEFAULT_HAIKU_MODEL", + "ANTHROPIC_SMALL_FAST_MODEL", + "CLAUDE_CODE_SUBAGENT_MODEL", +} + +// claudeCodeChecksAtHome is claudeCodeChecks on ~/.claude/settings.json. With no +// home directory it says so rather than reading .claude/settings.json from wherever +// agentop was started, which is what joining an empty home would do. +func claudeCodeChecksAtHome(c routerconfig.Config) []settingsCheck { + home, err := os.UserHomeDir() + if err == nil && home == "" { + err = errors.New("it is empty") + } + if err != nil { + return []settingsCheck{{false, fmt.Sprintf("Claude Code's settings are not checked: cannot determine your home directory (%v)", err)}} + } + return claudeCodeChecks(filepath.Join(home, settingsRel), c) +} + +// claudeCodeChecks checks the settings file at settingsPath: that ANTHROPIC_BASE_URL +// is one of c's servers, and that neither the "model" key nor any model variable +// names a non-Claude model. It reads that one file, so it cannot see a file passed +// with `claude --settings`, nor a variable set in the shell. +func claudeCodeChecks(settingsPath string, c routerconfig.Config) []settingsCheck { + shown := homeTilde(settingsPath) + doc, err := readSettings(settingsPath) + if err != nil { + return []settingsCheck{{false, fmt.Sprintf("cannot read %s: %v", shown, err)}} + } + env := envStrings(doc) + model, _ := doc["model"].(string) + return append([]settingsCheck{baseURLCheck(env["ANTHROPIC_BASE_URL"], shown, c)}, modelChecks(model, env, shown)...) +} + +func baseURLCheck(raw, shown string, c routerconfig.Config) settingsCheck { + if raw == "" { + return settingsCheck{false, fmt.Sprintf("%s sets no ANTHROPIC_BASE_URL, so Claude Code talks to Anthropic and nothing is routed. Point it at one of the servers above.", shown)} + } + // raw is never quoted: a pasted key can sit in the user info, path, query or + // fragment. Of a URL that parses, scheme://host[:port] is quoted, and only when raw + // has no '@' anywhere: a username containing '/', '?' or '#' ends the authority + // early, and url.Parse then takes the key for the host. A URL with an '@' is + // still matched to a server by host, but a mismatch names no host. + u, err := url.Parse(raw) + if err != nil || u.Host == "" { + return settingsCheck{false, fmt.Sprintf("ANTHROPIC_BASE_URL in %s is not a URL with a host, so nothing is routed. Point it at one of the servers above.", shown)} + } + host := routerconfig.Hostname(u.Host) + for _, name := range slices.Sorted(maps.Keys(c.Servers)) { + if ep, err := routerconfig.ParseURL(c.Servers[name].URL); err == nil && ep.Hostname == host { + return settingsCheck{true, fmt.Sprintf("Claude Code points at %s (%s)", name, shown)} + } + } + where := "a URL that is" + if !strings.Contains(raw, "@") { + where = u.Scheme + "://" + u.Host + ", which is" + } + return settingsCheck{false, fmt.Sprintf("Claude Code points at %s not one of these servers, so nothing is routed (ANTHROPIC_BASE_URL in %s). Point it at one of the servers above.", where, shown)} +} + +// modelChecks checks model, the settings' top-level "model" key, and env's model +// variables. +func modelChecks(model string, env map[string]string, shown string) []settingsCheck { + var out []settingsCheck + check := func(m, where string) { + if m != "" && !isClaudeModel(m) { + out = append(out, settingsCheck{false, fmt.Sprintf( + "Claude Code asks for %s instead of Claude's models (%s in %s). Cortex can't map that back: remove the line, and route Claude Code only to servers that serve Claude's names.", + m, where, shown)}) + } + } + check(model, `"model"`) + for _, v := range claudeCodeModelVars { + check(env[v], v) + } + if len(out) == 0 { + out = append(out, settingsCheck{true, fmt.Sprintf("Claude Code asks for Claude's own model names (%s)", shown)}) + } + return out +} + +// isClaudeModel reports whether m is one of Claude Code's own aliases — with or +// without a context suffix such as [1m] — or a name containing "claude". +func isClaudeModel(m string) bool { + m = strings.ToLower(strings.TrimSpace(m)) + if i := strings.IndexByte(m, '['); i > 0 { + m = m[:i] + } + switch m { + case "default", "best", "fable", "opus", "opusplan", "sonnet", "haiku": + return true + } + return strings.Contains(m, "claude") +} + +// printCheck writes one check as " ✓ text", wrapped at 80 columns with the +// continuation lines under the text. +func printCheck(w io.Writer, ok bool, text string) { + mark := "✓" + if !ok { + mark = "✗" + } + line, words := " "+mark, 0 + for _, word := range strings.Fields(text) { + if words > 0 && utf8.RuneCountInString(line)+1+utf8.RuneCountInString(word) > 80 { + fmt.Fprintln(w, line) + line = " " + } + line += " " + word + words++ + } + fmt.Fprintln(w, line) +} + +// homeTilde shows path with the home directory as ~, the way the docs name it. +func homeTilde(path string) string { + home, err := os.UserHomeDir() + if err != nil || home == "" { + return path + } + if rel, err := filepath.Rel(home, path); err == nil && !strings.HasPrefix(rel, "..") { + return filepath.Join("~", rel) + } + return path +} diff --git a/cmd/agentop/cmd_server_test.go b/cmd/agentop/cmd_server_test.go new file mode 100644 index 000000000..494ebc098 --- /dev/null +++ b/cmd/agentop/cmd_server_test.go @@ -0,0 +1,435 @@ +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "net" + "os" + "path/filepath" + "strings" + "testing" + "unicode/utf8" + + "github.com/rossoctl/cortex/core/plugins/inferencerouter/routerconfig" +) + +// routerBlock is the inference-router entry agentop writes, with two servers and +// claude-code routed to glm. +const routerBlock = ` - name: inference-router + config: + servers: + ete: + url: https://ete.example.com + key: sk-ete + glm: + url: https://glm.example.com:8443 + key: sk-glm + agents: + claude-code: glm +` + +// closedAddr is a loopback address nothing listens on. +func closedAddr(t *testing.T) string { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + addr := l.Addr().String() + l.Close() + return addr +} + +// serverEnv gives a test a scratch HOME, so nothing reads or writes the real +// ~/.claude or ~/.cortex, and a config in it whose stats address is statsAddr and +// whose outbound chain ends with router (none when ""). It returns the config's +// path, which sits directly in that HOME: filepath.Dir of it is the scratch home. +func serverEnv(t *testing.T, statsAddr, router string) string { + t.Helper() + home := t.TempDir() + t.Setenv("HOME", home) + src := fmt.Sprintf(`mode: proxy-sidecar +stats: + address: %s +pipeline: + outbound: + plugins: + - name: inference-parser + - name: tool-prune + config: + remove: [] +%s`, statsAddr, router) + path := filepath.Join(home, "cortex.yaml") + if err := os.WriteFile(path, []byte(src), 0o600); err != nil { + t.Fatal(err) + } + return path +} + +// writeClaudeSettings puts env in home's .claude/settings.json. home is a +// parameter, not $HOME, so a test that calls this before serverEnv writes into a +// directory it named rather than over the developer's real settings. +func writeClaudeSettings(t *testing.T, home string, env map[string]string) { + t.Helper() + writeClaudeSettingsDoc(t, home, map[string]any{"env": env}) +} + +// writeClaudeSettingsDoc puts doc, a whole settings document, in home's +// .claude/settings.json. +func writeClaudeSettingsDoc(t *testing.T, home string, doc map[string]any) { + t.Helper() + path := filepath.Join(home, ".claude", "settings.json") + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + b, _ := json.Marshal(doc) + if err := os.WriteFile(path, b, 0o600); err != nil { + t.Fatal(err) + } +} + +func runServerCmd(t *testing.T, stdin string, args ...string) (code int, stdout, stderr string) { + t.Helper() + var out, errb bytes.Buffer + code = runServer(args, strings.NewReader(stdin), &out, &errb) + return code, out.String(), errb.String() +} + +// flat joins s's words with single spaces, so a phrase can be found across the +// line wrapping printCheck applies. +func flat(s string) string { return strings.Join(strings.Fields(s), " ") } + +func TestServer_SaysWhenThereAreNoServers(t *testing.T) { + path := serverEnv(t, closedAddr(t), "") + code, out, _ := runServerCmd(t, "", "--config", path) + if code != 0 || !strings.Contains(out, "No inference servers yet") || !strings.Contains(out, "agentop server add ") { + t.Errorf("exit %d, stdout:\n%s", code, out) + } +} + +func TestServer_ListsEachServerWithItsHostMappingAndAgents(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + code, out, errOut := runServerCmd(t, "", "--config", path) + if code != 0 { + t.Fatalf("exit %d: %s", code, errOut) + } + lines := strings.Split(out, "\n") + if got := flat(lines[0]); got != "ete ete.example.com uses Claude Code's names" { + t.Errorf("line 1 = %q", got) + } + if got := flat(lines[1]); got != "glm glm.example.com:8443 uses Claude Code's names claude-code" { + t.Errorf("line 2 = %q", got) + } + if strings.Index(lines[0], "uses") != strings.Index(lines[1], "uses") { + t.Errorf("columns do not line up:\n%s\n%s", lines[0], lines[1]) + } +} + +func TestServer_ListsPlainHTTPServersByTheirWholeURL(t *testing.T) { + path := serverEnv(t, closedAddr(t), strings.Replace(routerBlock, "https://ete.example.com", "http://localhost:4000", 1)) + _, out, _ := runServerCmd(t, "", "--config", path) + if !strings.Contains(out, "http://localhost:4000") { + t.Errorf("want the plain-http server listed with its scheme:\n%s", out) + } +} + +func TestServer_ChecksWhereClaudeCodePoints(t *testing.T) { + for _, tc := range []struct { + name, baseURL, want string + }{ + {"a server", "https://ete.example.com", "✓ Claude Code points at ete (~/.claude/settings.json)"}, + {"a server, other port", "https://GLM.example.com:9999/", "✓ Claude Code points at glm (~/.claude/settings.json)"}, + {"elsewhere", "https://api.anthropic.com", "✗ Claude Code points at https://api.anthropic.com, which is not one of these servers, so nothing is routed (ANTHROPIC_BASE_URL in ~/.claude/settings.json). Point it at one of the servers above."}, + {"nowhere", "", "✗ ~/.claude/settings.json sets no ANTHROPIC_BASE_URL"}, + } { + t.Run(tc.name, func(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + env := map[string]string{} + if tc.baseURL != "" { + env["ANTHROPIC_BASE_URL"] = tc.baseURL + } + writeClaudeSettings(t, filepath.Dir(path), env) + _, out, _ := runServerCmd(t, "", "--config", path) + if !strings.Contains(flat(out), tc.want) { + t.Errorf("want %q in:\n%s", tc.want, out) + } + }) + } +} + +func TestServer_FlagsAModelVariableThatNamesAnotherModel(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + writeClaudeSettings(t, filepath.Dir(path), map[string]string{ + "ANTHROPIC_BASE_URL": "https://ete.example.com", + "ANTHROPIC_MODEL": "glm-5.3", + "ANTHROPIC_DEFAULT_OPUS_MODEL": "claude-opus-5-5", + "ANTHROPIC_SMALL_FAST_MODEL": "haiku", + "CLAUDE_CODE_SUBAGENT_MODEL": "opus[1m]", + }) + _, out, _ := runServerCmd(t, "", "--config", path) + got := flat(out) + if want := "✗ Claude Code asks for glm-5.3 instead of Claude's models (ANTHROPIC_MODEL in ~/.claude/settings.json). Cortex can't map that back: remove the line"; !strings.Contains(got, want) { + t.Errorf("want %q in:\n%s", want, out) + } + if strings.Count(got, "✗") != 1 { + t.Errorf("want exactly one failed check — Claude's own names and aliases pass:\n%s", out) + } +} + +func TestServer_PassesSettingsThatNameOnlyClaudesModels(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + writeClaudeSettings(t, filepath.Dir(path), map[string]string{"ANTHROPIC_BASE_URL": "https://ete.example.com"}) + _, out, _ := runServerCmd(t, "", "--config", path) + if want := "✓ Claude Code asks for Claude's own model names (~/.claude/settings.json)"; !strings.Contains(flat(out), want) { + t.Errorf("want %q in:\n%s", want, out) + } +} + +func TestIsClaudeModel(t *testing.T) { + for m, want := range map[string]bool{ + "opus": true, "sonnet": true, "haiku": true, "opusplan": true, "default": true, "sonnet[1m]": true, + "best": true, "fable": true, "Fable[1m]": true, + "claude-opus-5-5": true, "us.anthropic.claude-sonnet-5": true, "Claude-Haiku-4-5": true, + "glm-5.3": false, "gpt-5": false, "premium-ide": false, + } { + if got := isClaudeModel(m); got != want { + t.Errorf("isClaudeModel(%q) = %v, want %v", m, got, want) + } + } +} + +func TestMappingText(t *testing.T) { + for _, tc := range []struct { + s routerconfig.Server + want string + }{ + {routerconfig.Server{}, "uses Claude Code's names"}, + {routerconfig.Server{Opus: "glm-5.3", Sonnet: "glm-5.3", Haiku: "glm-5.3"}, "all → glm-5.3"}, + {routerconfig.Server{Opus: "big", Sonnet: "mid", Haiku: "small"}, "opus → big · sonnet → mid · haiku → small"}, + } { + if got := mappingText(tc.s); got != tc.want { + t.Errorf("mappingText(%+v) = %q, want %q", tc.s, got, tc.want) + } + } +} + +func TestServer_HelpGoesToStdoutAndAWrongActionToStderr(t *testing.T) { + if code, out, _ := runServerCmd(t, "", "--help"); code != 0 || !strings.Contains(out, "agentop server add ") { + t.Errorf("--help: exit %d, stdout:\n%s", code, out) + } + if code, _, errOut := runServerCmd(t, "", "rename"); code != 2 || !strings.Contains(errOut, `unknown action "rename"`) { + t.Errorf("rename: exit %d, stderr:\n%s", code, errOut) + } +} + +func TestServer_AnActionAfterAFlagIsRefusedWithTheFix(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + code, _, errOut := runServerCmd(t, "", "--config", path, "remove", "ete") + if code != 2 || !strings.Contains(errOut, "the action comes first") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } +} + +// The router maps Claude's names, so every place Claude Code takes a model from is +// checked: the settings' top-level "model" setting, and each model variable, +// ANTHROPIC_DEFAULT_FABLE_MODEL among them. Claude's aliases pass wherever they sit. +func TestServer_ChecksEveryWayClaudeCodePicksAModel(t *testing.T) { + for _, tc := range []struct { + name string + doc map[string]any + wantX string // the one ✗ expected; "" for none + }{ + {"model key, another model", map[string]any{"model": "glm-5.3"}, + `✗ Claude Code asks for glm-5.3 instead of Claude's models ("model" in ~/.claude/settings.json). Cortex can't map that back: remove the line`}, + {"fable variable, another model", map[string]any{"env": map[string]any{"ANTHROPIC_DEFAULT_FABLE_MODEL": "glm-5.3"}}, + `✗ Claude Code asks for glm-5.3 instead of Claude's models (ANTHROPIC_DEFAULT_FABLE_MODEL in ~/.claude/settings.json)`}, + {"model key fable", map[string]any{"model": "fable"}, ""}, + {"model key opus", map[string]any{"model": "opus"}, ""}, + {"model key, a claude id", map[string]any{"model": "claude-opus-5-5[1m]"}, ""}, + {"fable and best variables", map[string]any{"env": map[string]any{"ANTHROPIC_MODEL": "fable", "CLAUDE_CODE_SUBAGENT_MODEL": "best"}}, ""}, + } { + t.Run(tc.name, func(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + env, _ := tc.doc["env"].(map[string]any) + if env == nil { + env = map[string]any{} + } + env["ANTHROPIC_BASE_URL"] = "https://ete.example.com" + tc.doc["env"] = env + writeClaudeSettingsDoc(t, filepath.Dir(path), tc.doc) + _, out, _ := runServerCmd(t, "", "--config", path) + got := flat(out) + if tc.wantX == "" { + if want := "✓ Claude Code asks for Claude's own model names"; strings.Contains(got, "✗") || !strings.Contains(got, want) { + t.Errorf("want no ✗ and %q in:\n%s", want, out) + } + return + } + if strings.Count(got, "✗") != 1 || !strings.Contains(got, tc.wantX) { + t.Errorf("want exactly one ✗, %q, in:\n%s", tc.wantX, out) + } + }) + } +} + +// A key pasted into a server URL as user info must not reach the terminal, and +// url.URL.Redacted would not stop it: it masks a password but not a username. +func TestServer_ListsNoURLCredentials(t *testing.T) { + router := strings.NewReplacer( + "https://ete.example.com", "https://sk-SECRET-ETE@ete.example.com/sk-SECRET-PATH", + "https://glm.example.com:8443", "https://user:sk-SECRET-GLM@glm.example.com:8443", + ).Replace(routerBlock) + path := serverEnv(t, closedAddr(t), router) + code, out, errOut := runServerCmd(t, "", "--config", path) + if code != 0 { + t.Fatalf("exit %d: %s", code, errOut) + } + if all := out + errOut; strings.Contains(all, "sk-SECRET") || strings.Contains(all, "user:") { + t.Errorf("the listing prints URL user info:\n%s%s", out, errOut) + } + lines := strings.Split(out, "\n") + if got := flat(lines[0]); !strings.HasPrefix(got, "ete not a valid URL uses") { + t.Errorf("line 1 = %q, want not a valid URL and nothing of the URL", got) + } + if got := flat(lines[1]); !strings.HasPrefix(got, "glm not a valid URL uses") { + t.Errorf("line 2 = %q, want not a valid URL and nothing of the URL", got) + } +} + +func TestServer_ChecksPrintNoURLCredentials(t *testing.T) { + for _, tc := range []struct { + name, baseURL, want string + }{ + {"elsewhere, key as username", "https://sk-SECRET@api.anthropic.com/v1", + "✗ Claude Code points at a URL that is not one of these servers, so nothing is routed"}, + {"elsewhere, key as password", "https://user:sk-SECRET@api.anthropic.com", + "✗ Claude Code points at a URL that is not one of these servers, so nothing is routed"}, + {"a server, with user info", "https://user:sk-SECRET@ete.example.com", + "✓ Claude Code points at ete"}, + {"does not parse", "https://user:sk-SECRET@api.anthropic.com:port", + "✗ ANTHROPIC_BASE_URL in ~/.claude/settings.json is not a URL with a host, so nothing is routed. Point it at one of the servers above."}, + {"no slashes", "https:user:sk-SECRET@api.anthropic.com", + "✗ ANTHROPIC_BASE_URL in ~/.claude/settings.json is not a URL with a host"}, + {"elsewhere, key in the query", "https://other.example.com/?key=sk-SECRET", + "✗ Claude Code points at https://other.example.com, which is not one of these servers"}, + {"elsewhere, key in the fragment", "https://other.example.com/v1#sk-SECRET", + "✗ Claude Code points at https://other.example.com, which is not one of these servers"}, + {"elsewhere, key as a path segment", "https://other.example.com/v1/sk-SECRET", + "✗ Claude Code points at https://other.example.com, which is not one of these servers"}, + {"elsewhere, with a port", "https://other.example.com:8443/v1/sk-SECRET", + "✗ Claude Code points at https://other.example.com:8443, which is not one of these servers"}, + } { + t.Run(tc.name, func(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + writeClaudeSettings(t, filepath.Dir(path), map[string]string{"ANTHROPIC_BASE_URL": tc.baseURL}) + _, out, errOut := runServerCmd(t, "", "--config", path) + if all := out + errOut; strings.Contains(all, "sk-SECRET") || strings.Contains(all, "user:") { + t.Errorf("the checks print a key from the URL:\n%s%s", out, errOut) + } + if !strings.Contains(flat(out), tc.want) { + t.Errorf("want %q in:\n%s", tc.want, out) + } + }) + } +} + +// A server no agent is routed to must not end the column block: tabwriter aligns a +// column only across consecutive rows that all have it. +func TestServer_AgentsLineUpAroundAServerWithNone(t *testing.T) { + path := serverEnv(t, closedAddr(t), ` - name: inference-router + config: + servers: + aaa: + url: https://a.example.com + key: sk-a + opus: big + sonnet: mid + haiku: small + bbb: + url: https://b.example.com + key: sk-b + ccc: + url: https://c.example.com + key: sk-c + agents: + claude-code: aaa + opencode: ccc +`) + code, out, errOut := runServerCmd(t, "", "--config", path) + if code != 0 { + t.Fatalf("exit %d: %s", code, errOut) + } + lines := strings.Split(out, "\n") + // Columns, not bytes: the mapping's arrows are multi-byte. + col := func(line, word string) int { + i := strings.Index(line, word) + if i < 0 { + t.Fatalf("no %q in %q", word, line) + } + return utf8.RuneCountInString(line[:i]) + } + if a, c := col(lines[0], "claude-code"), col(lines[2], "opencode"); a != c { + t.Errorf("agents start in columns %d and %d:\n%s", a, c, out) + } + for i, line := range lines[:3] { + if strings.TrimRight(line, " ") != line { + t.Errorf("line %d ends in spaces: %q", i+1, line) + } + } +} + +// With no home directory there is no ~/.claude/settings.json to check. The +// listing still prints, and the checks say why they did not run instead of +// reading .claude/settings.json from wherever agentop was started. +func TestServer_AnUnknownHomeIsReportedNotReadFromTheWorkingDirectory(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + cwd := t.TempDir() + writeClaudeSettings(t, cwd, map[string]string{"ANTHROPIC_BASE_URL": "https://ete.example.com"}) + t.Chdir(cwd) + t.Setenv("HOME", "") + code, out, errOut := runServerCmd(t, "", "--config", path) + if code != 0 { + t.Fatalf("exit %d: %s", code, errOut) + } + got := flat(out) + if !strings.Contains(got, "ete ete.example.com") { + t.Errorf("want the listing still printed:\n%s", out) + } + if strings.Contains(got, "points at") { + t.Errorf("read the working directory's .claude/settings.json:\n%s", out) + } + if want := "✗ Claude Code's settings are not checked: cannot determine your home directory"; !strings.Contains(got, want) { + t.Errorf("want %q in:\n%s", want, out) + } +} + +// A key given as a username that contains '/', '?' or '#' ends the authority +// early, so url.Parse takes the KEY for the host: https://sk-SECRET/x@h.example.com +// parses with Host "sk-SECRET". Nothing of a refused server URL may be shown, and +// no host of an ANTHROPIC_BASE_URL containing '@'. +func TestServer_AKeyTakenForTheHostIsNeverShown(t *testing.T) { + for _, raw := range []string{ + "https://sk-SECRET/x@h.example.com", + "https://sk-SECRET?x@h.example.com", + "https://sk-SECRET#x@h.example.com", + } { + t.Run(raw, func(t *testing.T) { + path := serverEnv(t, closedAddr(t), strings.Replace(routerBlock, "https://ete.example.com", raw, 1)) + writeClaudeSettings(t, filepath.Dir(path), map[string]string{"ANTHROPIC_BASE_URL": raw}) + code, out, errOut := runServerCmd(t, "", "--config", path) + if code != 0 { + t.Fatalf("exit %d: %s", code, errOut) + } + if all := strings.ToLower(out + errOut); strings.Contains(all, "secret") { + t.Errorf("prints the key:\n%s%s", out, errOut) + } + if got := flat(strings.Split(out, "\n")[0]); !strings.HasPrefix(got, "ete not a valid URL uses") { + t.Errorf("line 1 = %q, want the server listed as not a valid URL, with nothing of the URL", got) + } + if want := "✗ Claude Code points at a URL that is not one of these servers, so nothing is routed (ANTHROPIC_BASE_URL in ~/.claude/settings.json). Point it at one of the servers above."; !strings.Contains(flat(out), want) { + t.Errorf("want %q in:\n%s", want, out) + } + }) + } +} diff --git a/cmd/agentop/cmd_server_write.go b/cmd/agentop/cmd_server_write.go new file mode 100644 index 000000000..995c05ec6 --- /dev/null +++ b/cmd/agentop/cmd_server_write.go @@ -0,0 +1,421 @@ +package main + +import ( + "context" + "errors" + "flag" + "fmt" + "io" + "maps" + "os" + "slices" + "strings" + + "github.com/charmbracelet/x/term" + + "github.com/rossoctl/cortex/cmd/agentop/edit" + "github.com/rossoctl/cortex/core/config" + "github.com/rossoctl/cortex/core/plugins/inferencerouter/routerconfig" +) + +// serverConfirm asks before `server add` replaces a server. Its own var, as +// claudeCodeConfirm is, so a test stubbing it cannot disarm another command's prompt. +var serverConfirm = confirm + +// writePluginConfig is edit.WritePluginConfig. A var so a test can stand in for a +// proxy that stops answering mid-poll, which the real poller takes about 16s of +// backoff to give up on. +var writePluginConfig = edit.WritePluginConfig + +// readServerKey reads a server's API key at a prompt that does not echo, on the +// controlling terminal rather than stdin, so it works when stdin is a pipe and the +// prompt is not written into redirected output. A var so tests can stand in for the +// terminal a test process does not have. +var readServerKey = func(name string) (string, error) { + tty, err := os.OpenFile("/dev/tty", os.O_RDWR, 0) + if err != nil { + return "", errors.New("no terminal to read the key from; pipe it in with --key-stdin") + } + defer tty.Close() + fmt.Fprintf(tty, "API key for %s: ", name) + b, err := term.ReadPassword(tty.Fd()) + fmt.Fprintln(tty) + if err != nil { + return "", fmt.Errorf("reading the key: %w", err) + } + return string(b), nil +} + +// stdinIsTerminal reports whether stdin is a terminal. A var so a test can say it +// is: a test's stdin never is. +var stdinIsTerminal = func(stdin io.Reader) bool { + f, ok := stdin.(*os.File) + return ok && term.IsTerminal(f.Fd()) +} + +// readKey is the key for server name: from stdin with --key-stdin, else at the +// hidden prompt. Never from an argument, which would put it in shell history. +func readKey(name string, fromStdin bool, stdin io.Reader) (string, error) { + if !fromStdin { + return readServerKey(name) + } + b, err := io.ReadAll(io.LimitReader(stdin, 64<<10)) + if err != nil { + return "", fmt.Errorf("reading the key from stdin: %w", err) + } + key := strings.TrimRight(string(b), "\r\n") + if strings.ContainsAny(key, "\r\n") { + return "", errors.New("stdin holds more than one line; --key-stdin reads the key alone") + } + return key, nil +} + +func serverAdd(args []string, stdin io.Reader, stdout, stderr io.Writer) int { + flags := flag.NewFlagSet("server add", flag.ContinueOnError) + cfgPath := flags.String("config", "", "Cortex config file (default ~/.cortex/config.yaml)") + keyStdin := flags.Bool("key-stdin", false, "read the API key from stdin instead of a prompt") + yes := flags.Bool("yes", false, "replace an existing server without asking") + pos, code, ok := serverFlags(flags, args, stdout, stderr) + if !ok { + return code + } + if len(pos) != 2 { + fmt.Fprintln(stderr, "agentop server add: want a name and a URL: agentop server add ") + return 2 + } + name, rawURL := pos[0], pos[1] + if err := routerconfig.CheckName(name); err != nil { + fmt.Fprintf(stderr, "agentop server add: %v\n", err) + return 2 + } + // ParseURL's error, never rawURL: a refused URL may carry a key anywhere in it, + // and ParseURL's errors quote no part of the URL. + ep, err := routerconfig.ParseURL(rawURL) + if err != nil { + fmt.Fprintf(stderr, "agentop server add: %v\n", err) + return 2 + } + // Before anything is asked or read: --key-stdin reads stdin as typed, so on a + // terminal the key would echo, character by character, onto the screen and into + // any recording of it. The prompt without the flag reads it without echo. + if *keyStdin && stdinIsTerminal(stdin) { + fmt.Fprintln(stderr, "agentop server add: --key-stdin reads the key from stdin, and stdin is a terminal, "+ + "so the key would echo as you typed it. Pipe the key in, or drop --key-stdin to be asked for it without echo.") + return 1 + } + + cfg, path, statsURL, err := serverTarget(*cfgPath) + if err != nil { + fmt.Fprintf(stderr, "agentop server add: %v\n", err) + return 1 + } + c, _, err := readRouter(cfg) + if err != nil { + fmt.Fprintf(stderr, "agentop server add: %v\n", err) + return 1 + } + for _, other := range slices.Sorted(maps.Keys(c.Servers)) { + if oep, err := routerconfig.ParseURL(c.Servers[other].URL); other != name && err == nil && oep.Hostname == ep.Hostname { + fmt.Fprintf(stderr, "agentop server add: %s is already on %s. Each server needs a host of its own, "+ + "because agentop names a session's server from the host its requests went to.\n", other, ep.Hostname) + return 1 + } + } + old, replacing := c.Servers[name] + if replacing { + fmt.Fprintf(stdout, "%s is already configured (%s). Replacing it gives it this URL and key, "+ + "and the sessions on it use them from their next request.\n", name, serverHost(old)) + if !*yes && !serverConfirm(stdout) { + return exitDeclined + } + } + + key, err := readKey(name, *keyStdin, stdin) + if err == nil { + err = routerconfig.CheckKey(key) + } + if err == nil && strings.Contains(key, "$") { + // config.Load expands $NAME and ${NAME} across the whole file, so a literal $ + // in a key would reach the server as something else. + err = errors.New("the key contains $, which Cortex reads as an environment variable when it loads the config; " + + "put the key in an environment variable the proxy has, and write key: ${NAME} by hand") + } + if err != nil { + fmt.Fprintf(stderr, "agentop server add: %v\n", err) + return 1 + } + if ep.PlaintextRemote() { + fmt.Fprintf(stderr, "agentop server add: warning: %s is plain http on another machine, so requests routed there "+ + "cross the network decrypted, key and prompts included.\n", ep.URL()) + } + + ch := edit.ConfigChange{Chain: "outbound", Plugin: routerName, Path: []string{"servers", name}, + Value: edit.MapValue("url", ep.URL(), "key", key), CreatePlugin: true} + done := "Added " + name + "." + if replacing { + done = "Replaced " + name + "." + } + if len(agentsOn(c, name)) == 0 { + done += " No agent uses it yet. Route one with\n agentop server use " + name + " --agent claude-code" + } + return runServerWrite(stdout, stderr, path, statsURL, ch, writeReport{ + done: done, + unchanged: name + " already has that URL and key; nothing to change.", + note: routerInactive(cfg, path), + }) +} + +func serverRemove(args []string, stdout, stderr io.Writer) int { + flags := flag.NewFlagSet("server remove", flag.ContinueOnError) + cfgPath := flags.String("config", "", "Cortex config file (default ~/.cortex/config.yaml)") + pos, code, ok := serverFlags(flags, args, stdout, stderr) + if !ok { + return code + } + if len(pos) != 1 { + fmt.Fprintln(stderr, "agentop server remove: want a server name: agentop server remove ") + return 2 + } + name := pos[0] + // First, as add does: every message below echoes name, and a name CheckName + // accepts carries nothing a terminal would act on. Its own refusal quotes it. + if err := routerconfig.CheckName(name); err != nil { + fmt.Fprintf(stderr, "agentop server remove: %v\n", err) + return 2 + } + cfg, path, statsURL, err := serverTarget(*cfgPath) + if err != nil { + fmt.Fprintf(stderr, "agentop server remove: %v\n", err) + return 1 + } + c, _, err := readRouter(cfg) + if err != nil { + fmt.Fprintf(stderr, "agentop server remove: %v\n", err) + return 1 + } + if _, ok := c.Servers[name]; !ok { + fmt.Fprintf(stderr, "agentop server remove: no server named %s%s\n", name, configuredServers(c)) + return 1 + } + agents := agentsOn(c, name) + // The only server first, since no command removes it. With agents routed to it + // the refusal names the step that makes it carry no traffic too, rather than + // naming that step alone and refusing again once it is taken. + if len(c.Servers) == 1 { + fmt.Fprintf(stderr, "agentop server remove: %s is the only server, and %s needs one. ", name, routerName) + if len(agents) > 0 { + fmt.Fprintf(stderr, "It is %s's server for new sessions; stop routing to it with\n", strings.Join(agents, " and ")) + for _, a := range agents { + fmt.Fprintf(stderr, " agentop server reset --agent %s\n", a) + } + fmt.Fprint(stderr, "and it changes no traffic. ") + } else { + fmt.Fprint(stderr, "With no agent routed to it it changes no traffic. ") + } + fmt.Fprintf(stderr, "To take the router out altogether, delete its entry from %s.\n", homeTilde(path)) + return 1 + } + if len(agents) > 0 { + var other string + for _, s := range slices.Sorted(maps.Keys(c.Servers)) { + if s != name { + other = s + break + } + } + fmt.Fprintf(stderr, "%s is %s's server for new sessions. Pick another first:\n", name, strings.Join(agents, " and ")) + for _, a := range agents { + fmt.Fprintf(stderr, " agentop server use %s --agent %s\n", other, a) + } + return 1 + } + ch := edit.ConfigChange{Chain: "outbound", Plugin: routerName, Path: []string{"servers", name}} + return runServerWrite(stdout, stderr, path, statsURL, ch, writeReport{ + done: "Removed " + name + ".", + live: fmt.Sprintf("A session that started on it now gets an error asking for a new session; adding %s back restores it.", name), + }) +} + +func serverUse(args []string, stdout, stderr io.Writer) int { + flags := flag.NewFlagSet("server use", flag.ContinueOnError) + cfgPath := flags.String("config", "", "Cortex config file (default ~/.cortex/config.yaml)") + agent := flags.String("agent", "", "the agent whose new sessions go to the server, such as claude-code") + pos, code, ok := serverFlags(flags, args, stdout, stderr) + if !ok { + return code + } + if len(pos) != 1 || *agent == "" { + fmt.Fprintln(stderr, "agentop server use: want a server and an agent: agentop server use --agent ") + return 2 + } + name := pos[0] + // First, as add does: see serverRemove. + if err := routerconfig.CheckName(name); err != nil { + fmt.Fprintf(stderr, "agentop server use: %v\n", err) + return 2 + } + if err := routerconfig.CheckAgent(*agent); err != nil { + fmt.Fprintf(stderr, "agentop server use: %v\n", err) + return 2 + } + cfg, path, statsURL, err := serverTarget(*cfgPath) + if err != nil { + fmt.Fprintf(stderr, "agentop server use: %v\n", err) + return 1 + } + c, _, err := readRouter(cfg) + if err != nil { + fmt.Fprintf(stderr, "agentop server use: %v\n", err) + return 1 + } + if _, ok := c.Servers[name]; !ok { + fmt.Fprintf(stderr, "agentop server use: no server named %s%s. Add it first:\n agentop server add %s \n", + name, configuredServers(c), name) + return 1 + } + ch := edit.ConfigChange{Chain: "outbound", Plugin: routerName, Path: []string{"agents", *agent}, Value: edit.ScalarValue(name)} + return runServerWrite(stdout, stderr, path, statsURL, ch, writeReport{ + done: fmt.Sprintf("New %s sessions → %s.", *agent, name), + live: runningSessionsStay, + unchanged: fmt.Sprintf("New %s sessions already go to %s.", *agent, name), + note: routerInactive(cfg, path), + }) +} + +func serverReset(args []string, stdout, stderr io.Writer) int { + flags := flag.NewFlagSet("server reset", flag.ContinueOnError) + cfgPath := flags.String("config", "", "Cortex config file (default ~/.cortex/config.yaml)") + agent := flags.String("agent", "", "the agent to stop routing, such as claude-code") + pos, code, ok := serverFlags(flags, args, stdout, stderr) + if !ok { + return code + } + if len(pos) != 0 || *agent == "" { + fmt.Fprintln(stderr, "agentop server reset: want an agent: agentop server reset --agent ") + return 2 + } + if err := routerconfig.CheckAgent(*agent); err != nil { + fmt.Fprintf(stderr, "agentop server reset: %v\n", err) + return 2 + } + cfg, path, statsURL, err := serverTarget(*cfgPath) + if err != nil { + fmt.Fprintf(stderr, "agentop server reset: %v\n", err) + return 1 + } + c, _, err := readRouter(cfg) + if err != nil { + fmt.Fprintf(stderr, "agentop server reset: %v\n", err) + return 1 + } + if c.Agents[*agent] == "" { + fmt.Fprintf(stdout, "%s is not routed; nothing to change.\n", *agent) + return 0 + } + ch := edit.ConfigChange{Chain: "outbound", Plugin: routerName, Path: []string{"agents", *agent}} + return runServerWrite(stdout, stderr, path, statsURL, ch, writeReport{ + done: fmt.Sprintf("New %s sessions are no longer routed.", *agent), + live: runningSessionsStay, + unchanged: fmt.Sprintf("%s is not routed; nothing to change.", *agent), + }) +} + +// runningSessionsStay is what use and reset mean for sessions already running, +// said once the running proxy has the change. It holds a session by its pin or by +// the request to a server its history records. One that has sent nothing since the +// proxy started has neither, and a quiet one the store has evicted (session.max_sessions, +// least recently used first, or a configured session.ttl) has no history, so either +// looks new unless the router pinned it; hence "can be". +const runningSessionsStay = "Sessions already running stay where they are, except one that has sent nothing " + + "since the proxy last started, or that the proxy has dropped from memory (it keeps the most recently used, " + + "100 by default), which can be treated as a new one." + +// verifyRouter is the check every `agentop server` write makes of its result: the +// router entry must pass the plugin's own validation, so agentop never writes a +// config the proxy will refuse to reload. +func verifyRouter(cfg *config.Config) error { + for _, e := range cfg.Pipeline.Outbound.Plugins { + if e.Name == routerName { + if _, err := routerconfig.Decode(e.Config); err != nil { + return fmt.Errorf("%s config: %w", routerName, err) + } + } + } + return nil +} + +// writeReport is what runServerWrite says about a change, by how it landed. +type writeReport struct { + // done is the change, said once the proxy has it or will load it at start. + done string + // live is what the change means for sessions already running. Said only when + // the running proxy reloaded it: that proxy's pins and history are what hold a + // running session where it is, and a proxy that starts later has neither. + live string + // unchanged is said when there was nothing to write. + unchanged string + // note, when set, follows any of those: why the router will route nothing + // whatever was changed (see routerInactive). + note string +} + +// runServerWrite makes one change to the router entry and reports how it landed. +func runServerWrite(stdout, stderr io.Writer, path, statsURL string, ch edit.ConfigChange, r writeReport) int { + res, err := writePluginConfig(context.Background(), edit.ConfigWrite{ + Path: path, StatsURL: statsURL, Changes: []edit.ConfigChange{ch}, Verify: verifyRouter, + }) + if err != nil { + fmt.Fprintf(stderr, "agentop server: %v\n", err) + return 1 + } + say := func(lines ...string) { + for _, l := range lines { + if l != "" { + fmt.Fprintln(stdout, l) + } + } + } + shown := homeTilde(path) + switch res.Outcome { + case edit.WriteUnchanged: + say(r.unchanged, r.note) + case edit.WriteReloaded: + say(r.done, r.live, r.note) + case edit.WriteNotRunning: + next := fmt.Sprintf("Written to %s. No Cortex answered at its stats address, so this applies when the proxy next starts.", shown) + if r.live != "" { + // Not r.live: a proxy that starts holds no pins and no history. + next += " A proxy that starts knows where no session is, so it treats every session it sees as a new one, " + + "the ones running now included." + } + say(r.done, next, r.note) + case edit.WriteReloadFailed: + // WritePluginConfig has put the file back, so it does not hold the change; + // saying it was written would send the user looking for an edit that is not + // there. Where it could not, it returned an error saying so, handled above. + fmt.Fprintf(stderr, "agentop server: the change was not applied: the proxy refused it and keeps its previous configuration, "+ + "so %s was put back as it was:\n %s\n", shown, res.ReloadError) + return 1 + case edit.WriteStatusUnreachable: + // Not a refusal: nothing said no. The proxy went away, and whether it took + // the change before it did is unknown, so the file was put back to what it + // last ran, which is what it starts from if it was stopped. + fmt.Fprintf(stderr, "agentop server: the change was not confirmed: the proxy stopped answering at %s before it reported the reload, "+ + "so %s was put back as it was; check the proxy is running and try again:\n %s\n", statsURL, shown, res.ReloadError) + return 1 + case edit.WriteReloadTimedOut: + fmt.Fprintf(stderr, "agentop server: wrote %s, but the proxy reported no reload within %s; check %s/reload/status\n", + shown, edit.LocalPollDeadline, statsURL) + return 1 + } + return 0 +} + +// configuredServers is "; configured: a, b" for an error naming a missing server, +// or "" when there are none. +func configuredServers(c routerconfig.Config) string { + if len(c.Servers) == 0 { + return "" + } + return "; configured: " + strings.Join(slices.Sorted(maps.Keys(c.Servers)), ", ") +} diff --git a/cmd/agentop/cmd_server_write_test.go b/cmd/agentop/cmd_server_write_test.go new file mode 100644 index 000000000..d52c98f9a --- /dev/null +++ b/cmd/agentop/cmd_server_write_test.go @@ -0,0 +1,540 @@ +package main + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/rossoctl/cortex/cmd/agentop/edit" +) + +// fakeStats is a stats server whose /reload/status reports a reload at every poll, +// or, with failAfter > 0, a failed one from that poll on. It counts the polls. +type fakeStats struct { + srv *httptest.Server + polls atomic.Int32 + failAfter int32 +} + +func newFakeStats(t *testing.T, failAfter int32) *fakeStats { + t.Helper() + f := &fakeStats{failAfter: failAfter} + f.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/reload/status" { + http.NotFound(w, r) + return + } + n := f.polls.Add(1) + st := map[string]any{"last_success": time.Now(), "reloads_ok": 1, "reloads_failed": 0} + if f.failAfter > 0 { + st["last_success"] = time.Now().Add(-time.Hour) + if n >= f.failAfter { + st["reloads_failed"], st["last_error"] = 1, `configure "inference-router": refused` + } + } + _ = json.NewEncoder(w).Encode(st) + })) + t.Cleanup(f.srv.Close) + return f +} + +// addr is the stats address a config names for this server. +func (f *fakeStats) addr() string { return strings.TrimPrefix(f.srv.URL, "http://") } + +func readConfig(t *testing.T, path string) string { + t.Helper() + b, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + return string(b) +} + +// stubKeyPrompt stands in for the terminal: it returns key and records the server +// name it was asked for. A test that must not prompt passes key "" and checks the +// name stayed empty. +func stubKeyPrompt(t *testing.T, key string) *string { + t.Helper() + var asked string + prev := readServerKey + readServerKey = func(name string) (string, error) { asked = name; return key, nil } + t.Cleanup(func() { readServerKey = prev }) + return &asked +} + +// stubConfirm answers the replace question with answer and counts the asks. +func stubConfirm(t *testing.T, answer bool) *int { + t.Helper() + var asks int + prev := serverConfirm + serverConfirm = func(io.Writer) bool { asks++; return answer } + t.Cleanup(func() { serverConfirm = prev }) + return &asks +} + +func TestServerAdd_CreatesTheRouterAtTheEndOfTheChain(t *testing.T) { + stats := newFakeStats(t, 0) + path := serverEnv(t, stats.addr(), "") + code, out, errOut := runServerCmd(t, "sk-ete\n", "add", "ete", "https://ete.example.com/", "--key-stdin", "--config", path) + if code != 0 { + t.Fatalf("exit %d: %s", code, errOut) + } + if !strings.Contains(out, "Added ete. No agent uses it yet") || !strings.Contains(out, "agentop server use ete --agent claude-code") { + t.Errorf("stdout:\n%s", out) + } + want := ` - name: tool-prune + config: + remove: [] + - name: inference-router + config: + servers: + ete: + url: https://ete.example.com + key: sk-ete +` + if got := readConfig(t, path); !strings.HasSuffix(got, want) { + t.Errorf("config does not end with the new entry:\n%s", got) + } +} + +func TestServerAdd_ReadsTheKeyAtTheHiddenPrompt(t *testing.T) { + asked := stubKeyPrompt(t, "sk-typed") + path := serverEnv(t, newFakeStats(t, 0).addr(), "") + if code, _, errOut := runServerCmd(t, "", "add", "ete", "https://ete.example.com", "--config", path); code != 0 { + t.Fatalf("exit %d: %s", code, errOut) + } + if *asked != "ete" { + t.Errorf("prompted for %q, want ete", *asked) + } + if !strings.Contains(readConfig(t, path), "key: sk-typed") { + t.Error("the typed key was not written") + } +} + +func TestServerAdd_ReplacingAsksFirst(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + before := readConfig(t, path) + + asks := stubConfirm(t, false) + asked := stubKeyPrompt(t, "sk-rotated") + code, out, _ := runServerCmd(t, "", "add", "ete", "https://ete.example.com", "--config", path) + if code != exitDeclined || *asks != 1 || !strings.Contains(out, "ete is already configured (ete.example.com)") { + t.Errorf("declined: exit %d, %d asks, stdout:\n%s", code, *asks, out) + } + if *asked != "" || readConfig(t, path) != before { + t.Error("a declined replace read a key or wrote the file") + } + + code, out, errOut := runServerCmd(t, "", "add", "ete", "https://ete.example.com", "--yes", "--config", path) + if code != 0 || *asks != 1 || !strings.Contains(out, "Replaced ete.") { + t.Errorf("--yes: exit %d, %d asks, stdout:\n%s%s", code, *asks, out, errOut) + } + if got := readConfig(t, path); !strings.Contains(got, "key: sk-rotated") || strings.Contains(got, "key: sk-ete") { + t.Errorf("the key was not rotated:\n%s", got) + } +} + +func TestServerAdd_AnUnchangedServerIsNotRewritten(t *testing.T) { + stubConfirm(t, true) + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + code, out, _ := runServerCmd(t, "sk-ete", "add", "ete", "https://ete.example.com:443", "--key-stdin", "--yes", "--config", path) + if code != 0 || !strings.Contains(out, "ete already has that URL and key; nothing to change.") { + t.Errorf("exit %d, stdout:\n%s", code, out) + } +} + +func TestServerAdd_RefusesASecondServerOnOneHost(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + code, _, errOut := runServerCmd(t, "sk-x", "add", "east", "http://ETE.example.com:4000", "--key-stdin", "--config", path) + if code != 1 || !strings.Contains(errOut, "ete is already on ete.example.com") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } +} + +func TestServerAdd_RefusesABadNameOrURLBeforeAskingForAKey(t *testing.T) { + asked := stubKeyPrompt(t, "sk") + path := serverEnv(t, newFakeStats(t, 0).addr(), "") + for _, args := range [][]string{ + {"add", "ETE", "https://ete.example.com"}, + {"add", "ete", "https://ete.example.com/v1"}, + {"add", "ete", "https://ete.example.com:99999"}, + {"add", "ete"}, + } { + if code, _, _ := runServerCmd(t, "", append(args, "--config", path)...); code != 2 { + t.Errorf("%v: exit %d, want 2", args, code) + } + } + if *asked != "" { + t.Error("asked for a key for an invalid server") + } +} + +func TestServerAdd_RefusesAKeyTheConfigLoaderWouldExpand(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), "") + before := readConfig(t, path) + code, _, errOut := runServerCmd(t, "sk-$HOME", "add", "ete", "https://ete.example.com", "--key-stdin", "--config", path) + if code != 1 || !strings.Contains(errOut, "the key contains $") || strings.Contains(errOut, "sk-$HOME") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } + if readConfig(t, path) != before { + t.Error("the config was written") + } +} + +func TestServerAdd_WarnsOnPlainHTTPToAnotherMachine(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), "") + _, _, errOut := runServerCmd(t, "sk", "add", "lan", "http://10.0.0.5:4000", "--key-stdin", "--config", path) + if !strings.Contains(errOut, "plain http on another machine") { + t.Errorf("stderr:\n%s", errOut) + } +} + +// The model flags arrive with model mapping; until then an unknown flag is the +// honest answer. +func TestServerAdd_DoesNotOfferModelFlagsYet(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), "") + code, _, errOut := runServerCmd(t, "sk", "add", "glm", "https://glm.example.com", "--opus", "glm-5.3", "--key-stdin", "--config", path) + if code != 2 || !strings.Contains(errOut, "flag provided but not defined: -opus") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } +} + +func TestServerAdd_WithNoProxyRunningWritesForTheNextStart(t *testing.T) { + path := serverEnv(t, closedAddr(t), "") + code, out, errOut := runServerCmd(t, "sk-ete", "add", "ete", "https://ete.example.com", "--key-stdin", "--config", path) + if code != 0 || !strings.Contains(out, "applies when the proxy next starts") { + t.Errorf("exit %d, stdout:\n%s%s", code, out, errOut) + } + if !strings.Contains(readConfig(t, path), "key: sk-ete") { + t.Error("the config was not written") + } +} + +func TestServerAdd_ReportsAReloadTheProxyRefused(t *testing.T) { + // Poll 1 is serverTarget's probe and poll 2 the baseline; the reload fails after. + path := serverEnv(t, newFakeStats(t, 3).addr(), "") + before := readConfig(t, path) + code, _, errOut := runServerCmd(t, "sk-ete", "add", "ete", "https://ete.example.com", "--key-stdin", "--config", path) + if code != 1 || !strings.Contains(errOut, "the proxy refused it and keeps its previous configuration") || + !strings.Contains(errOut, `configure "inference-router": refused`) { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } + // The refused change was taken back out, so the message must not claim the + // file holds it, and the file must not: a refused config left on disk is what + // the proxy would next start from. + if strings.Contains(errOut, "wrote") || !strings.Contains(errOut, "was put back as it was") { + t.Errorf("want the file reported as put back, not as written:\n%s", errOut) + } + if got := readConfig(t, path); strings.Contains(got, "key: sk-ete") || got != before { + t.Errorf("the refused change is still in the config:\n%s", got) + } +} + +func TestServerRemove_RefusesAServerAnAgentIsRoutedTo(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + before := readConfig(t, path) + code, _, errOut := runServerCmd(t, "", "remove", "glm", "--config", path) + if code != 1 || !strings.Contains(errOut, "glm is claude-code's server for new sessions. Pick another first:\n agentop server use ete --agent claude-code") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } + if readConfig(t, path) != before { + t.Error("the config was written") + } +} + +func TestServerRemove_RemovesAServerNoAgentUses(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + code, out, errOut := runServerCmd(t, "", "remove", "ete", "--config", path) + if code != 0 || !strings.Contains(out, "Removed ete.") { + t.Fatalf("exit %d, stdout:\n%s%s", code, out, errOut) + } + got := readConfig(t, path) + if strings.Contains(got, "ete.example.com") || !strings.Contains(got, "glm.example.com") { + t.Errorf("want only ete gone:\n%s", got) + } +} + +func TestServerRemove_RefusesTheOnlyServer(t *testing.T) { + only := strings.Replace(routerBlock, ` glm: + url: https://glm.example.com:8443 + key: sk-glm + agents: + claude-code: glm +`, "", 1) + path := serverEnv(t, newFakeStats(t, 0).addr(), only) + code, _, errOut := runServerCmd(t, "", "remove", "ete", "--config", path) + if code != 1 || !strings.Contains(errOut, "ete is the only server") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } +} + +func TestServerRemove_NamesTheServersThatExist(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + code, _, errOut := runServerCmd(t, "", "remove", "east", "--config", path) + if code != 1 || !strings.Contains(errOut, "no server named east; configured: ete, glm") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } +} + +func TestServerUse_RoutesTheAgentsNewSessions(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + code, out, errOut := runServerCmd(t, "", "use", "ete", "--agent", "opencode", "--config", path) + // A running session stays because the proxy holds its pin or the request its + // history records: it has neither for a session it has not seen since it + // started, and no history for a quiet one its store has since evicted. + if code != 0 || out != "New opencode sessions → ete.\nSessions already running stay where they are, except one that has "+ + "sent nothing since the proxy last started, or that the proxy has dropped from memory (it keeps the most recently "+ + "used, 100 by default), which can be treated as a new one.\n" { + t.Fatalf("exit %d, stdout:\n%s%s", code, out, errOut) + } + if !strings.Contains(readConfig(t, path), " claude-code: glm\n opencode: ete\n") { + t.Errorf("config:\n%s", readConfig(t, path)) + } +} + +func TestServerUse_AnAgentAlreadyThereIsNotRewritten(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + code, out, _ := runServerCmd(t, "", "use", "glm", "--agent", "claude-code", "--config", path) + if code != 0 || !strings.Contains(out, "New claude-code sessions already go to glm.") { + t.Errorf("exit %d, stdout:\n%s", code, out) + } +} + +func TestServerUse_RefusesAServerThatIsNotConfigured(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + code, _, errOut := runServerCmd(t, "", "use", "east", "--agent", "claude-code", "--config", path) + if code != 1 || !strings.Contains(errOut, "no server named east; configured: ete, glm. Add it first:\n agentop server add east ") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } +} + +func TestServerUse_RefusesABadAgentName(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + for _, agent := range []string{"Claude Code", "unknown", ""} { + if code, _, _ := runServerCmd(t, "", "use", "ete", "--agent", agent, "--config", path); code != 2 { + t.Errorf("--agent %q: exit %d, want 2", agent, code) + } + } +} + +// use and remove check the server name first, as add does: a name no server can +// have is a usage error (exit 2) before any config is read, and the refusal quotes +// it, so a control sequence typed or pasted into the argument never reaches the +// terminal raw. Every later message names a server that passed the check. +func TestServerUseAndRemove_RefuseABadServerNameFirst(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + before := readConfig(t, path) + missing := filepath.Join(t.TempDir(), "no-such-config.yaml") + for _, args := range [][]string{ + {"use", "ETE", "--agent", "claude-code", "--config", path}, + {"use", "a\x1b[2Jb", "--agent", "claude-code", "--config", path}, + {"use", "a\x1b[2Jb", "--agent", "claude-code", "--config", missing}, + {"remove", "ETE", "--config", path}, + {"remove", "a\x1b[2Jb", "--config", path}, + {"remove", "a\x1b[2Jb", "--config", missing}, + } { + code, out, errOut := runServerCmd(t, "", args...) + if code != 2 || !strings.Contains(errOut, "is not a server name") { + t.Errorf("%q: exit %d, want 2 and the name refused; stderr:\n%s", args, code, errOut) + } + if strings.ContainsRune(out+errOut, '\x1b') { + t.Errorf("%q: a raw ESC reached the output:\n%q", args, out+errOut) + } + } + if readConfig(t, path) != before { + t.Error("the config was written") + } +} + +func TestServerReset_StopsRoutingTheAgent(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + code, out, errOut := runServerCmd(t, "", "reset", "--agent", "claude-code", "--config", path) + if code != 0 || !strings.Contains(out, "New claude-code sessions are no longer routed.") { + t.Fatalf("exit %d, stdout:\n%s%s", code, out, errOut) + } + if got := readConfig(t, path); strings.Contains(got, "agents:") { + t.Errorf("want the emptied agents map gone:\n%s", got) + } +} + +func TestServerReset_AnAgentThatIsNotRoutedChangesNothing(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + before := readConfig(t, path) + code, out, _ := runServerCmd(t, "", "reset", "--agent", "opencode", "--config", path) + if code != 0 || !strings.Contains(out, "opencode is not routed; nothing to change.") || readConfig(t, path) != before { + t.Errorf("exit %d, stdout:\n%s", code, out) + } +} + +// Every write is checked against the plugin's own rules before it lands, so a +// config the proxy would refuse is never written — here, one a hand edit already +// broke with a model the router does not accept yet. +func TestServerWrites_NeverWriteAConfigThePluginRefuses(t *testing.T) { + broken := strings.Replace(routerBlock, " key: sk-ete\n", " key: sk-ete\n opus: glm-5.3\n", 1) + path := serverEnv(t, newFakeStats(t, 0).addr(), broken) + before := readConfig(t, path) + code, _, errOut := runServerCmd(t, "", "use", "ete", "--agent", "opencode", "--config", path) + if code != 1 || !strings.Contains(errOut, "model mapping needs chained body writers") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } + if readConfig(t, path) != before { + t.Error("the config was written") + } +} + +// Replacing a server whose stored URL took the key for its host (see +// TestServer_AKeyTakenForTheHostIsNeverShown) names it without that URL. +func TestServerAdd_ReplacingShowsNothingOfARefusedStoredURL(t *testing.T) { + stubConfirm(t, true) + for _, raw := range []string{ + "https://sk-SECRET/x@h.example.com", + "https://sk-SECRET?x@h.example.com", + "https://sk-SECRET#x@h.example.com", + } { + t.Run(raw, func(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), strings.Replace(routerBlock, "https://ete.example.com", raw, 1)) + code, out, errOut := runServerCmd(t, "sk-new", "add", "ete", "https://ete.example.com", "--key-stdin", "--yes", "--config", path) + if code != 0 { + t.Fatalf("exit %d: %s%s", code, out, errOut) + } + if all := strings.ToLower(out + errOut); strings.Contains(all, "secret") { + t.Errorf("prints the stored key:\n%s%s", out, errOut) + } + if !strings.Contains(out, "ete is already configured (not a valid URL).") { + t.Errorf("stdout:\n%s", out) + } + }) + } +} + +// stubWrite stands in for edit.WritePluginConfig with a fixed answer. +func stubWrite(t *testing.T, res edit.WriteResult, err error) { + t.Helper() + prev := writePluginConfig + writePluginConfig = func(context.Context, edit.ConfigWrite) (edit.WriteResult, error) { return res, err } + t.Cleanup(func() { writePluginConfig = prev }) +} + +// A proxy that stopped answering refused nothing. The command says what happened — +// it stopped answering, and the file was put back — and exits 1, rather than +// reporting a refusal the proxy never made. +func TestServerWrites_TellAProxyThatStoppedAnsweringFromARefusal(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), routerBlock) + stubWrite(t, edit.WriteResult{Outcome: edit.WriteStatusUnreachable, RolledBack: true, + ReloadError: "reload status endpoint unreachable (is the local proxy still running?)"}, nil) + code, out, errOut := runServerCmd(t, "", "use", "ete", "--agent", "opencode", "--config", path) + if code != 1 { + t.Errorf("exit %d, want 1; stdout:\n%s", code, out) + } + if strings.Contains(errOut, "refused") || !strings.Contains(errOut, "stopped answering") || + !strings.Contains(errOut, "was put back as it was") || !strings.Contains(errOut, "is the local proxy still running?") { + t.Errorf("stderr:\n%s", errOut) + } +} + +// --key-stdin reads the key from stdin as typed, so on a terminal every character +// of the key would echo. The command refuses before asking anything, and reads and +// writes nothing. +func TestServerAdd_RefusesKeyStdinFromATerminal(t *testing.T) { + prev := stdinIsTerminal + stdinIsTerminal = func(io.Reader) bool { return true } + t.Cleanup(func() { stdinIsTerminal = prev }) + path := serverEnv(t, newFakeStats(t, 0).addr(), "") + before := readConfig(t, path) + + code, _, errOut := runServerCmd(t, "sk-typed\n", "add", "ete", "https://ete.example.com", "--key-stdin", "--config", path) + if code != 1 || !strings.Contains(errOut, "stdin is a terminal") || !strings.Contains(errOut, "Pipe the key in, or drop --key-stdin") { + t.Errorf("exit %d, stderr:\n%s", code, errOut) + } + if readConfig(t, path) != before { + t.Error("the config was written") + } +} + +// observeRouter is routerBlock with the router's entry under on_error: policy. +func observeRouter(policy string) string { + return strings.Replace(routerBlock, " - name: inference-router\n", + " - name: inference-router\n on_error: "+policy+"\n", 1) +} + +// Under on_error: observe the router moves nothing and only records would_route, +// and under off it does not run: either way nothing is routed, which use, add and +// the listing must say rather than report a route that will not happen. +func TestServer_SaysWhenTheRouterCannotRoute(t *testing.T) { + for policy, want := range map[string]string{ + "observe": "on_error: observe, so nothing is routed", + "off": "on_error: off, so it does not run and nothing is routed", + } { + t.Run(policy, func(t *testing.T) { + path := serverEnv(t, newFakeStats(t, 0).addr(), observeRouter(policy)) + _, out, _ := runServerCmd(t, "", "--config", path) + if !strings.Contains(flat(out), "✗ The "+routerName+" entry runs under "+want) { + t.Errorf("listing:\n%s", out) + } + _, out, errOut := runServerCmd(t, "", "use", "ete", "--agent", "opencode", "--config", path) + if !strings.Contains(flat(out), want) { + t.Errorf("use:\n%s%s", out, errOut) + } + _, out, errOut = runServerCmd(t, "sk-lan", "add", "lan", "http://127.0.0.1:4000", "--key-stdin", "--config", path) + if !strings.Contains(flat(out), want) { + t.Errorf("add:\n%s%s", out, errOut) + } + }) + } + path := serverEnv(t, newFakeStats(t, 0).addr(), observeRouter("enforce")) + if _, out, _ := runServerCmd(t, "", "use", "ete", "--agent", "opencode", "--config", path); strings.Contains(out, "on_error") { + t.Errorf("enforce named on_error:\n%s", out) + } +} + +// The only server, with an agent routed to it, needs two steps to remove, and the +// refusal names both at once rather than one and then, after it, the other. +func TestServerRemove_TheOnlyServerWithAnAgentNamesBothSteps(t *testing.T) { + only := strings.Replace(routerBlock, ` ete: + url: https://ete.example.com + key: sk-ete +`, "", 1) + path := serverEnv(t, newFakeStats(t, 0).addr(), only) + code, _, errOut := runServerCmd(t, "", "remove", "glm", "--config", path) + for _, want := range []string{"glm is the only server", "agentop server reset --agent claude-code", "delete its entry"} { + if !strings.Contains(flat(errOut), want) { + t.Errorf("stderr lacks %q:\n%s", want, errOut) + } + } + if code != 1 { + t.Errorf("exit %d, want 1", code) + } +} + +// A proxy that is not running holds no pins and, once started, no history, so it +// routes every session it then sees as a new one. Written for the next start, the +// change must not promise that running sessions stay where they are. +func TestServerWrites_ForTheNextStartPromiseNoRunningSessionStays(t *testing.T) { + for _, args := range [][]string{ + {"use", "ete", "--agent", "opencode"}, + {"reset", "--agent", "claude-code"}, + {"remove", "ete"}, + } { + t.Run(args[0], func(t *testing.T) { + path := serverEnv(t, closedAddr(t), routerBlock) + code, out, errOut := runServerCmd(t, "", append(args, "--config", path)...) + if code != 0 || !strings.Contains(out, "applies when the proxy next starts") { + t.Fatalf("exit %d, stdout:\n%s%s", code, out, errOut) + } + if strings.Contains(out, "stay where they are") || strings.Contains(out, "now gets an error") { + t.Errorf("promises what a restart does not keep:\n%s", out) + } + if !strings.Contains(flat(out), "treats every session it sees as a new one, the ones running now included") { + t.Errorf("does not say what the next start does:\n%s", out) + } + }) + } +} diff --git a/cmd/agentop/edit/pluginconfig.go b/cmd/agentop/edit/pluginconfig.go new file mode 100644 index 000000000..0be679c83 --- /dev/null +++ b/cmd/agentop/edit/pluginconfig.go @@ -0,0 +1,429 @@ +package edit + +import ( + "errors" + "fmt" + "reflect" + "strings" + + "gopkg.in/yaml.v3" +) + +// ConfigChange is one edit to the config: block of a named plugin entry in a +// Cortex config file. It knows nothing about any plugin: what a value means, and +// whether it is allowed, is the caller's to decide. +type ConfigChange struct { + // Chain is the pipeline chain the entry is in: "outbound" or "inbound". + Chain string + // Plugin is the entry's name:. + Plugin string + // Path is the key path below config:, outermost first, such as + // {"agents", "claude-code"}. At least one key. + Path []string + // Value is the value to set: a scalar, or a mapping of scalars and mappings + // (see ScalarValue and MapValue). Nil removes the key, and with it every + // mapping the removal leaves empty, config: included — and every comment inside + // what it removes. + Value *yaml.Node + // CreatePlugin appends an entry for Plugin at the end of Chain when the chain + // has none. Without it a missing entry is an error. A removal never creates one. + CreatePlugin bool +} + +// ScalarValue is a string value for ConfigChange.Value. It is written as a YAML +// string whatever it looks like, so "123" or "true" stay strings. +func ScalarValue(s string) *yaml.Node { + return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: s} +} + +// MapValue is a mapping of string keys to string values, in the order given: +// MapValue("url", u, "key", k). It panics on an odd number of arguments, which is a +// programming error rather than input. +func MapValue(pairs ...string) *yaml.Node { + if len(pairs)%2 != 0 { + panic("edit.MapValue: odd number of arguments") + } + n := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"} + for i := 0; i < len(pairs); i += 2 { + n.Content = append(n.Content, ScalarValue(pairs[i]), ScalarValue(pairs[i+1])) + } + return n +} + +// SetPluginConfig returns src with ch applied, and src itself when there is +// nothing to change. +// +// Positions come from the YAML parser and the edit is made to plain lines, as +// `agentop config migrate-pricing` does, so every line the change does not touch +// survives byte for byte, comments included. A change does touch the lines it +// replaces or removes, and their comments go with them: a replaced value's, and on +// a removal the key's whole block and every mapping the removal leaves empty, with +// any comment inside, however deep. A comment above a removed key stays, since it may +// be about what follows. Round-tripping the document through a YAML encoder would +// reflow it and drop the comments the local config ships with. +// +// It edits indented (block) YAML. A mapping it has to descend into, or a chain's +// plugins list, written in flow style ({...} or [...]) is refused with an error +// saying so; a value it replaces whole may be in any style. +func SetPluginConfig(src []byte, ch ConfigChange) ([]byte, error) { + if len(ch.Path) == 0 { + return nil, errors.New("edit: a config change needs at least one key") + } + // Refuse a src holding U+0085 (NEL), U+2028 (LS), U+2029 (PS), or a \r not + // immediately followed by \n, wherever it is: in a comment or a quoted value as + // much as between lines. The parser counts each as a line break and the split + // below does not, so after one, every yaml.Node.Line points further down lines + // than the line it was read from, and an edit lands there. A CRLF passes: both + // count it as one break. + if b := uncountedBreak(string(src)); b != "" { + return nil, fmt.Errorf("edit: the config contains %s, which YAML counts as a line break and this editor does not; "+ + "remove it or make it a newline, then try again", b) + } + var doc yaml.Node + if err := yaml.Unmarshal(src, &doc); err != nil { + return nil, fmt.Errorf("reading the config's structure: %w", err) + } + if len(doc.Content) == 0 || !blockMapping(doc.Content[0]) { + return nil, errors.New("the config is not an indented mapping; not editing it") + } + lines := strings.Split(string(src), "\n") + + plugins, err := chainPlugins(doc.Content[0], ch) + if err != nil { + return nil, err + } + item, err := findEntry(lines, plugins, ch.Plugin) + if err != nil { + return nil, err + } + + var out []string + switch { + case item == nil && ch.Value == nil: + return src, nil + case item == nil && !ch.CreatePlugin: + return nil, fmt.Errorf("pipeline.%s has no %s entry", ch.Chain, ch.Plugin) + case item == nil: + out, err = appendEntry(lines, plugins, ch) + default: + out, err = editEntry(lines, item, ch) + } + if err != nil { + return nil, err + } + if out == nil { + return src, nil + } + return []byte(strings.Join(out, "\n")), nil +} + +// pair is a key node and its value node. +type pair struct{ k, v *yaml.Node } + +// chainPlugins is the plugins: list of pipeline.. +func chainPlugins(root *yaml.Node, ch ConfigChange) (*yaml.Node, error) { + _, pipe := mappingEntry(root, "pipeline") + _, chain := mappingEntry(pipe, ch.Chain) + _, plugins := mappingEntry(chain, "plugins") + if plugins == nil || plugins.Kind != yaml.SequenceNode || plugins.Style&yaml.FlowStyle != 0 || len(plugins.Content) == 0 { + return nil, fmt.Errorf("pipeline.%s.plugins is not an indented list with at least one entry; add the %s entry by hand", + ch.Chain, ch.Plugin) + } + return plugins, nil +} + +// findEntry is the plugins list item named name, or nil when there is none. +func findEntry(lines []string, plugins *yaml.Node, name string) (*yaml.Node, error) { + var found *yaml.Node + for _, item := range plugins.Content { + if item.Kind == yaml.ScalarNode && item.Value == name { + return nil, fmt.Errorf("the %s entry is written as a bare name; write it as `- name: %s` to give it config", name, name) + } + if item.Kind != yaml.MappingNode { + continue + } + if _, n := mappingEntry(item, "name"); n == nil || n.Value != name { + continue + } + if found != nil { + return nil, fmt.Errorf("two %s entries in one chain; edit the config by hand", name) + } + if item.Style&yaml.FlowStyle != 0 || !strings.HasPrefix(strings.TrimSpace(lines[item.Line-1]), "-") { + return nil, fmt.Errorf("the %s entry is not written as `- name: %s` with its keys indented below; edit it by hand", name, name) + } + found = item + } + return found, nil +} + +// appendEntry adds `- name: ` and its config: after the chain's last entry, +// at the indentation that entry uses. +func appendEntry(lines []string, plugins *yaml.Node, ch ConfigChange) ([]string, error) { + dash := plugins.Column - 1 + last := plugins.Content[len(plugins.Content)-1] + keys := dash + 2 + if last.Kind == yaml.MappingNode { + keys = last.Column - 1 + } + name, err := scalarText(ch.Plugin) + if err != nil { + return nil, err + } + body, err := renderPath(ch.Path, ch.Value, keys+2) + if err != nil { + return nil, err + } + add := append([]string{ + pad(dash) + "-" + pad(keys-dash-1) + "name: " + name, + pad(keys) + "config:", + }, body...) + end := blockEnd(lines, last.Line-1, dash, false) + return splice(lines, end+1, end+1, add), nil +} + +// editEntry applies ch inside an existing entry, adding config: when it has none. +func editEntry(lines []string, item *yaml.Node, ch ConfigChange) ([]string, error) { + ck, cv := mappingEntry(item, "config") + if ck == nil { + if ch.Value == nil { + return nil, nil + } + keys := item.Column - 1 + body, err := renderPath(ch.Path, ch.Value, keys+2) + if err != nil { + return nil, err + } + end := blockEnd(lines, item.Line-1, indentOf(lines[item.Line-1]), false) + return splice(lines, end+1, end+1, append([]string{pad(keys) + "config:"}, body...)), nil + } + return setIn(lines, []pair{{ck, cv}}, ch.Path, ch.Value, ch.Plugin) +} + +// setIn applies the change below the mapping that is the value of the last pair in +// chain. chain runs from config: down, so a removal can take the mappings it +// empties with it. A nil result means nothing changed. +func setIn(lines []string, chain []pair, path []string, value *yaml.Node, plugin string) ([]string, error) { + parent := chain[len(chain)-1] + if emptyValue(lines, parent) { + if value == nil { + return nil, nil + } + body, err := renderPath(path, value, parent.k.Column-1+2) + if err != nil { + return nil, err + } + return splice(lines, parent.k.Line, parent.k.Line, body), nil + } + if !blockMapping(parent.v) { + return nil, fmt.Errorf("%s %s is not an indented mapping; rewrite it as one, or edit it by hand", plugin, dotted(chain)) + } + + k, v := mappingEntry(parent.v, path[0]) + if k == nil { + if value == nil { + return nil, nil + } + body, err := renderPath(path, value, parent.v.Column-1) + if err != nil { + return nil, err + } + end := valueEnd(lines, parent) + return splice(lines, end+1, end+1, body), nil + } + if len(path) > 1 { + return setIn(lines, append(chain, pair{k, v}), path[1:], value, plugin) + } + + if value == nil { + // Remove the highest mapping that holds nothing but the way down to this key. + top := pair{k, v} + for i := len(chain) - 1; i >= 0 && len(chain[i].v.Content) == 2; i-- { + top = chain[i] + } + return splice(lines, top.k.Line-1, valueEnd(lines, top)+1, nil), nil + } + if sameValue(v, value) { + return nil, nil + } + body, err := renderPath(path, value, k.Column-1) + if err != nil { + return nil, err + } + return splice(lines, k.Line-1, valueEnd(lines, pair{k, v})+1, body), nil +} + +// renderPath writes path as nested keys at indent, the last holding value. +func renderPath(path []string, value *yaml.Node, indent int) ([]string, error) { + key, err := scalarText(path[0]) + if err != nil { + return nil, err + } + if len(path) == 1 { + return renderKey(key, value, indent) + } + rest, err := renderPath(path[1:], value, indent+2) + if err != nil { + return nil, err + } + return append([]string{pad(indent) + key + ":"}, rest...), nil +} + +// renderKey writes `key: value`, or `key:` and value's pairs indented below it. +func renderKey(key string, value *yaml.Node, indent int) ([]string, error) { + switch value.Kind { + case yaml.ScalarNode: + s, err := scalarText(value.Value) + if err != nil { + return nil, err + } + return []string{pad(indent) + key + ": " + s}, nil + case yaml.MappingNode: + if len(value.Content) == 0 { + return nil, errors.New("edit: cannot write an empty mapping; it would read back as null") + } + out := []string{pad(indent) + key + ":"} + for i := 0; i+1 < len(value.Content); i += 2 { + k, err := scalarText(value.Content[i].Value) + if err != nil { + return nil, err + } + child, err := renderKey(k, value.Content[i+1], indent+2) + if err != nil { + return nil, err + } + out = append(out, child...) + } + return out, nil + } + return nil, fmt.Errorf("edit: cannot write a YAML value of kind %d; only scalars and mappings", value.Kind) +} + +// scalarText is s as a one-line YAML string, quoted only when it has to be. The +// error never repeats s: the values written here include API keys. +// +// It refuses s holding any character YAML counts as a line break: \n, \r, U+0085 +// (NEL), U+2028 (LS) or U+2029 (PS). yaml.v3 would write a \n as a block over +// several lines, and an LS or PS raw, which the next SetPluginConfig would refuse +// the file for; it would escape a \r or NEL, but those are refused as well, so no +// value carries a line break of any kind. +func scalarText(s string) (string, error) { + if strings.ContainsAny(s, "\n\r\u0085\u2028\u2029") { + return "", errors.New("edit: a value with a line break cannot be written on one line") + } + b, err := yaml.Marshal(ScalarValue(s)) + if err != nil { + return "", err + } + return strings.TrimSuffix(string(b), "\n"), nil +} + +// uncountedBreak names the first character in src that the YAML parser counts as +// a line break and strings.Split(src, "\n") does not split on, or is "" when there +// is none. Those are U+0085 (NEL), U+2028 (LS), U+2029 (PS), and a \r not +// immediately followed by \n. The name is all an error may say: src can hold API keys. +func uncountedBreak(src string) string { + for i, r := range src { + switch { + case r == '\u0085': + return "U+0085 (NEXT LINE)" + case r == '\u2028': + return "U+2028 (LINE SEPARATOR)" + case r == '\u2029': + return "U+2029 (PARAGRAPH SEPARATOR)" + case r == '\r' && !strings.HasPrefix(src[i+1:], "\n"): + return "a carriage return not followed by a newline" + } + } + return "" +} + +// sameValue reports whether the node in the file already holds want, whatever its +// style: `{url: x, key: y}` holds what an indented url and key hold. +func sameValue(have, want *yaml.Node) bool { + var a, b any + if have.Decode(&a) != nil || want.Decode(&b) != nil { + return false + } + return reflect.DeepEqual(a, b) +} + +// valueEnd is the index of the last line of p's value. +func valueEnd(lines []string, p pair) int { + compactList := p.v.Kind == yaml.SequenceNode && p.v.Style&yaml.FlowStyle == 0 + return blockEnd(lines, p.k.Line-1, p.k.Column-1, compactList) +} + +// blockEnd is the index of the last line of the block that starts at lines[start]: +// every following line indented deeper than indent, and with listAtIndent also the +// `- ` items of a list written at indent itself (`key:` then `- a` below it, which +// YAML allows). Blank lines and comments shallower than the block are passed over +// rather than counted, so a comment introducing the next key stays with that key. +func blockEnd(lines []string, start, indent int, listAtIndent bool) int { + end := start + for i := start + 1; i < len(lines); i++ { + t := strings.TrimSpace(lines[i]) + if t == "" { + continue + } + ind := indentOf(lines[i]) + inside := ind > indent || (listAtIndent && ind == indent && (t == "-" || strings.HasPrefix(t, "- "))) + if !inside { + if strings.HasPrefix(t, "#") { + continue + } + break + } + end = i + } + return end +} + +// emptyValue reports whether p's key has no value at all — `agents:` with nothing +// after it but a comment — so children can go on the following lines. +func emptyValue(lines []string, p pair) bool { + if p.v.Kind != yaml.ScalarNode || p.v.Tag != "!!null" { + return false + } + line := lines[p.k.Line-1] + rest := line[min(len(line), p.k.Column-1+len(p.k.Value)):] + rest = strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(rest), ":")) + return rest == "" || strings.HasPrefix(rest, "#") +} + +func blockMapping(n *yaml.Node) bool { + return n != nil && n.Kind == yaml.MappingNode && n.Style&yaml.FlowStyle == 0 +} + +// mappingEntry is the key and value nodes for key in m, or nils. m may be nil. +func mappingEntry(m *yaml.Node, key string) (k, v *yaml.Node) { + if m == nil || m.Kind != yaml.MappingNode { + return nil, nil + } + for i := 0; i+1 < len(m.Content); i += 2 { + if m.Content[i].Value == key { + return m.Content[i], m.Content[i+1] + } + } + return nil, nil +} + +// dotted names the mapping chain ends at, for errors: config.agents. +func dotted(chain []pair) string { + keys := make([]string, len(chain)) + for i, p := range chain { + keys[i] = p.k.Value + } + return strings.Join(keys, ".") +} + +// splice replaces lines[from:to] with add. +func splice(lines []string, from, to int, add []string) []string { + out := make([]string, 0, len(lines)-(to-from)+len(add)) + out = append(out, lines[:from]...) + out = append(out, add...) + return append(out, lines[to:]...) +} + +func indentOf(line string) int { return len(line) - len(strings.TrimLeft(line, " ")) } + +func pad(n int) string { return strings.Repeat(" ", n) } diff --git a/cmd/agentop/edit/pluginconfig_test.go b/cmd/agentop/edit/pluginconfig_test.go new file mode 100644 index 000000000..193c6883b --- /dev/null +++ b/cmd/agentop/edit/pluginconfig_test.go @@ -0,0 +1,440 @@ +package edit + +import ( + "strings" + "testing" + + "gopkg.in/yaml.v3" +) + +// localConfig is shaped like the config a laptop install writes: comments, the +// pipeline's outbound chain with tool-prune last, and a top-level key after the +// pipeline, so an entry appended past the chain's end would land in the wrong block. +const localConfig = `# The running proxy watches this file. +mode: proxy-sidecar +stats: + address: 127.0.0.1:47602 +pipeline: + outbound: + plugins: + - name: inference-parser + # Keep it last: it rewrites the request body. + - name: tool-prune + on_error: enforce + config: + remove: [] +session: + client_affinity: true +` + +// withRouter is localConfig after agentop added a server and routed an agent, with +// comments a person added since. +const withRouter = `# The running proxy watches this file. +mode: proxy-sidecar +stats: + address: 127.0.0.1:47602 +pipeline: + outbound: + plugins: + - name: inference-parser + # Keep it last: it rewrites the request body. + - name: tool-prune + on_error: enforce + config: + remove: [] + - name: inference-router + # servers Claude Code can be sent to + config: + servers: + ete: + url: https://ete.example.com + key: sk-ete # rotated in October + # only agents listed here are routed + agents: + claude-code: ete +session: + client_affinity: true +` + +func change(path []string, value ...string) ConfigChange { + ch := ConfigChange{Chain: "outbound", Plugin: "inference-router", Path: path} + switch len(value) { + case 0: + case 1: + ch.Value = ScalarValue(value[0]) + default: + ch.Value = MapValue(value...) + } + return ch +} + +func apply(t *testing.T, src string, ch ConfigChange) string { + t.Helper() + out, err := SetPluginConfig([]byte(src), ch) + if err != nil { + t.Fatalf("SetPluginConfig: %v", err) + } + return string(out) +} + +func assertYAML(t *testing.T, got, want string) { + t.Helper() + if got != want { + t.Errorf("result differs\n--- got ---\n%s\n--- want ---\n%s\n--- diff ---\n%s", got, want, Diff([]byte(want), []byte(got))) + } +} + +func TestSetPluginConfig_CreatesTheEntryAtTheEndOfTheChain(t *testing.T) { + ch := change([]string{"servers", "ete"}, "url", "https://ete.example.com", "key", "sk-ete") + ch.CreatePlugin = true + assertYAML(t, apply(t, localConfig, ch), `# The running proxy watches this file. +mode: proxy-sidecar +stats: + address: 127.0.0.1:47602 +pipeline: + outbound: + plugins: + - name: inference-parser + # Keep it last: it rewrites the request body. + - name: tool-prune + on_error: enforce + config: + remove: [] + - name: inference-router + config: + servers: + ete: + url: https://ete.example.com + key: sk-ete +session: + client_affinity: true +`) +} + +func TestSetPluginConfig_AMissingEntryIsAnErrorUnlessAskedToCreateIt(t *testing.T) { + _, err := SetPluginConfig([]byte(localConfig), change([]string{"agents", "claude-code"}, "ete")) + if err == nil || !strings.Contains(err.Error(), "pipeline.outbound has no inference-router entry") { + t.Fatalf("err = %v, want a missing-entry error", err) + } +} + +func TestSetPluginConfig_AddsAKeyAfterItsSiblingsAndKeepsComments(t *testing.T) { + assertYAML(t, apply(t, withRouter, change([]string{"servers", "glm"}, "url", "https://glm.example.com:8443", "key", "sk-glm")), + strings.Replace(withRouter, ` key: sk-ete # rotated in October +`, ` key: sk-ete # rotated in October + glm: + url: https://glm.example.com:8443 + key: sk-glm +`, 1)) +} + +func TestSetPluginConfig_CreatesAMissingMap(t *testing.T) { + src := strings.Replace(withRouter, ` # only agents listed here are routed + agents: + claude-code: ete +`, "", 1) + assertYAML(t, apply(t, src, change([]string{"agents", "opencode"}, "ete")), + strings.Replace(src, ` key: sk-ete # rotated in October +`, ` key: sk-ete # rotated in October + agents: + opencode: ete +`, 1)) +} + +func TestSetPluginConfig_ReplacesAScalar(t *testing.T) { + assertYAML(t, apply(t, withRouter, change([]string{"agents", "claude-code"}, "glm")), + strings.Replace(withRouter, "claude-code: ete", "claude-code: glm", 1)) +} + +func TestSetPluginConfig_ReplacesAMapWhole(t *testing.T) { + assertYAML(t, apply(t, withRouter, change([]string{"servers", "ete"}, "url", "https://ete.example.com", "key", "sk-new")), + strings.Replace(withRouter, " key: sk-ete # rotated in October\n", " key: sk-new\n", 1)) +} + +func TestSetPluginConfig_ReplacesAFlowStyleValueWhole(t *testing.T) { + src := strings.Replace(withRouter, ` ete: + url: https://ete.example.com + key: sk-ete # rotated in October +`, ` ete: { url: https://ete.example.com, key: sk-ete } +`, 1) + assertYAML(t, apply(t, src, change([]string{"servers", "ete"}, "url", "https://ete.example.com", "key", "sk-new")), + strings.Replace(src, " ete: { url: https://ete.example.com, key: sk-ete }\n", ` ete: + url: https://ete.example.com + key: sk-new +`, 1)) +} + +// Equal values are no change, in whatever style the file holds them — so a write +// that changes nothing writes nothing. +func TestSetPluginConfig_AnEqualValueChangesNothing(t *testing.T) { + for name, src := range map[string]string{ + "indented": withRouter, + "flow": strings.Replace(withRouter, ` ete: + url: https://ete.example.com + key: sk-ete # rotated in October +`, ` ete: {key: sk-ete, url: https://ete.example.com} +`, 1), + } { + t.Run(name, func(t *testing.T) { + if got := apply(t, src, change([]string{"servers", "ete"}, "url", "https://ete.example.com", "key", "sk-ete")); got != src { + t.Errorf("an equal value changed the file:\n%s", Diff([]byte(src), []byte(got))) + } + if got := apply(t, src, change([]string{"agents", "claude-code"}, "ete")); got != src { + t.Errorf("an equal scalar changed the file:\n%s", Diff([]byte(src), []byte(got))) + } + }) + } +} + +func TestSetPluginConfig_RemovesAKey(t *testing.T) { + src := strings.Replace(withRouter, " claude-code: ete\n", " claude-code: ete\n opencode: ete\n", 1) + assertYAML(t, apply(t, src, change([]string{"agents", "opencode"})), withRouter) +} + +// A map left empty by a removal goes with it; the comment above it stays, since it +// may be about what follows. +func TestSetPluginConfig_RemovingTheLastKeyRemovesItsMap(t *testing.T) { + assertYAML(t, apply(t, withRouter, change([]string{"agents", "claude-code"})), + strings.Replace(withRouter, ` agents: + claude-code: ete +`, "", 1)) +} + +// A removal takes the comments inside what it removes. Removing the last agent +// removes agents:, and a comment under it goes too, however deep; only the comment +// above it stays. Pinned so the docs can say exactly which comments survive. +func TestSetPluginConfig_ARemovalTakesTheCommentsInsideWhatItRemoves(t *testing.T) { + src := strings.Replace(withRouter, " claude-code: ete\n", + " # claude-code stays on ete until glm is ready\n claude-code: ete # see above\n", 1) + assertYAML(t, apply(t, src, change([]string{"agents", "claude-code"})), + strings.Replace(withRouter, ` agents: + claude-code: ete +`, "", 1)) +} + +func TestSetPluginConfig_RemovingWhatIsAbsentChangesNothing(t *testing.T) { + for name, tc := range map[string]struct { + src string + ch ConfigChange + }{ + "absent key": {withRouter, change([]string{"agents", "opencode"})}, + "absent map": {withRouter, change([]string{"models", "opus"})}, + "absent entry": {localConfig, change([]string{"agents", "claude-code"})}, + } { + t.Run(name, func(t *testing.T) { + if got := apply(t, tc.src, tc.ch); got != tc.src { + t.Errorf("removing nothing changed the file:\n%s", Diff([]byte(tc.src), []byte(got))) + } + }) + } +} + +// A compact list — `remove:` with its items at the key's own indent — ends where +// its items do, and removing it empties config:, which goes too. +func TestSetPluginConfig_RemovesACompactListAndTheConfigItEmpties(t *testing.T) { + src := strings.Replace(localConfig, " remove: []\n", " remove:\n - Bash\n - Read\n", 1) + ch := ConfigChange{Chain: "outbound", Plugin: "tool-prune", Path: []string{"remove"}} + assertYAML(t, apply(t, src, ch), strings.Replace(localConfig, ` config: + remove: [] +`, "", 1)) +} + +func TestSetPluginConfig_GivesAnEntryWithNoConfigOne(t *testing.T) { + src := strings.Replace(localConfig, ` config: + remove: [] +`, "", 1) + ch := ConfigChange{Chain: "outbound", Plugin: "tool-prune", Path: []string{"remove"}, Value: ScalarValue("Bash")} + assertYAML(t, apply(t, src, ch), strings.Replace(src, " on_error: enforce\n", + " on_error: enforce\n config:\n remove: Bash\n", 1)) +} + +func TestSetPluginConfig_FillsAnEmptyConfig(t *testing.T) { + src := strings.Replace(localConfig, " config:\n remove: []\n", " config: # nothing yet\n", 1) + ch := ConfigChange{Chain: "outbound", Plugin: "tool-prune", Path: []string{"paths", "messages"}, Value: ScalarValue("/v1/messages")} + assertYAML(t, apply(t, src, ch), strings.Replace(src, " config: # nothing yet\n", + " config: # nothing yet\n paths:\n messages: /v1/messages\n", 1)) +} + +// A whole document indented is valid YAML and turned up in a real config; positions +// come from the parser, so the new entry follows the file's own columns. +func TestSetPluginConfig_FollowsTheFilesOwnIndentation(t *testing.T) { + src := " pipeline:\n outbound:\n plugins:\n - name: tool-prune\n" + ch := change([]string{"agents", "claude-code"}, "ete") + ch.CreatePlugin = true + assertYAML(t, apply(t, src, ch), + " pipeline:\n outbound:\n plugins:\n - name: tool-prune\n - name: inference-router\n config:\n agents:\n claude-code: ete\n") +} + +func TestSetPluginConfig_QuotesWhatWouldNotReadBackAsAString(t *testing.T) { + ch := change([]string{"agents", "123"}, "true") + got := apply(t, withRouter, ch) + if !strings.Contains(got, ` "123": "true"`+"\n") { + t.Errorf("want the key and value quoted:\n%s", got) + } +} + +func TestSetPluginConfig_RefusesWhatItCannotEditLineByLine(t *testing.T) { + for name, tc := range map[string]struct { + src, want string + }{ + "flow-style config": { + strings.Replace(withRouter, ` config: + servers: + ete: + url: https://ete.example.com + key: sk-ete # rotated in October + # only agents listed here are routed + agents: + claude-code: ete +`, " config: {servers: {ete: {url: https://ete.example.com, key: k}}}\n", 1), + "inference-router config is not an indented mapping", + }, + "bare-name entry": { + strings.Replace(localConfig, " - name: tool-prune\n on_error: enforce\n config:\n remove: []\n", + " - tool-prune\n - inference-router\n", 1), + "written as a bare name", + }, + "flow-style plugins list": { + "pipeline:\n outbound:\n plugins: [inference-parser]\n", + "pipeline.outbound.plugins is not an indented list", + }, + "no pipeline": {"mode: proxy-sidecar\n", "pipeline.outbound.plugins is not an indented list"}, + "two entries": { + strings.Replace(withRouter, " - name: tool-prune\n", " - name: inference-router\n - name: tool-prune\n", 1), + "two inference-router entries", + }, + } { + t.Run(name, func(t *testing.T) { + ch := change([]string{"agents", "opencode"}, "ete") + ch.CreatePlugin = true + _, err := SetPluginConfig([]byte(tc.src), ch) + if err == nil || !strings.Contains(err.Error(), tc.want) { + t.Fatalf("err = %v, want it to contain %q", err, tc.want) + } + }) + } +} + +// Keys are written through here, so an error about a value must not print it. +func TestSetPluginConfig_ALineBreakIsRefusedWithoutEchoingTheValue(t *testing.T) { + _, err := SetPluginConfig([]byte(withRouter), change([]string{"servers", "ete"}, "url", "https://ete.example.com", "key", "sk-secret\nx")) + if err == nil || strings.Contains(err.Error(), "sk-secret") { + t.Fatalf("err = %v, want a refusal that does not contain the value", err) + } +} + +// uncountedBreaks are the characters yaml.v3 counts as a line break when it numbers +// lines and strings.Split(src, "\n") does not split on, each with the name the +// editor's refusal gives it. A lone \r is one; a CRLF is not, since both count it once. +var uncountedBreaks = []struct{ char, name string }{ + {"\u0085", "U+0085"}, + {"\u2028", "U+2028"}, + {"\u2029", "U+2029"}, + {"\r", "carriage return"}, +} + +// withUncountedBreak is withRouter with one uncounted break put where the document +// still parses, ahead of agents.claude-code: inside a quoted value, or ending a +// comment line in place of its \n. It fails the test if a result does not parse, so +// a refusal of it is the editor's guard and never the parser's. +func withUncountedBreak(t *testing.T, char string) map[string]string { + t.Helper() + srcs := map[string]string{ + "in a quoted value": strings.Replace(withRouter, "key: sk-ete #", `key: "sk-a`+char+`b" #`, 1), + "ending a comment": strings.Replace(withRouter, "# servers Claude Code can be sent to\n", "# servers Claude Code can be sent to"+char, 1), + } + for where, src := range srcs { + if src == withRouter { + t.Fatalf("%s: the fixture no longer has the line the break goes in", where) + } + var doc yaml.Node + if err := yaml.Unmarshal([]byte(src), &doc); err != nil { + t.Fatalf("%s: the fixture does not parse, so it cannot test the editor: %v", where, err) + } + } + return srcs +} + +// Keys and values are written one to a line, so every character YAML counts as a +// line break is refused in them, and the refusal does not repeat the value. +func TestSetPluginConfig_RefusesSpecialLineBreaksInValues(t *testing.T) { + for _, b := range uncountedBreaks { + t.Run(b.name, func(t *testing.T) { + ch := change([]string{"servers", "ete"}, "url", "https://ete.example.com", "key", "sk-a"+b.char+"b") + out, err := SetPluginConfig([]byte(withRouter), ch) + if err == nil { + t.Fatalf("err = nil, want the value refused; it wrote:\n%s", Diff([]byte(withRouter), out)) + } + if !strings.Contains(err.Error(), "a value with a line break") || strings.Contains(err.Error(), "sk-a") { + t.Errorf("err = %q, want the line-break refusal, without the value", err) + } + }) + } +} + +// A source holding an uncounted break anywhere is refused before it is edited, with +// an error naming the character and quoting nothing of the file. +func TestSetPluginConfig_RefusesSourceWithSpecialLineBreaks(t *testing.T) { + for _, b := range uncountedBreaks { + for where, src := range withUncountedBreak(t, b.char) { + t.Run(b.name+" "+where, func(t *testing.T) { + out, err := SetPluginConfig([]byte(src), change([]string{"agents", "opencode"}, "ete")) + if err == nil { + t.Fatalf("err = nil, want the source refused; it wrote:\n%s", Diff([]byte(src), out)) + } + msg := err.Error() + if !strings.HasPrefix(msg, "edit: ") || !strings.Contains(msg, b.name) || !strings.Contains(msg, "counts as a line break") { + t.Errorf("err = %q, want an edit: refusal naming %s", msg, b.name) + } + if strings.Contains(msg, "sk-a") || strings.Contains(msg, "servers Claude Code") || strings.Contains(msg, b.char) { + t.Errorf("err = %q quotes the source", msg) + } + }) + } + } +} + +// What the refusal prevents: past an uncounted break, each yaml.Node.Line is one +// more than the line strings.Split puts that node on, so a change to +// agents.claude-code would replace session: instead. The same document with the +// break made a \n, which both count, takes the change where it belongs, so the +// refusal is the break's alone. +func TestSetPluginConfig_SpecialLineBreaksDoNotCorruptLaterEdits(t *testing.T) { + ch := change([]string{"agents", "claude-code"}, "glm") + for _, b := range uncountedBreaks { + for where, src := range withUncountedBreak(t, b.char) { + t.Run(b.name+" "+where, func(t *testing.T) { + counted := strings.ReplaceAll(src, b.char, "\n") + assertYAML(t, apply(t, counted, ch), strings.Replace(counted, "claude-code: ete", "claude-code: glm", 1)) + + in := []byte(src) + out, err := SetPluginConfig(in, ch) + if err == nil { + t.Fatalf("the edit was made, not refused:\n%s", Diff(in, out)) + } + if out != nil || string(in) != src { + t.Errorf("a refused edit returned %d bytes or changed its input", len(out)) + } + }) + } + } +} + +// Only those breaks are refused. U+FFFD is the rune a byte-wise "\x85" decodes to, +// and a CRLF is a single break to the parser and to strings.Split alike. +func TestSetPluginConfig_RefusesOnlyTheLineBreaksItCannotCount(t *testing.T) { + t.Run("U+FFFD in a comment", func(t *testing.T) { + src := strings.Replace(withRouter, "# rotated in October", "# rotated in October \ufffd", 1) + assertYAML(t, apply(t, src, change([]string{"agents", "claude-code"}, "glm")), + strings.Replace(src, "claude-code: ete", "claude-code: glm", 1)) + }) + t.Run("U+FFFD in a value", func(t *testing.T) { + assertYAML(t, apply(t, withRouter, change([]string{"servers", "ete"}, "url", "https://ete.example.com", "key", "sk-\ufffd")), + strings.Replace(withRouter, " key: sk-ete # rotated in October\n", " key: sk-\ufffd\n", 1)) + }) + t.Run("CRLF line endings", func(t *testing.T) { + src := strings.ReplaceAll(withRouter, "\n", "\r\n") + got := apply(t, src, change([]string{"agents", "claude-code"}, "glm")) + assertYAML(t, strings.ReplaceAll(got, "\r\n", "\n"), strings.Replace(withRouter, "claude-code: ete", "claude-code: glm", 1)) + }) +} diff --git a/cmd/agentop/edit/pluginwrite.go b/cmd/agentop/edit/pluginwrite.go new file mode 100644 index 000000000..e1f19e897 --- /dev/null +++ b/cmd/agentop/edit/pluginwrite.go @@ -0,0 +1,177 @@ +package edit + +import ( + "bytes" + "context" + "fmt" + "os" + + "github.com/rossoctl/cortex/core/config" +) + +// ConfigWrite is one write of plugin config changes to a local Cortex config file. +type ConfigWrite struct { + // Path is the config file the proxy was started with and watches. + Path string + // StatsURL is the base URL of that proxy's stats server, where /reload/status + // says whether it took the write. Empty when no proxy answers there; the file is + // then written and nothing is polled. + StatsURL string + // Changes are applied in order, each to the result of the one before. + Changes []ConfigChange + // Verify, when set, checks the result after config.Load has accepted it. This is + // where a caller puts its plugin's own rules, which config.Load does not know: + // a result Verify refuses is never written. + Verify func(*config.Config) error +} + +// WriteOutcome is how a write ended once the file was, or was not, written. +type WriteOutcome int + +const ( + // WriteUnchanged: the file already held every change. Nothing was written and + // nothing polled: the reloader ignores a byte-identical file, so its + // last_success would never move and the poll would wait out its deadline. + WriteUnchanged WriteOutcome = iota + // WriteReloaded: written, and the proxy reloaded it. + WriteReloaded + // WriteReloadFailed: written, and the proxy refused it, keeping the + // configuration it had. ReloadError says why. The file is put back as it was + // (RolledBack), since a refused config left on disk is what the proxy next starts + // from, and it may not start from it. + WriteReloadFailed + // WriteReloadTimedOut: written, and the proxy reported neither a reload nor a + // failure within LocalPollDeadline. The file is left as written; the reload may + // be slow or the state uncertain. + WriteReloadTimedOut + // WriteNotRunning: StatsURL was empty (no proxy to poll). The file was written. + WriteNotRunning + // WriteStatusUnreachable: written, and then the proxy stopped answering at + // StatsURL before it reported a reload or a refusal. Nothing refused the change; + // whether the proxy took it is unknown. The file is put back as after a refusal + // (RolledBack), as the TUI's RollbackCmd does: a proxy that went away next + // starts from the file, and this keeps that the configuration it last ran. + // ReloadError is the poller's account of what failed. + WriteStatusUnreachable +) + +// WriteResult is what WritePluginConfig reports. +type WriteResult struct { + Outcome WriteOutcome + // ReloadError is the proxy's error for WriteReloadFailed, and the poller's for + // WriteStatusUnreachable. + ReloadError string + // RolledBack reports, for those two outcomes, that the file was put back as it + // was. It is false beside an error saying why it was not. + RolledBack bool +} + +// WritePluginConfig applies w.Changes to the file at w.Path and waits for the proxy +// to reload it, for at most LocalPollDeadline or until ctx is done. +// +// An error from before the write means nothing was written: a change cannot be +// made, the result does not load or fails Verify, or the file changed while it +// worked. Once the file is written the error is nil and the WriteResult says what +// the proxy did, with one exception: the proxy refused the change or stopped +// answering, and the file could not be put back, because someone changed it in the +// meantime (it is then left as found) or because the restore failed. The WriteResult +// still says what the proxy did, RolledBack is false, and the error says why the +// file was not put back, with ReloadError in it. No error quotes the file, which +// holds API keys. +func WritePluginConfig(ctx context.Context, w ConfigWrite) (WriteResult, error) { + orig, err := os.ReadFile(w.Path) + if err != nil { + return WriteResult{}, fmt.Errorf("read %s: %w", w.Path, err) + } + updated := orig + for _, ch := range w.Changes { + if updated, err = SetPluginConfig(updated, ch); err != nil { + return WriteResult{}, fmt.Errorf("%s: %w", w.Path, err) + } + } + if bytes.Equal(updated, orig) { + return WriteResult{Outcome: WriteUnchanged}, nil + } + if err := checkLoads(updated, w.Verify); err != nil { + return WriteResult{}, fmt.Errorf("not writing %s: %w", w.Path, err) + } + + store := FileStore{Path: w.Path} + if err := store.CheckUnchanged(ctx, &FetchedPipeline{Original: orig}); err != nil { + return WriteResult{}, err + } + applyTime, err := store.Apply(ctx, updated) + if err != nil { + return WriteResult{}, err + } + if w.StatsURL == "" { + return WriteResult{Outcome: WriteNotRunning}, nil + } + + pctx, cancel := context.WithTimeout(ctx, LocalPollDeadline) + defer cancel() + res := PollUntilReloaded(pctx, w.StatsURL, applyTime, store.Describe().UnreachableHint) + switch res.Status { + case PollSuccess: + return WriteResult{Outcome: WriteReloaded}, nil + case PollFailure: + out := WriteResult{Outcome: WriteReloadFailed, ReloadError: res.LastError} + what := "the proxy refused the change" + if res.Unreachable { + out.Outcome, what = WriteStatusUnreachable, "the proxy stopped answering before it reported the reload" + } + if err := putBack(store, updated, orig); err != nil { + return out, fmt.Errorf("%s (%s), and %w", what, res.LastError, err) + } + out.RolledBack = true + return out, nil + default: + return WriteResult{Outcome: WriteReloadTimedOut}, nil + } +} + +// putBack restores orig over written after a reload that did not take, but only +// while the file still holds exactly written. A person who edited it meanwhile, in +// an editor or another agentop, made a change of their own, which a restore would +// silently undo; that file is left as found. Its errors name the path and never the +// file's bytes. +func putBack(store FileStore, written, orig []byte) error { + current, err := os.ReadFile(store.Path) + if err != nil { + return fmt.Errorf("re-reading %s to put it back failed, so it may still hold the change; check it by hand: %w", store.Path, err) + } + if !bytes.Equal(current, written) { + return fmt.Errorf("%s changed while the proxy was reloading it, so it was left as found; "+ + "check it by hand before the proxy next starts", store.Path) + } + if _, err := store.Apply(context.Background(), orig); err != nil { + return fmt.Errorf("putting %s back failed, so it still holds the change; fix it by hand before the proxy next starts: %w", store.Path, err) + } + return nil +} + +// checkLoads reports whether b loads as a Cortex config and passes verify, as the +// proxy would load it. Checked from a private temp file, never beside the config: +// the proxy watches that directory, and b can hold API keys. +func checkLoads(b []byte, verify func(*config.Config) error) error { + f, err := os.CreateTemp("", "agentop-config-check-*.yaml") + if err != nil { + return err + } + defer os.Remove(f.Name()) + if _, err := f.Write(b); err != nil { + f.Close() + return err + } + if err := f.Close(); err != nil { + return err + } + cfg, err := config.Load(f.Name()) + if err != nil { + return fmt.Errorf("the result would not load: %w", err) + } + if verify != nil { + return verify(cfg) + } + return nil +} diff --git a/cmd/agentop/edit/pluginwrite_test.go b/cmd/agentop/edit/pluginwrite_test.go new file mode 100644 index 000000000..7c76484f6 --- /dev/null +++ b/cmd/agentop/edit/pluginwrite_test.go @@ -0,0 +1,418 @@ +package edit + +import ( + "context" + "encoding/json" + "errors" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/rossoctl/cortex/core/config" +) + +func readFile(t *testing.T, path string) string { + t.Helper() + b, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + return string(b) +} + +func routeClaudeToGLM() ConfigChange { return change([]string{"agents", "claude-code"}, "glm") } + +func TestWritePluginConfig_WritesAndReportsTheReload(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + srv := makeStatusServer(t, func() ReloadStatus { return ReloadStatus{LastSuccess: time.Now()} }) + + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: srv.URL, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if res.Outcome != WriteReloaded { + t.Errorf("outcome = %v, want WriteReloaded", res.Outcome) + } + if got, want := readFile(t, path), strings.Replace(withRouter, "claude-code: ete", "claude-code: glm", 1); got != want { + t.Errorf("file differs:\n%s", Diff([]byte(want), []byte(got))) + } + if st, _ := os.Stat(path); st.Mode().Perm() != 0o600 { + t.Errorf("mode = %v, want the file's own 0600 kept", st.Mode().Perm()) + } +} + +func TestWritePluginConfig_ReportsAReloadTheProxyRefused(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + var calls atomic.Int32 + srv := makeStatusServer(t, func() ReloadStatus { + if calls.Add(1) == 1 { + return ReloadStatus{ReloadsFailed: 2} + } + return ReloadStatus{ReloadsFailed: 3, LastError: `configure "inference-router": servers: at least one server is required`} + }) + + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: srv.URL, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if res.Outcome != WriteReloadFailed || !strings.Contains(res.ReloadError, "at least one server") { + t.Errorf("result = %+v, want WriteReloadFailed with the proxy's error", res) + } +} + +func TestWritePluginConfig_WithNoProxyWritesAndSaysSo(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if res.Outcome != WriteNotRunning { + t.Errorf("outcome = %v, want WriteNotRunning", res.Outcome) + } + if !strings.Contains(readFile(t, path), "claude-code: glm") { + t.Error("the file was not written") + } +} + +// The reloader ignores a byte-identical file, so a poll after a no-op write would +// wait out its whole deadline for a reload that never comes. +func TestWritePluginConfig_AnUnchangedValueIsNeitherWrittenNorPolled(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + before, _ := os.Stat(path) + var calls atomic.Int32 + srv := makeStatusServer(t, func() ReloadStatus { calls.Add(1); return ReloadStatus{LastSuccess: time.Now()} }) + + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: srv.URL, + Changes: []ConfigChange{change([]string{"agents", "claude-code"}, "ete")}}) + if err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if res.Outcome != WriteUnchanged { + t.Errorf("outcome = %v, want WriteUnchanged", res.Outcome) + } + if n := calls.Load(); n != 0 { + t.Errorf("polled /reload/status %d times for a write that did not happen", n) + } + if after, _ := os.Stat(path); !after.ModTime().Equal(before.ModTime()) { + t.Error("the file was rewritten") + } +} + +func TestWritePluginConfig_NeverWritesAResultThatWillNotLoad(t *testing.T) { + src := withRouter + "mtls:\n mode: sideways\n" + path := writeFixtureContent(t, src, 0o600) + _, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err == nil || !strings.Contains(err.Error(), "would not load") { + t.Fatalf("err = %v, want a load refusal", err) + } + if readFile(t, path) != src { + t.Error("a result that does not load was written") + } +} + +func TestWritePluginConfig_NeverWritesAResultVerifyRefuses(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + refuse := func(*config.Config) error { return errors.New("glm is not a server") } + _, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, Changes: []ConfigChange{routeClaudeToGLM()}, Verify: refuse}) + if err == nil || !strings.Contains(err.Error(), "glm is not a server") { + t.Fatalf("err = %v, want Verify's refusal", err) + } + if readFile(t, path) != withRouter { + t.Error("a result Verify refused was written") + } +} + +// Verify sees the config the proxy would load, so it can check the plugin's own block. +func TestWritePluginConfig_VerifySeesTheResult(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + var saw string + look := func(c *config.Config) error { + for _, e := range c.Pipeline.Outbound.Plugins { + if e.Name == "inference-router" { + saw = string(e.Config) + } + } + return nil + } + if _, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, Changes: []ConfigChange{routeClaudeToGLM()}, Verify: look}); err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if !strings.Contains(saw, `"claude-code":"glm"`) { + t.Errorf("Verify saw %s, want the changed agents block", saw) + } +} + +func TestWritePluginConfig_AChangeThatCannotBeMadeWritesNothing(t *testing.T) { + path := writeFixtureContent(t, localConfig, 0o600) + _, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err == nil || !strings.Contains(err.Error(), "has no inference-router entry") { + t.Fatalf("err = %v, want the missing-entry error", err) + } + if readFile(t, path) != localConfig { + t.Error("the file changed") + } +} + +// Changes apply in order, each to the result of the one before: add a server, then +// route an agent to it, in one write and one reload. +func TestWritePluginConfig_AppliesChangesInOrder(t *testing.T) { + path := writeFixtureContent(t, localConfig, 0o600) + add := change([]string{"servers", "ete"}, "url", "https://ete.example.com", "key", "sk-ete") + add.CreatePlugin = true + use := change([]string{"agents", "claude-code"}, "ete") + if _, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, Changes: []ConfigChange{add, use}}); err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + got := readFile(t, path) + if !strings.Contains(got, " key: sk-ete\n agents:\n claude-code: ete\n") { + t.Errorf("want the server then the agent:\n%s", got) + } +} + +// A refused reload restores the original bytes and mode, and reports RolledBack=true. +func TestWritePluginConfig_OnRefusedReloadRestoresOriginalAndReportsRolledBack(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + before := readFile(t, path) + var calls atomic.Int32 + srv := makeStatusServer(t, func() ReloadStatus { + if calls.Add(1) == 1 { + // First poll: ReloadsFailed=2 becomes the baseline + return ReloadStatus{ReloadsFailed: 2} + } + // Second poll: ReloadsFailed increments to 3, triggering failure + return ReloadStatus{ReloadsFailed: 3, LastError: `configure "inference-router": servers: at least one server is required`} + }) + + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: srv.URL, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if res.Outcome != WriteReloadFailed { + t.Errorf("outcome = %v, want WriteReloadFailed", res.Outcome) + } + if !res.RolledBack { + t.Error("RolledBack = false, want true") + } + if readFile(t, path) != before { + t.Error("file was not restored to original") + } + if st, _ := os.Stat(path); st.Mode().Perm() != 0o600 { + t.Errorf("mode = %v, want 0600 kept", st.Mode().Perm()) + } +} + +// After a rollback, writing the same change again is NOT reported unchanged. +func TestWritePluginConfig_AfterRollbackWritingSameChangeIsNotUnchanged(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + var calls atomic.Int32 + srv := makeStatusServer(t, func() ReloadStatus { + c := calls.Add(1) + if c == 1 { + // First poll of first write: ReloadsFailed=2 becomes baseline + return ReloadStatus{ReloadsFailed: 2} + } else if c == 2 { + // Second poll of first write: ReloadsFailed=3, triggers failure + return ReloadStatus{ReloadsFailed: 3, LastError: "test refusal"} + } else if c == 3 { + // First poll of second write: ReloadsFailed=3 becomes new baseline + return ReloadStatus{ReloadsFailed: 3} + } + // Subsequent polls: accept the reload + return ReloadStatus{LastSuccess: time.Now()} + }) + + // First write fails and rolls back + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: srv.URL, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("first WritePluginConfig: %v", err) + } + if res.Outcome != WriteReloadFailed { + t.Errorf("first outcome = %v, want WriteReloadFailed", res.Outcome) + } + if !res.RolledBack { + t.Error("first RolledBack = false, want true") + } + + // Second write of the same change should NOT be reported unchanged + // (the file was rolled back to original, so the change is new again) + res2, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: srv.URL, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("second WritePluginConfig: %v", err) + } + if res2.Outcome == WriteUnchanged { + t.Error("second outcome = WriteUnchanged, want WriteReloaded (file is back to original after rollback)") + } + if res2.Outcome != WriteReloaded { + t.Errorf("second outcome = %v, want WriteReloaded", res2.Outcome) + } +} + +// A timed-out reload leaves the file with the new bytes (state is uncertain). The +// deadline is the caller's ctx when that is shorter than LocalPollDeadline, so the +// test waits a fraction of a second rather than the real 30s. +func TestWritePluginConfig_OnTimeoutLeavesNewBytes(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + original := readFile(t, path) + srv := makeStatusServer(t, func() ReloadStatus { + // Never change status, so poll times out + return ReloadStatus{} + }) + + ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond) + defer cancel() + res, err := WritePluginConfig(ctx, ConfigWrite{Path: path, StatsURL: srv.URL, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if res.Outcome != WriteReloadTimedOut { + t.Errorf("outcome = %v, want WriteReloadTimedOut", res.Outcome) + } + after := readFile(t, path) + if after == original { + t.Error("file was restored; on timeout, new bytes should remain") + } + if !strings.Contains(after, "claude-code: glm") { + t.Error("file should still have the new bytes after timeout") + } +} + +// refuseOnSecondPoll is a /reload/status that takes its baseline on the first +// poll and reports a refused reload on the second, after running during — the +// moment a person's own edit or a failing disk would land. +func refuseOnSecondPoll(t *testing.T, during func()) string { + t.Helper() + var calls atomic.Int32 + return makeStatusServer(t, func() ReloadStatus { + if calls.Add(1) == 1 { + return ReloadStatus{ReloadsFailed: 2} + } + during() + return ReloadStatus{ReloadsFailed: 3, LastError: `configure "inference-router": servers: at least one server is required`} + }).URL +} + +// A person who edits the file while the proxy reloads agentop's write keeps the +// edit: the rollback puts back only what agentop wrote. The error says the file was +// left as found and why the proxy refused, and quotes nothing from the file, which +// holds API keys. +func TestWritePluginConfig_ARefusalLeavesAFileEditedDuringTheReloadAsFound(t *testing.T) { + path := writeFixtureContent(t, withRouter, 0o600) + edited := strings.Replace(withRouter, "# only agents listed here are routed", "# edited by hand meanwhile", 1) + stats := refuseOnSecondPoll(t, func() { + if err := os.WriteFile(path, []byte(edited), 0o600); err != nil { + t.Error(err) + } + }) + + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: stats, Changes: []ConfigChange{routeClaudeToGLM()}}) + if got := readFile(t, path); got != edited { + t.Errorf("the edit made during the reload was overwritten:\n%s", Diff([]byte(edited), []byte(got))) + } + if err == nil { + t.Fatalf("err = nil, result %+v; want an error saying the file was left as found", res) + } + for _, want := range []string{"changed while", "left as found", "at least one server is required"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("err = %q, want it to contain %q", err, want) + } + } + for _, leak := range []string{"sk-ete", "edited by hand", "claude-code: glm"} { + if strings.Contains(err.Error(), leak) { + t.Errorf("err = %q quotes the file (%q)", err, leak) + } + } + if res.Outcome != WriteReloadFailed || res.RolledBack || !strings.Contains(res.ReloadError, "at least one server") { + t.Errorf("result = %+v, want WriteReloadFailed with the proxy's error, not rolled back", res) + } +} + +// When the file cannot be put back after a refusal, the error still says why the +// proxy refused it, and the result still says what the proxy did. +func TestWritePluginConfig_AFailedRestoreStillSaysWhyTheProxyRefused(t *testing.T) { + if os.Geteuid() == 0 { + t.Skip("root can write into a read-only directory") + } + path := writeFixtureContent(t, withRouter, 0o600) + dir := filepath.Dir(path) + t.Cleanup(func() { _ = os.Chmod(dir, 0o700) }) + stats := refuseOnSecondPoll(t, func() { + // The restore writes a sibling temp file, which a read-only directory refuses. + if err := os.Chmod(dir, 0o500); err != nil { + t.Error(err) + } + }) + + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: stats, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err == nil { + t.Fatalf("err = nil, result %+v; want the failed restore reported", res) + } + for _, want := range []string{"at least one server is required", "by hand"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("err = %q, want it to contain %q", err, want) + } + } + if strings.Contains(err.Error(), "sk-ete") { + t.Errorf("err = %q quotes the file", err) + } + if res.Outcome != WriteReloadFailed || res.RolledBack || res.ReloadError == "" { + t.Errorf("result = %+v, want WriteReloadFailed with the proxy's error, not rolled back", res) + } + if !strings.Contains(readFile(t, path), "claude-code: glm") { + t.Error("the file does not hold the change, yet the restore was reported failed") + } +} + +// shortenPollSchedule makes PollUntilReloaded's unreachable path take milliseconds +// rather than its real ~16s of backoff. +func shortenPollSchedule(t *testing.T) { + t.Helper() + interval, max := pollInterval, pollMaxBackoff + pollInterval, pollMaxBackoff = time.Millisecond, 4*time.Millisecond + t.Cleanup(func() { pollInterval, pollMaxBackoff = interval, max }) +} + +// A proxy that answers once and then stops answering has not refused anything, +// and the result must not say it did. The file is still put back: whether the +// proxy took the change is unknown, and one that went away may next start from it. +func TestWritePluginConfig_TellsAProxyThatStoppedAnsweringFromARefusal(t *testing.T) { + shortenPollSchedule(t) + path := writeFixtureContent(t, withRouter, 0o600) + var calls atomic.Int32 + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if calls.Add(1) == 1 { + _ = json.NewEncoder(w).Encode(ReloadStatus{ReloadsFailed: 2}) + return + } + http.Error(w, "gone", http.StatusServiceUnavailable) + })) + t.Cleanup(srv.Close) + + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: srv.URL, Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if res.Outcome != WriteStatusUnreachable || !strings.Contains(res.ReloadError, "unreachable") { + t.Errorf("result = %+v: want WriteStatusUnreachable, not a refusal, with the poller's error", res) + } + if !res.RolledBack || readFile(t, path) != withRouter { + t.Errorf("result = %+v; want the file put back as it was", res) + } +} + +// A refusal is still a refusal with the schedule shortened: the two outcomes differ. +func TestWritePluginConfig_ARefusalIsStillARefusal(t *testing.T) { + shortenPollSchedule(t) + path := writeFixtureContent(t, withRouter, 0o600) + res, err := WritePluginConfig(context.Background(), ConfigWrite{Path: path, StatsURL: refuseOnSecondPoll(t, func() {}), Changes: []ConfigChange{routeClaudeToGLM()}}) + if err != nil { + t.Fatalf("WritePluginConfig: %v", err) + } + if res.Outcome != WriteReloadFailed || !res.RolledBack { + t.Errorf("result = %+v, want WriteReloadFailed, rolled back", res) + } +} diff --git a/cmd/agentop/edit/status.go b/cmd/agentop/edit/status.go index 632603eec..fca2e0ce5 100644 --- a/cmd/agentop/edit/status.go +++ b/cmd/agentop/edit/status.go @@ -31,6 +31,11 @@ const ( type PollResult struct { Status PollResultStatus LastError string // populated when Status == PollFailure + // Unreachable marks a PollFailure that is no refusal: /reload/status stopped + // answering, and LastError is the synthesized "unreachable" text. A caller + // telling the user what happened needs the difference — the proxy refused the + // change, or it went away and nobody knows whether it took it. + Unreachable bool } // baselineFailedSentinel marks the "baseline not yet captured" state for @@ -38,15 +43,19 @@ type PollResult struct { // current value so we only react to NEW failures after our apply. const baselineFailedSentinel int64 = -1 -// pollInterval is the cadence between /reload/status fetches. 1s balances -// user-visible spinner progress with not hammering the cluster on slow -// reloads. -const pollInterval = 1 * time.Second +// The poll schedule. Vars, not consts, only so a test can shorten them: the +// unreachable path below waits out the whole backoff, about 16s, before it gives up. +var ( + // pollInterval is the cadence between /reload/status fetches. 1s balances + // user-visible spinner progress with not hammering the cluster on slow + // reloads. + pollInterval = 1 * time.Second -// pollMaxBackoff caps the exponential backoff applied to consecutive -// transport errors. Keeps a flapping endpoint from getting hammered -// without making the user wait too long once it recovers. -const pollMaxBackoff = 5 * time.Second + // pollMaxBackoff caps the exponential backoff applied to consecutive + // transport errors. Keeps a flapping endpoint from getting hammered + // without making the user wait too long once it recovers. + pollMaxBackoff = 5 * time.Second +) // unreachableThreshold is the number of consecutive transport-layer // failures (network errors, non-200) after which the poller gives up @@ -124,8 +133,9 @@ func PollUntilReloaded(ctx context.Context, statusURL string, applyTime time.Tim consecErrors++ if consecErrors >= unreachableThreshold { return PollResult{ - Status: PollFailure, - LastError: "reload status endpoint unreachable (" + unreachableHint + ")", + Status: PollFailure, + LastError: "reload status endpoint unreachable (" + unreachableHint + ")", + Unreachable: true, } } if wait < pollMaxBackoff { diff --git a/cmd/agentop/edit/status_test.go b/cmd/agentop/edit/status_test.go index 068c0c05c..530593fc6 100644 --- a/cmd/agentop/edit/status_test.go +++ b/cmd/agentop/edit/status_test.go @@ -5,6 +5,7 @@ import ( "encoding/json" "net/http" "net/http/httptest" + "strings" "sync/atomic" "testing" "time" @@ -87,3 +88,33 @@ func TestPollUntilReloaded_HTTPError(t *testing.T) { t.Fatalf("status = %v, want PollTimeout", res.Status) } } + +// A refusal and an endpoint that stopped answering are both PollFailure; only the +// second is Unreachable, which is what lets a caller say which happened. +func TestPollUntilReloaded_MarksAnEndpointThatStoppedAnsweringUnreachable(t *testing.T) { + shortenPollSchedule(t) + var calls atomic.Int32 + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if calls.Add(1) == 1 { + _ = json.NewEncoder(w).Encode(ReloadStatus{}) + return + } + http.Error(w, "gone", http.StatusServiceUnavailable) + })) + defer srv.Close() + res := PollUntilReloaded(context.Background(), srv.URL, time.Now(), "") + if res.Status != PollFailure || !res.Unreachable || !strings.Contains(res.LastError, "unreachable") { + t.Fatalf("result = %+v, want an Unreachable PollFailure", res) + } + + var polls atomic.Int32 + refused := makeStatusServer(t, func() ReloadStatus { + if polls.Add(1) == 1 { + return ReloadStatus{ReloadsFailed: 0} + } + return ReloadStatus{ReloadsFailed: 1, LastError: "bad"} + }) + if res := PollUntilReloaded(context.Background(), refused.URL, time.Now(), ""); res.Status != PollFailure || res.Unreachable { + t.Fatalf("refusal = %+v, want a PollFailure that is not Unreachable", res) + } +} diff --git a/cmd/agentop/main.go b/cmd/agentop/main.go index a0d433156..a8424606d 100644 --- a/cmd/agentop/main.go +++ b/cmd/agentop/main.go @@ -40,7 +40,7 @@ var version = "dev" // One list rather than two: the unknown-subcommand error used to hardcode its own // copy, so adding a subcommand meant editing both and forgetting one left a typo // getting an incomplete list. A test holds the usage block to this slice. -var dispatchableSubcommands = []string{"observe", "setup", "doctor", "uninstall", "service", "configure", "claude-code", "exec", "tools", "pipeline", "pricing", "cost", "experimental"} +var dispatchableSubcommands = []string{"observe", "setup", "doctor", "uninstall", "service", "configure", "claude-code", "exec", "server", "tools", "pipeline", "pricing", "cost", "experimental"} // unknownSubcommandMessage is the error for an unrecognised first argument. func unknownSubcommandMessage(name string) string { @@ -66,6 +66,8 @@ Usage: bobshell, codex, opencode agentop exec -- CMD [ARG...] run CMD with Cortex's proxy and CA in its environment, for tools with no settings file + agentop server [] the inference server each agent's new sessions + use: add, remove, use, reset; alone, list them agentop tools tool-definition costs: scan agentop pipeline the plugin pipeline in effect: get agentop pricing show the model rates in effect (--host ) @@ -122,6 +124,8 @@ func main() { os.Exit(runClaudeCode(os.Args[2:], os.Stdout, os.Stderr)) case "exec": os.Exit(runExec(os.Args[2:], os.Stdout, os.Stderr)) + case "server": + os.Exit(runServer(os.Args[2:], os.Stdin, os.Stdout, os.Stderr)) case "service": os.Exit(runService(os.Args[2:], os.Stdout, os.Stderr)) case "observe": diff --git a/cmd/agentop/tui/detail_pane.go b/cmd/agentop/tui/detail_pane.go index 3bc069671..aeaa3e3fa 100644 --- a/cmd/agentop/tui/detail_pane.go +++ b/cmd/agentop/tui/detail_pane.go @@ -55,6 +55,9 @@ func (m *model) showDetail(r eventRow, resetScroll bool) { if r.tunnel != nil { content = tunnelHeader(r.tunnel) + "\n\n" + content } + if header := redirectHeader(e); header != "" { + content = header + "\n\n" + content + } if header := tlsHeader(e.TLS); header != "" { content = header + "\n\n" + content } @@ -91,6 +94,19 @@ func tunnelHeader(tunnel *pipeline.SessionEvent) string { return b.String() } +// redirectHeader names both hosts of a request a plugin sent elsewhere. The events +// table's host column already shows where it went; this line says where the client +// asked for, which is the other half of the story. Empty when nothing redirected it, +// so the caller can prepend unconditionally. +// +// redirected: ete-litellm.example.com → glm-litellm.example.com +func redirectHeader(e *pipeline.SessionEvent) string { + if e.RequestedHost == "" { + return "" + } + return fmt.Sprintf("redirected: %s → %s", e.RequestedHost, e.Host) +} + // tlsHeader builds a one-block summary of the TLS connection state. // Returns the empty string when tls is nil (plaintext events) so the // caller can prepend unconditionally. diff --git a/cmd/agentop/tui/detail_pane_test.go b/cmd/agentop/tui/detail_pane_test.go index beac21f15..bd184a9cf 100644 --- a/cmd/agentop/tui/detail_pane_test.go +++ b/cmd/agentop/tui/detail_pane_test.go @@ -147,3 +147,20 @@ func TestFilterForDetail_KeepsALoneLegacyRecord(t *testing.T) { t.Errorf("a lone legacy record was dropped, so an older proxy shows no cost: %v", pl) } } + +func TestRedirectHeader_EmptyWithoutARedirect(t *testing.T) { + if got := redirectHeader(&pipeline.SessionEvent{Host: "ete-litellm.example.com"}); got != "" { + t.Errorf("redirectHeader = %q for an event nothing redirected, want empty", got) + } +} + +func TestRedirectHeader_NamesBothHosts(t *testing.T) { + got := redirectHeader(&pipeline.SessionEvent{ + Host: "glm-litellm.example.com", + RequestedHost: "ete-litellm.example.com", + }) + want := "redirected: ete-litellm.example.com → glm-litellm.example.com" + if got != want { + t.Errorf("redirectHeader = %q, want %q", got, want) + } +} diff --git a/cmd/cortex-cpex/main.go b/cmd/cortex-cpex/main.go index 1f26169a1..cc1cca606 100644 --- a/cmd/cortex-cpex/main.go +++ b/cmd/cortex-cpex/main.go @@ -163,12 +163,17 @@ func main() { // live traffic priced from a config that was refused. Plugins only store the // resolver during Configure and never resolve through it, so building against // the old table is safe. - deps := plugins.Deps{SPIFFE: provider, Pricing: pricingRegistry} - in, err := plugins.BuildWithDeps(c.Pipeline.Inbound.Plugins, deps) + // Each chain is built for the listener that runs it, so a plugin the listener + // cannot honor fails here — on startup and on every reload — instead of + // running to no effect. The reverse proxy honors no redirect, which is what + // keeps a WritesDestination plugin off the inbound chain. + inDeps := plugins.Deps{SPIFFE: provider, Pricing: pricingRegistry, Listener: reverseproxy.Support()} + outDeps := plugins.Deps{SPIFFE: provider, Pricing: pricingRegistry, Listener: forwardproxy.Support(c.MTLS != nil)} + in, err := plugins.BuildWithDeps(c.Pipeline.Inbound.Plugins, inDeps) if err != nil { return nil, nil, nil, fmt.Errorf("inbound: %w", err) } - out, err := plugins.BuildWithDeps(c.Pipeline.Outbound.Plugins, deps) + out, err := plugins.BuildWithDeps(c.Pipeline.Outbound.Plugins, outDeps) if err != nil { return nil, nil, nil, fmt.Errorf("outbound: %w", err) } diff --git a/cmd/cortex-envoy/main.go b/cmd/cortex-envoy/main.go index 2361d848a..e2d91d83e 100644 --- a/cmd/cortex-envoy/main.go +++ b/cmd/cortex-envoy/main.go @@ -185,7 +185,10 @@ func main() { // live traffic priced from a config that was refused. Plugins only store the // resolver during Configure and never resolve through it, so building against // the old table is safe. - deps := plugins.Deps{SPIFFE: provider, Pricing: pricingRegistry} + // ext_proc answers Envoy with header and body mutations only, so it honors no + // redirect on either chain; a WritesDestination plugin fails the build here + // instead of recording redirects that never happen. + deps := plugins.Deps{SPIFFE: provider, Pricing: pricingRegistry, Listener: extproc.Support()} in, err := plugins.BuildWithDeps(c.Pipeline.Inbound.Plugins, deps) if err != nil { return nil, nil, nil, fmt.Errorf("inbound: %w", err) diff --git a/cmd/cortex/main.go b/cmd/cortex/main.go index 333e854c1..0579d821d 100644 --- a/cmd/cortex/main.go +++ b/cmd/cortex/main.go @@ -506,12 +506,17 @@ func main() { // live traffic priced from a config that was refused. Plugins only store the // resolver during Configure and never resolve through it, so building against // the old table is safe. - deps := plugins.Deps{SPIFFE: provider, Pricing: pricingRegistry} - in, err := plugins.BuildWithDeps(c.Pipeline.Inbound.Plugins, deps) + // Each chain is built for the listener that runs it, so a plugin the listener + // cannot honor fails here — on startup and on every reload — instead of + // running to no effect. The reverse proxy honors no redirect, which is what + // keeps a WritesDestination plugin off the inbound chain. + inDeps := plugins.Deps{SPIFFE: provider, Pricing: pricingRegistry, Listener: reverseproxy.Support()} + outDeps := plugins.Deps{SPIFFE: provider, Pricing: pricingRegistry, Listener: forwardproxy.Support(c.MTLS != nil)} + in, err := plugins.BuildWithDeps(c.Pipeline.Inbound.Plugins, inDeps) if err != nil { return nil, nil, nil, fmt.Errorf("inbound: %w", err) } - out, err := plugins.BuildWithDeps(c.Pipeline.Outbound.Plugins, deps) + out, err := plugins.BuildWithDeps(c.Pipeline.Outbound.Plugins, outDeps) if err != nil { return nil, nil, nil, fmt.Errorf("outbound: %w", err) } diff --git a/cmd/cortex/plugins_inferencerouter.go b/cmd/cortex/plugins_inferencerouter.go new file mode 100644 index 000000000..f08484087 --- /dev/null +++ b/cmd/cortex/plugins_inferencerouter.go @@ -0,0 +1,5 @@ +//go:build include_plugin_inferencerouter + +package main + +import _ "github.com/rossoctl/cortex/core/plugins/inferencerouter" diff --git a/core/listener/extproc/support.go b/core/listener/extproc/support.go new file mode 100644 index 000000000..66297d79e --- /dev/null +++ b/core/listener/extproc/support.go @@ -0,0 +1,12 @@ +package extproc + +import "github.com/rossoctl/cortex/core/pipeline" + +// Support is what the ext_proc listener honors, for plugins.Deps. It answers Envoy +// with header and body mutations and never holds the upstream connection; it skips +// every ":"-prefixed header, :authority included, so it cannot change where Envoy +// sends a request. Honoring a redirect would need Envoy-side routing that the Helm +// chart owns. +func Support() pipeline.ListenerSupport { + return pipeline.ListenerSupport{Listener: "ext_proc listener"} +} diff --git a/core/listener/extproc/support_test.go b/core/listener/extproc/support_test.go new file mode 100644 index 000000000..665bdefb1 --- /dev/null +++ b/core/listener/extproc/support_test.go @@ -0,0 +1,13 @@ +package extproc + +import ( + "testing" + + "github.com/rossoctl/cortex/core/pipeline" +) + +func TestSupport_RefusesARedirect(t *testing.T) { + if got := Support().Unsupported(pipeline.PluginCapabilities{WritesDestination: true}); got != "WritesDestination" { + t.Error("ext_proc admits a redirect; it cannot change where Envoy sends a request") + } +} diff --git a/core/listener/forwardproxy/inferencerouter_test.go b/core/listener/forwardproxy/inferencerouter_test.go new file mode 100644 index 000000000..3359a7d16 --- /dev/null +++ b/core/listener/forwardproxy/inferencerouter_test.go @@ -0,0 +1,220 @@ +package forwardproxy + +import ( + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "slices" + "strings" + "sync" + "testing" + "time" + + "github.com/rossoctl/cortex/core/memstore" + "github.com/rossoctl/cortex/core/pipeline" + "github.com/rossoctl/cortex/core/plugins/inferenceparser" + "github.com/rossoctl/cortex/core/plugins/inferencerouter" + "github.com/rossoctl/cortex/core/session" +) + +// seen is every request an origin received, by method and Authorization header — +// HEADs included, unlike origin.gets, so a key on any request is caught. +type seen struct { + mu sync.Mutex + calls []string +} + +func (s *seen) record(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + s.mu.Lock() + s.calls = append(s.calls, r.Method+" "+r.Header.Get("Authorization")) + s.mu.Unlock() + next.ServeHTTP(w, r) + }) +} + +func (s *seen) all() []string { + s.mu.Lock() + defer s.mu.Unlock() + return append([]string(nil), s.calls...) +} + +// recordingOrigin is newOrigin with every request's method and Authorization kept. +func recordingOrigin(t *testing.T, name string) (*origin, *seen) { + t.Helper() + s := &seen{} + o := newOrigin(t, name, func(srv *httptest.Server) { + srv.Config.Handler = s.record(srv.Config.Handler) + srv.StartTLS() + }) + return o, s +} + +// A client that CONNECTs to one host and names an inference server's host inside the +// bridged TLS must not get that server's key sent to the host it CONNECTed to. The +// router decides on the Host header, which on a bridged request is the client's word, +// while the forward proxy dials the CONNECT authority unless a redirect says +// otherwise. So the router always redirects a routed request to its server, and the +// listener dials RedirectTarget — the server — whatever the Host header or the +// CONNECT named. +func TestConnectBridge_TheRoutersKeyGoesOnlyToItsServer(t *testing.T) { + const key = "ete-secret-key" + ete, eteSeen := recordingOrigin(t, "FROM-ETE") + attacker, attackerSeen := recordingOrigin(t, "FROM-ATTACKER") + + router := inferencerouter.New() + if err := router.Configure(json.RawMessage(`{ + "servers": {"ete": {"url": "` + ete.URL + `", "key": "` + key + `"}}, + "agents": {"claude-code": "ete"} + }`)); err != nil { + t.Fatalf("Configure: %v", err) + } + p, err := pipeline.New([]pipeline.Plugin{router}) + if err != nil { + t.Fatalf("pipeline.New: %v", err) + } + + // The bridge intercepts the attacker's port, the one the client CONNECTs to. + // httptest gives every TLS server one certificate, so trusting the attacker's + // trusts ete's: the bridge would complete either handshake, and only the dial + // target decides who gets the request. + engine := redirectBridge(t, portOf(attacker.authority()), certPEM(attacker.Server)) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + shared := memstore.New() + defer shared.Close() + srv := &Server{ + OutboundPipeline: pipeline.NewHolder(p), + Sessions: store, + Shared: shared, + Client: http.DefaultClient, + TLSBridge: engine, + } + proxy := httptest.NewServer(srv.Handler()) + defer proxy.Close() + + status, body := bridgedGet(t, proxy, engine.CAPEM, attacker.authority(), "/v1/messages", func(r *http.Request) { + r.Host = ete.authority() + r.Header.Set("User-Agent", "claude-cli/2.1.0 (external, cli)") + r.Header.Set("Authorization", "Bearer client-key") + }) + + for _, call := range attackerSeen.all() { + if call != http.MethodHead+" " { + t.Errorf("the CONNECT authority received %q; only the bridge's keyless HEAD probe may reach it", call) + } + } + if hosts, _ := attacker.gets(); len(hosts) != 0 { + t.Errorf("the CONNECT authority served the request (hosts %v): the router's key went where the client CONNECTed", hosts) + } + if status != http.StatusOK || body != "FROM-ETE" { + t.Errorf("response = %d %q, want 200 FROM-ETE: the routed request must reach its server", status, body) + } + var routed bool + for _, call := range eteSeen.all() { + if call == http.MethodGet+" Bearer "+key { + routed = true + } + } + if !routed { + t.Errorf("ete saw %v, want a GET carrying Bearer %s", eteSeen.all(), key) + } +} + +// keyOrigin is a plain-http origin that records the Authorization of every request +// it serves, and answers with its name. +func keyOrigin(t *testing.T, name string) (*httptest.Server, *seen) { + t.Helper() + s := &seen{} + srv := httptest.NewServer(s.record(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + _, _ = w.Write([]byte(name)) + }))) + t.Cleanup(srv.Close) + return srv, s +} + +// The documented first setup, through the listener: a Claude Code session is +// already talking to ete when the router is first added and claude-code routed to +// glm, and sends nothing in between. The router has never seen that session, so it +// holds no pin; the session store's history does, and the next turn stays on ete, +// with ete's key, rather than moving to glm mid-conversation. A session that starts +// afterwards goes to glm. Both servers are plain http on loopback, which the router +// sends to without a TLS bridge; 127.0.0.1 and localhost are two hosts to it. +func TestForwardProxy_ASessionRunningBeforeRoutingWasConfiguredStaysOnItsServer(t *testing.T) { + ete, eteSeen := keyOrigin(t, "FROM-ETE") + glm, glmSeen := keyOrigin(t, "FROM-GLM") + eteURL := ete.URL + glmURL := strings.Replace(glm.URL, "127.0.0.1", "localhost", 1) + + parserOnly, err := pipeline.New([]pipeline.Plugin{inferenceparser.NewInferenceParser()}) + if err != nil { + t.Fatalf("pipeline.New: %v", err) + } + holder := pipeline.NewHolder(parserOnly) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + shared := memstore.New() + defer shared.Close() + srv := &Server{ + OutboundPipeline: holder, + Sessions: store, + Shared: shared, + SessionIDHeaders: []string{session.ClaudeCodeSessionHeader}, + Client: http.DefaultClient, + } + proxy := httptest.NewServer(srv.Handler()) + defer proxy.Close() + + turn := func(sessionID string) string { + t.Helper() + body := `{"model":"claude-opus-5-5","max_tokens":8,"messages":[{"role":"user","content":"hi"}]}` + req, err := http.NewRequest(http.MethodPost, eteURL+"/v1/messages", strings.NewReader(body)) + if err != nil { + t.Fatal(err) + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "claude-cli/2.1.286 (external, cli)") + req.Header.Set("Authorization", "Bearer client-key") + req.Header.Set(session.ClaudeCodeSessionHeader, sessionID) + resp, err := proxyClient(proxy, nil).Do(req) + if err != nil { + t.Fatalf("request through the proxy: %v", err) + } + defer resp.Body.Close() + b, _ := io.ReadAll(resp.Body) + return string(b) + } + + // Before the router: the session talks to ete with its own key. + if got := turn("running"); got != "FROM-ETE" { + t.Fatalf("first turn reached %q, want ete", got) + } + + // agentop server add ete, add glm, use glm --agent claude-code: one reload. + router := inferencerouter.New() + if err := router.Configure(json.RawMessage(`{ + "servers": {"ete": {"url": "` + eteURL + `", "key": "ete-key"}, "glm": {"url": "` + glmURL + `", "key": "glm-key"}}, + "agents": {"claude-code": "glm"} + }`)); err != nil { + t.Fatalf("Configure: %v", err) + } + routed, err := pipeline.New([]pipeline.Plugin{inferenceparser.NewInferenceParser(), router}) + if err != nil { + t.Fatalf("pipeline.New: %v", err) + } + holder.Store(routed) + + if got := turn("running"); got != "FROM-ETE" { + t.Errorf("the running session's next turn reached %q, want ete: it moved mid-conversation", got) + } + if got := turn("fresh"); got != "FROM-GLM" { + t.Errorf("a new session reached %q, want glm", got) + } + if got, want := eteSeen.all(), []string{"POST Bearer client-key", "POST Bearer ete-key"}; !slices.Equal(got, want) { + t.Errorf("ete saw %q, want %q: the running session keeps its server and gets that server's key", got, want) + } + if got, want := glmSeen.all(), []string{"POST Bearer glm-key"}; !slices.Equal(got, want) { + t.Errorf("glm saw %q, want only the new session's %q", got, want) + } +} diff --git a/core/listener/forwardproxy/redirect_test.go b/core/listener/forwardproxy/redirect_test.go new file mode 100644 index 000000000..5a9f1e4fc --- /dev/null +++ b/core/listener/forwardproxy/redirect_test.go @@ -0,0 +1,519 @@ +package forwardproxy + +import ( + "bufio" + "context" + "crypto/tls" + "crypto/x509" + "encoding/pem" + "io" + "net" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "sync" + "testing" + "time" + + "github.com/rossoctl/cortex/core/pipeline" + "github.com/rossoctl/cortex/core/session" + "github.com/rossoctl/cortex/core/tlsbridge" +) + +// redirectPlugin redirects every request to target and records what Redirect +// answered, separately for CONNECTs and for the requests the proxy re-originates. +// It declares WritesDestination, as a routing plugin does. +type redirectPlugin struct { + target string + + mu sync.Mutex + sawConnect bool + connectErr error + sawRequest bool + requestErr error +} + +func (p *redirectPlugin) Name() string { return "redirector" } +func (p *redirectPlugin) Capabilities() pipeline.PluginCapabilities { + return pipeline.PluginCapabilities{WritesDestination: true, Description: "test"} +} +func (p *redirectPlugin) OnRequest(_ context.Context, pctx *pipeline.Context) pipeline.Action { + u, err := url.Parse(p.target) + if err != nil { + panic(err) + } + err = pctx.Redirect(u) + p.mu.Lock() + defer p.mu.Unlock() + if pctx.Method == http.MethodConnect { + p.sawConnect, p.connectErr = true, err + } else { + p.sawRequest, p.requestErr = true, err + } + return pipeline.Action{Type: pipeline.Continue} +} +func (p *redirectPlugin) OnResponse(context.Context, *pipeline.Context) pipeline.Action { + return pipeline.Action{Type: pipeline.Continue} +} + +// denyPlugin rejects every request, so a test can read the denied row a redirected +// request leaves. +type denyPlugin struct{} + +func (denyPlugin) Name() string { return "denier" } +func (denyPlugin) Capabilities() pipeline.PluginCapabilities { + return pipeline.PluginCapabilities{Description: "test"} +} +func (denyPlugin) OnRequest(_ context.Context, pctx *pipeline.Context) pipeline.Action { + return pctx.DenyAndRecord("test_deny", "test.denied", "denied by the test") +} +func (denyPlugin) OnResponse(context.Context, *pipeline.Context) pipeline.Action { + return pipeline.Action{Type: pipeline.Continue} +} + +// hostWriter writes pctx.Host without declaring WritesDestination, as any plugin can: +// the field is exported. It must not be able to move a request. +type hostWriter struct{ host string } + +func (hostWriter) Name() string { return "host-writer" } +func (hostWriter) Capabilities() pipeline.PluginCapabilities { + return pipeline.PluginCapabilities{Description: "test"} +} +func (w hostWriter) OnRequest(_ context.Context, pctx *pipeline.Context) pipeline.Action { + pctx.Host = w.host + return pipeline.Action{Type: pipeline.Continue} +} +func (hostWriter) OnResponse(context.Context, *pipeline.Context) pipeline.Action { + return pipeline.Action{Type: pipeline.Continue} +} + +// origin is an httptest server that answers every request with its name and records +// the Host header and path of each GET. HEADs — the TLS bridge's upstream probe — +// are answered and not recorded. +type origin struct { + *httptest.Server + name string + + mu sync.Mutex + hosts []string + paths []string +} + +func newOrigin(t *testing.T, name string, start func(*httptest.Server)) *origin { + t.Helper() + o := &origin{name: name} + o.Server = httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.Method == http.MethodGet { + o.mu.Lock() + o.hosts = append(o.hosts, r.Host) + o.paths = append(o.paths, r.URL.Path) + o.mu.Unlock() + } + _, _ = io.WriteString(w, name) + })) + start(o.Server) + t.Cleanup(o.Close) + return o +} + +func (o *origin) gets() (hosts, paths []string) { + o.mu.Lock() + defer o.mu.Unlock() + return append([]string(nil), o.hosts...), append([]string(nil), o.paths...) +} + +func (o *origin) authority() string { return mustParseURL(o.URL).Host } + +// newRedirectProxy serves a forward proxy whose outbound pipeline is plugins. store +// and bridge may be nil. +func newRedirectProxy(t *testing.T, store *session.Store, bridge *tlsbridge.Engine, plugins ...pipeline.Plugin) *httptest.Server { + t.Helper() + p, err := pipeline.New(plugins) + if err != nil { + t.Fatalf("pipeline.New: %v", err) + } + srv := &Server{ + OutboundPipeline: pipeline.NewHolder(p), + Sessions: store, + Client: http.DefaultClient, + TLSBridge: bridge, + } + proxy := httptest.NewServer(srv.Handler()) + t.Cleanup(proxy.Close) + return proxy +} + +func proxyClient(proxy *httptest.Server, tlsConf *tls.Config) *http.Client { + return &http.Client{Transport: &http.Transport{ + Proxy: http.ProxyURL(mustParseURL(proxy.URL)), + TLSClientConfig: tlsConf, + }} +} + +// redirectBridge is a TLS bridge that intercepts CONNECTs to port and whose +// upstream client trusts exactly trustPEM. +func redirectBridge(t *testing.T, port int, trustPEM []byte) *tlsbridge.Engine { + t.Helper() + src, err := tlsbridge.NewEphemeralSource() + if err != nil { + t.Fatalf("NewEphemeralSource: %v", err) + } + up, err := tlsbridge.NewUpstreamClient(trustPEM, false) + if err != nil { + t.Fatalf("NewUpstreamClient: %v", err) + } + return &tlsbridge.Engine{ + Decision: mustBridgeDecision(t, port), + Term: tlsbridge.NewTerminator(tlsbridge.NewMinter(src, tlsbridge.MinterOpts{})), + Skip: tlsbridge.NewSkipSet(), + Upstream: up, + CAPEM: src.CACertPEM(), + } +} + +func certPEM(s *httptest.Server) []byte { + return pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: s.Certificate().Raw}) +} + +// bridgedGet CONNECTs to authority through proxy, completes the agent-side TLS +// handshake against the bridge's CA, and GETs path over the bridged connection — +// the sequence TestConnectBridge drives inline. It returns the status and body. +// edits change the decrypted request before it is sent: its Host header, which a +// client may set to something other than the CONNECT authority, or its headers. +func bridgedGet(t *testing.T, proxy *httptest.Server, bridgeCA []byte, authority, path string, edits ...func(*http.Request)) (int, string) { + t.Helper() + rawConn, err := net.Dial("tcp", mustParseURL(proxy.URL).Host) + if err != nil { + t.Fatalf("dial proxy: %v", err) + } + t.Cleanup(func() { _ = rawConn.Close() }) + _ = rawConn.SetDeadline(time.Now().Add(10 * time.Second)) + + connectReq, err := http.NewRequest(http.MethodConnect, "//"+authority, nil) + if err != nil { + t.Fatalf("new CONNECT request: %v", err) + } + connectReq.Host = authority + connectReq.URL.Host = authority + if err := connectReq.Write(rawConn); err != nil { + t.Fatalf("write CONNECT: %v", err) + } + br := bufio.NewReader(rawConn) + connectResp, err := http.ReadResponse(br, connectReq) + if err != nil { + t.Fatalf("read CONNECT response: %v", err) + } + _ = connectResp.Body.Close() + if connectResp.StatusCode != http.StatusOK { + t.Fatalf("CONNECT status = %d, want 200", connectResp.StatusCode) + } + + pool := x509.NewCertPool() + if !pool.AppendCertsFromPEM(bridgeCA) { + t.Fatal("bridge CA PEM did not parse") + } + host := hostOnly(authority) + tconn := tls.Client(&bufferedConn{Conn: rawConn, r: br}, &tls.Config{ + ServerName: host, + RootCAs: pool, + NextProtos: []string{"http/1.1"}, + }) + if err := tconn.Handshake(); err != nil { + t.Fatalf("agent-side TLS handshake through the bridge: %v", err) + } + + req, err := http.NewRequest(http.MethodGet, "https://"+host+path, nil) + if err != nil { + t.Fatalf("new request: %v", err) + } + for _, edit := range edits { + edit(req) + } + // Write from a goroutine: the proxy makes a blocking upstream round trip between + // reading the request and answering, as in TestConnectBridge. + writeErr := make(chan error, 1) + go func() { writeErr <- req.Write(tconn) }() + resp, err := http.ReadResponse(bufio.NewReader(tconn), req) + if err != nil { + t.Fatalf("read response over the bridged connection: %v", err) + } + defer resp.Body.Close() + body, err := io.ReadAll(resp.Body) + if err != nil { + t.Fatalf("read body: %v", err) + } + if werr := <-writeErr; werr != nil { + t.Fatalf("write request over the bridged connection: %v", werr) + } + return resp.StatusCode, strings.TrimSpace(string(body)) +} + +func eventsOf(t *testing.T, store *session.Store) []pipeline.SessionEvent { + t.Helper() + v := store.View(session.DefaultSessionID) + if v == nil { + t.Fatal("no events recorded") + } + return v.Events +} + +func TestForwardProxy_RedirectSendsAPlainRequestToTheTarget(t *testing.T) { + a := newOrigin(t, "FROM-A", (*httptest.Server).Start) + b := newOrigin(t, "FROM-B", (*httptest.Server).Start) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + plug := &redirectPlugin{target: b.URL} + proxy := newRedirectProxy(t, store, nil, plug) + + resp, err := proxyClient(proxy, nil).Get(a.URL + "/v1/messages") + if err != nil { + t.Fatalf("GET through the proxy: %v", err) + } + body, _ := io.ReadAll(resp.Body) + resp.Body.Close() + + if resp.StatusCode != http.StatusOK || string(body) != "FROM-B" { + t.Fatalf("response = %d %q, want 200 FROM-B", resp.StatusCode, body) + } + if plug.requestErr != nil { + t.Fatalf("Redirect on a plain proxied request: %v", plug.requestErr) + } + if hosts, _ := a.gets(); len(hosts) != 0 { + t.Errorf("origin a served %v; the redirect did not take", hosts) + } + hosts, paths := b.gets() + if len(hosts) != 1 || hosts[0] != b.authority() || paths[0] != "/v1/messages" { + t.Errorf("origin b saw hosts %v paths %v, want one GET for %s /v1/messages", hosts, paths, b.authority()) + } + for _, ev := range eventsOf(t, store) { + if ev.Host != b.authority() || ev.RequestedHost != a.authority() { + t.Errorf("%s row: host %q requested %q, want %q and %q", ev.Phase, ev.Host, ev.RequestedHost, b.authority(), a.authority()) + } + } +} + +// The listener applies the target Redirect validated, not the exported Host a later, +// undeclared plugin wrote; and the request row names where the bytes went. +func TestForwardProxy_ALaterHostWriteDoesNotSteerARedirect(t *testing.T) { + a := newOrigin(t, "FROM-A", (*httptest.Server).Start) + b := newOrigin(t, "FROM-B", (*httptest.Server).Start) + c := newOrigin(t, "FROM-C", (*httptest.Server).Start) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + proxy := newRedirectProxy(t, store, nil, &redirectPlugin{target: b.URL}, hostWriter{host: c.authority()}) + + resp, err := proxyClient(proxy, nil).Get(a.URL + "/x") + if err != nil { + t.Fatalf("GET through the proxy: %v", err) + } + body, _ := io.ReadAll(resp.Body) + resp.Body.Close() + + if resp.StatusCode != http.StatusOK || string(body) != "FROM-B" { + t.Fatalf("response = %d %q, want 200 FROM-B: only the redirect may choose the host", resp.StatusCode, body) + } + if hosts, _ := c.gets(); len(hosts) != 0 { + t.Errorf("origin c served %v; an undeclared plugin steered the request", hosts) + } + var found bool + for _, ev := range eventsOf(t, store) { + if ev.Phase != pipeline.SessionRequest { + continue + } + found = true + if ev.Host != b.authority() || ev.RequestedHost != a.authority() { + t.Errorf("request row: host %q requested %q, want %q and %q", ev.Host, ev.RequestedHost, b.authority(), a.authority()) + } + } + if !found { + t.Error("no request row") + } +} + +func TestForwardProxy_NoRedirectRecordsNoRequestedHost(t *testing.T) { + a := newOrigin(t, "FROM-A", (*httptest.Server).Start) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + proxy := newRedirectProxy(t, store, nil, &bridgeProbePlugin{}) + + resp, err := proxyClient(proxy, nil).Get(a.URL + "/x") + if err != nil { + t.Fatalf("GET through the proxy: %v", err) + } + resp.Body.Close() + for _, ev := range eventsOf(t, store) { + if ev.RequestedHost != "" { + t.Errorf("%s row records requestedHost %q for a request nothing redirected", ev.Phase, ev.RequestedHost) + } + } +} + +func TestConnectBridge_RedirectSendsTheDecryptedRequestToTheTarget(t *testing.T) { + a := newOrigin(t, "FROM-A", (*httptest.Server).StartTLS) + b := newOrigin(t, "FROM-B", (*httptest.Server).StartTLS) + // httptest gives every TLS server the same certificate, so trusting a's trusts + // b's. This test is about where the request goes; the next is about what the + // bridge verifies when it gets there. + engine := redirectBridge(t, portOf(a.authority()), certPEM(a.Server)) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + plug := &redirectPlugin{target: b.URL} + proxy := newRedirectProxy(t, store, engine, plug) + + status, body := bridgedGet(t, proxy, engine.CAPEM, a.authority(), "/secret") + + if status != http.StatusOK || body != "FROM-B" { + t.Fatalf("response = %d %q, want 200 FROM-B", status, body) + } + if hosts, _ := a.gets(); len(hosts) != 0 { + t.Errorf("origin a served GETs %v; the redirect did not take", hosts) + } + if hosts, paths := b.gets(); len(hosts) != 1 || hosts[0] != b.authority() || paths[0] != "/secret" { + t.Errorf("origin b saw hosts %v paths %v, want one GET for %s /secret", hosts, paths, b.authority()) + } + plug.mu.Lock() + defer plug.mu.Unlock() + if !plug.sawConnect || plug.connectErr == nil || !strings.Contains(plug.connectErr.Error(), "cannot be redirected") { + t.Errorf("Redirect on the CONNECT = %v; the CONNECT is dialed where the client named, so it must be refused", plug.connectErr) + } + if !plug.sawRequest || plug.requestErr != nil { + t.Errorf("Redirect on the decrypted request = %v (seen: %v), want it accepted", plug.requestErr, plug.sawRequest) + } + var found bool + for _, ev := range eventsOf(t, store) { + if ev.Phase != pipeline.SessionRequest || ev.HTTPPath != "/secret" { + continue + } + found = true + // The decrypted request's own Host header carries no port, so that is the + // host the client named. + if ev.Host != b.authority() || ev.RequestedHost != hostOnly(a.authority()) { + t.Errorf("request row: host %q requested %q, want %q and %q", ev.Host, ev.RequestedHost, b.authority(), hostOnly(a.authority())) + } + } + if !found { + t.Error("no request row for /secret") + } +} + +// The inference router's local-gateway case — a bridged https request sent to a +// plaintext http server — and the only test that pins the listener's scheme copy. +func TestConnectBridge_RedirectToAPlaintextTargetChangesTheScheme(t *testing.T) { + a := newOrigin(t, "FROM-A", (*httptest.Server).StartTLS) + b := newOrigin(t, "FROM-B", (*httptest.Server).Start) // plaintext + engine := redirectBridge(t, portOf(a.authority()), certPEM(a.Server)) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + proxy := newRedirectProxy(t, store, engine, &redirectPlugin{target: b.URL}) + if status, body := bridgedGet(t, proxy, engine.CAPEM, a.authority(), "/v1/messages"); status != 200 || body != "FROM-B" { + t.Fatalf("got %d %q, want 200 FROM-B", status, body) + } +} + +func TestConnectBridge_RedirectVerifiesTheTargetsCertificate(t *testing.T) { + a := newOrigin(t, "FROM-A", (*httptest.Server).StartTLS) + // b presents a certificate from a CA the bridge's upstream client does not trust. + other, err := tlsbridge.NewEphemeralSource() + if err != nil { + t.Fatalf("NewEphemeralSource: %v", err) + } + leaf, err := tlsbridge.NewMinter(other, tlsbridge.MinterOpts{}).GetCertificateForHost("127.0.0.1") + if err != nil { + t.Fatalf("mint b's certificate: %v", err) + } + b := newOrigin(t, "FROM-B", func(s *httptest.Server) { + s.TLS = &tls.Config{Certificates: []tls.Certificate{*leaf}} + s.StartTLS() + }) + engine := redirectBridge(t, portOf(a.authority()), certPEM(a.Server)) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + proxy := newRedirectProxy(t, store, engine, &redirectPlugin{target: b.URL}) + + status, _ := bridgedGet(t, proxy, engine.CAPEM, a.authority(), "/secret") + + if status != http.StatusBadGateway { + t.Fatalf("status = %d, want 502: the redirect target's certificate is untrusted", status) + } + if hosts, _ := b.gets(); len(hosts) != 0 { + t.Errorf("origin b served %v despite an untrusted certificate", hosts) + } + var kind string + for _, ev := range eventsOf(t, store) { + if ev.Phase == pipeline.SessionResponse && ev.Error != nil { + kind = ev.Error.Kind + } + } + if kind != "upstream_tls" { + t.Errorf("response row error kind = %q, want upstream_tls", kind) + } +} + +func TestForwardProxy_RedirectOnAConnectIsRefusedAndTheTunnelGoesWhereTheClientAsked(t *testing.T) { + a := newOrigin(t, "FROM-A", (*httptest.Server).StartTLS) + b := newOrigin(t, "FROM-B", (*httptest.Server).StartTLS) + plug := &redirectPlugin{target: b.URL} + proxy := newRedirectProxy(t, nil, nil, plug) // no bridge: the CONNECT is an opaque tunnel + + pool := x509.NewCertPool() + pool.AddCert(a.Certificate()) + resp, err := proxyClient(proxy, &tls.Config{RootCAs: pool}).Get(a.URL + "/x") + if err != nil { + t.Fatalf("GET through the tunnel: %v", err) + } + body, _ := io.ReadAll(resp.Body) + resp.Body.Close() + + if string(body) != "FROM-A" { + t.Errorf("body = %q, want FROM-A: a tunnel goes where the client asked", body) + } + if hosts, _ := b.gets(); len(hosts) != 0 { + t.Errorf("origin b served %v", hosts) + } + plug.mu.Lock() + defer plug.mu.Unlock() + if !plug.sawConnect || plug.connectErr == nil { + t.Errorf("Redirect on a CONNECT = %v (seen: %v), want a refusal", plug.connectErr, plug.sawConnect) + } + if plug.sawRequest { + t.Error("the pipeline saw a decrypted request on a tunnel with no bridge") + } +} + +func TestForwardProxy_ADeniedRedirectRecordsBothHosts(t *testing.T) { + a := newOrigin(t, "FROM-A", (*httptest.Server).Start) + b := newOrigin(t, "FROM-B", (*httptest.Server).Start) + store := session.New(5*time.Minute, 100, 0) + defer store.Close() + proxy := newRedirectProxy(t, store, nil, &redirectPlugin{target: b.URL}, denyPlugin{}) + + resp, err := proxyClient(proxy, nil).Get(a.URL + "/x") + if err != nil { + t.Fatalf("GET through the proxy: %v", err) + } + resp.Body.Close() + + if resp.StatusCode < 400 { + t.Fatalf("status = %d, want a rejection", resp.StatusCode) + } + var found bool + for _, ev := range eventsOf(t, store) { + if ev.Phase != pipeline.SessionDenied { + continue + } + found = true + if ev.Host != b.authority() || ev.RequestedHost != a.authority() { + t.Errorf("denied row: host %q requested %q, want %q and %q", ev.Host, ev.RequestedHost, b.authority(), a.authority()) + } + } + if !found { + t.Error("no denied row") + } + if hosts, _ := a.gets(); len(hosts) != 0 { + t.Errorf("origin a served a denied request: %v", hosts) + } + if hosts, _ := b.gets(); len(hosts) != 0 { + t.Errorf("origin b served a denied request: %v", hosts) + } +} diff --git a/core/listener/forwardproxy/server.go b/core/listener/forwardproxy/server.go index 7308f2c08..abef1c728 100644 --- a/core/listener/forwardproxy/server.go +++ b/core/listener/forwardproxy/server.go @@ -333,6 +333,12 @@ func (s *Server) serveOutbound(w http.ResponseWriter, r *http.Request, tl *tunne // — and silent on the day one does, which is why it is pinned rather than watched. pctx.ResolveClient() + // Every request that reaches here is re-originated from pctx below, plain or + // TLS-bridged, so a plugin may redirect it. The CONNECT and transparent-connection + // contexts are never marked: both dial the address the client chose, so a redirect + // there would be recorded and never happen. + pctx.MarkRedirectable() + // SkipHosts short-circuit: forward as a transparent proxy. No // pipeline run, no body buffering, no session recording, no // response-phase work. RunFinish is also skipped (no defer @@ -414,6 +420,22 @@ func (s *Server) serveOutbound(w http.ResponseWriter, r *http.Request, tl *tunne if !skipped { action := s.OutboundPipeline.Run(r.Context(), pctx) + // Send the request where the pipeline pointed it, before any row is recorded, + // so the row names the host the bytes went to. The target comes from + // RedirectTarget, the copy Redirect validated, never from pctx.Scheme and + // pctx.Host: those are exported, and a later plugin that never declared + // WritesDestination could otherwise steer a redirected request. They are + // re-asserted to the same values so the request row, the denied row and the + // redirect record all agree with the dial. r.URL.Host carries the dial target, + // SNI and certificate check, and r.Host the Host header: they move together, + // or the upstream sees one host while the transport dials another. The client + // choice below does not change — a bridged request still goes through the + // bridge's upstream client, which now verifies the new host. + if scheme, host, ok := pctx.RedirectTarget(); ok { + pctx.Scheme, pctx.Host = scheme, host + r.URL.Scheme, r.URL.Host, r.Host = scheme, host, host + } + if action.Type == pipeline.Reject { s.recordOutboundRejectIn(tl, pctx, action, s.recordingSessionID(sessionID, r.Header, chain)) // Render as a JSON-RPC error frame when the rejected @@ -923,19 +945,20 @@ func (s *Server) recordOutboundRequestEvent(tl *tunnelLog, pctx *pipeline.Contex // struct (e.g. token counts assigned in OnResponse). plugins := pipeline.SnapshotPlugins(pctx.Extensions.Custom) ev := pipeline.SessionEvent{ - At: time.Now(), - Direction: pipeline.Outbound, - Phase: pipeline.SessionRequest, - RequestID: pctx.RequestID(), - MCP: pipeline.SnapshotMCP(pctx.Extensions.MCP), - Inference: pipeline.SnapshotInference(pctx.Extensions.Inference), - Invocations: pipeline.SnapshotInvocations(pctx.Extensions.Invocations, pipeline.InvocationPhaseRequest), - Plugins: plugins, - Identity: pipeline.SnapshotIdentity(pctx), - Host: pctx.Host, - HTTPMethod: pctx.Method, - HTTPPath: pctx.Path, - Client: pctx.ClientInfo(), + At: time.Now(), + Direction: pipeline.Outbound, + Phase: pipeline.SessionRequest, + RequestID: pctx.RequestID(), + MCP: pipeline.SnapshotMCP(pctx.Extensions.MCP), + Inference: pipeline.SnapshotInference(pctx.Extensions.Inference), + Invocations: pipeline.SnapshotInvocations(pctx.Extensions.Invocations, pipeline.InvocationPhaseRequest), + Plugins: plugins, + Identity: pipeline.SnapshotIdentity(pctx), + Host: pctx.Host, + RequestedHost: pctx.RequestedHost(), + HTTPMethod: pctx.Method, + HTTPPath: pctx.Path, + Client: pctx.ClientInfo(), } s.appendOutbound(tl, sid, ev) } @@ -1149,22 +1172,23 @@ func (s *Server) recordOutboundResponseEvent(pctx *pipeline.Context, statusCode } plugins := pipeline.SnapshotPlugins(pctx.Extensions.Custom) ev := pipeline.SessionEvent{ - At: time.Now(), - Direction: pipeline.Outbound, - Phase: pipeline.SessionResponse, - RequestID: pctx.RequestID(), - MCP: pipeline.SnapshotMCP(pctx.Extensions.MCP), - Inference: pipeline.SnapshotInference(pctx.Extensions.Inference), - Invocations: pipeline.SnapshotInvocations(pctx.Extensions.Invocations, pipeline.InvocationPhaseResponse), - Plugins: plugins, - Identity: pipeline.SnapshotIdentity(pctx), - Host: pctx.Host, - HTTPMethod: pctx.Method, - HTTPPath: pctx.Path, - StatusCode: statusCode, - Error: pipeline.EventErrorOr(pctx, fail), - Duration: pipeline.DurationSince(pctx.StartedAt), - Client: pctx.ClientInfo(), + At: time.Now(), + Direction: pipeline.Outbound, + Phase: pipeline.SessionResponse, + RequestID: pctx.RequestID(), + MCP: pipeline.SnapshotMCP(pctx.Extensions.MCP), + Inference: pipeline.SnapshotInference(pctx.Extensions.Inference), + Invocations: pipeline.SnapshotInvocations(pctx.Extensions.Invocations, pipeline.InvocationPhaseResponse), + Plugins: plugins, + Identity: pipeline.SnapshotIdentity(pctx), + Host: pctx.Host, + RequestedHost: pctx.RequestedHost(), + HTTPMethod: pctx.Method, + HTTPPath: pctx.Path, + StatusCode: statusCode, + Error: pipeline.EventErrorOr(pctx, fail), + Duration: pipeline.DurationSince(pctx.StartedAt), + Client: pctx.ClientInfo(), } // Always record — see the request-phase comment. This is what surfaces // responses no plugin acted on (e.g. a generic 404), carrying StatusCode diff --git a/core/listener/forwardproxy/support.go b/core/listener/forwardproxy/support.go new file mode 100644 index 000000000..cb80b0d45 --- /dev/null +++ b/core/listener/forwardproxy/support.go @@ -0,0 +1,21 @@ +package forwardproxy + +import "github.com/rossoctl/cortex/core/pipeline" + +// Support is what the forward proxy honors, for plugins.Deps. It re-originates every +// request it runs the pipeline on, plain or TLS-bridged, so it honors a redirect. +// +// Except whenever an mtls: block is configured, which is what mtls reports. Under +// mode: strict the forward proxy dials every upstream through its mTLS dialer, which +// wraps the connection in its own TLS, so an https redirect would nest TLS inside TLS +// — untested, and mTLS is an in-cluster feature no redirecting plugin needs today. +// Under permissive there is no such dialer, since outbound stays plaintext, so the +// refusal is broader than strictly needed; it fails closed. The reloader refuses any +// mtls change, which keeps this check matched to the mtls config the listeners were +// built from. +func Support(mtls bool) pipeline.ListenerSupport { + if mtls { + return pipeline.ListenerSupport{Listener: "forward proxy with mtls"} + } + return pipeline.ListenerSupport{Listener: "forward proxy", Destination: true} +} diff --git a/core/listener/forwardproxy/support_test.go b/core/listener/forwardproxy/support_test.go new file mode 100644 index 000000000..e6042c0e8 --- /dev/null +++ b/core/listener/forwardproxy/support_test.go @@ -0,0 +1,21 @@ +package forwardproxy + +import ( + "testing" + + "github.com/rossoctl/cortex/core/pipeline" +) + +var destinationWriter = pipeline.PluginCapabilities{WritesDestination: true} + +func TestSupport_HonorsARedirectWithoutMTLS(t *testing.T) { + if got := Support(false).Unsupported(destinationWriter); got != "" { + t.Errorf("Support(false) refuses %s; the forward proxy re-originates every request it runs the pipeline on", got) + } +} + +func TestSupport_RefusesARedirectWithMTLS(t *testing.T) { + if got := Support(true).Unsupported(destinationWriter); got != "WritesDestination" { + t.Errorf("Support(true) admits a redirect; through the mTLS dialer an https target would nest TLS in TLS") + } +} diff --git a/core/listener/internal/sessionevent/sessionevent.go b/core/listener/internal/sessionevent/sessionevent.go index 0762bfbc0..f9f0999d9 100644 --- a/core/listener/internal/sessionevent/sessionevent.go +++ b/core/listener/internal/sessionevent/sessionevent.go @@ -51,17 +51,18 @@ func Deny(pctx *pipeline.Context, action pipeline.Action, dir pipeline.Direction message = action.Violation.Reason } return pipeline.SessionEvent{ - At: time.Now(), - Direction: dir, - Phase: pipeline.SessionDenied, - RequestID: pctx.RequestID(), - Invocations: pipeline.SnapshotInvocations(pctx.Extensions.Invocations, pipeline.InvocationPhaseRequest), - Plugins: pipeline.SnapshotPlugins(pctx.Extensions.Custom), - Identity: pipeline.SnapshotIdentity(pctx), - Host: pctx.Host, - HTTPMethod: pctx.Method, - HTTPPath: pctx.Path, - StatusCode: status, + At: time.Now(), + Direction: dir, + Phase: pipeline.SessionDenied, + RequestID: pctx.RequestID(), + Invocations: pipeline.SnapshotInvocations(pctx.Extensions.Invocations, pipeline.InvocationPhaseRequest), + Plugins: pipeline.SnapshotPlugins(pctx.Extensions.Custom), + Identity: pipeline.SnapshotIdentity(pctx), + Host: pctx.Host, + RequestedHost: pctx.RequestedHost(), + HTTPMethod: pctx.Method, + HTTPPath: pctx.Path, + StatusCode: status, Error: &pipeline.EventError{ Kind: "policy", Code: code, diff --git a/core/listener/reverseproxy/support.go b/core/listener/reverseproxy/support.go new file mode 100644 index 000000000..a668e5f44 --- /dev/null +++ b/core/listener/reverseproxy/support.go @@ -0,0 +1,11 @@ +package reverseproxy + +import "github.com/rossoctl/cortex/core/pipeline" + +// Support is what the reverse proxy honors, for plugins.Deps. It forwards inbound +// requests to the local application, so there is nowhere else to send one: it +// honors no redirect, which is also what keeps a WritesDestination plugin off an +// inbound chain. +func Support() pipeline.ListenerSupport { + return pipeline.ListenerSupport{Listener: "reverse proxy"} +} diff --git a/core/listener/reverseproxy/support_test.go b/core/listener/reverseproxy/support_test.go new file mode 100644 index 000000000..6b60fc521 --- /dev/null +++ b/core/listener/reverseproxy/support_test.go @@ -0,0 +1,13 @@ +package reverseproxy + +import ( + "testing" + + "github.com/rossoctl/cortex/core/pipeline" +) + +func TestSupport_RefusesARedirect(t *testing.T) { + if got := Support().Unsupported(pipeline.PluginCapabilities{WritesDestination: true}); got != "WritesDestination" { + t.Error("the reverse proxy admits a redirect; it forwards to the local app and has nowhere else to send one") + } +} diff --git a/core/pipeline/context.go b/core/pipeline/context.go index 2cc00e59d..b1ef82b95 100644 --- a/core/pipeline/context.go +++ b/core/pipeline/context.go @@ -236,6 +236,26 @@ type Context struct { currentPhase InvocationPhase currentPolicy ErrorPolicy + // currentMayRedirect is true while Pipeline.Run dispatches OnRequest into a plugin + // that declares WritesDestination — the one dispatch Redirect is accepted from. + currentMayRedirect bool + + // redirectable is set by a listener that builds the upstream request from this + // context itself, so a redirect can take effect. See MarkRedirectable. + redirectable bool + + // redirected says a Redirect took effect; requestedHost is the Host the client + // named, remembered by the first one. See Redirect and RequestedHost. + redirected bool + requestedHost string + + // redirectScheme and redirectHost are the target the last accepted Redirect + // validated. Redirect also writes them to Scheme and Host, but those are exported + // and any later plugin may overwrite them, so the listener applies these instead. + // See RedirectTarget. + redirectScheme string + redirectHost string + // bodyMutated / responseBodyMutated flag that a plugin called // SetBody / SetResponseBody on this context. Listeners read the flag // via BodyMutated() / ResponseBodyMutated() after Run / RunResponse @@ -451,6 +471,7 @@ func (c *Context) clearCurrent() { c.currentPlugin = "" c.currentPhase = "" c.currentPolicy = "" + c.currentMayRedirect = false } // RejectingPlugin returns the name of the plugin whose Reject action diff --git a/core/pipeline/destination.go b/core/pipeline/destination.go new file mode 100644 index 000000000..ea1f85dd4 --- /dev/null +++ b/core/pipeline/destination.go @@ -0,0 +1,156 @@ +package pipeline + +import ( + "errors" + "fmt" + "log/slog" + "net/url" + "strings" +) + +// MarkRedirectable says the listener builds the upstream request from this context +// itself, after the pipeline, so a Redirect can take effect. The forward proxy calls +// it for every request it re-originates, plain or TLS-bridged. +// +// A listener must not call it for a context it cannot redirect. The outbound +// pipeline also runs once per CONNECT and once per transparently redirected +// connection, and both dial the address the client chose whatever a plugin does: a +// redirect accepted there would be recorded and never happen. +// +// Listener-facing, like SetCurrentPlugin. Production plugins never call this. +func (c *Context) MarkRedirectable() { c.redirectable = true } + +// Redirectable reports whether the listener will honor a Redirect on this request. A +// routing plugin checks it first, so that it acts only on requests it can move. +func (c *Context) Redirectable() bool { return c.redirectable } + +// Redirect sends the request to target instead of the host the client named. +// +// target must be absolute — scheme http or https, a host, an optional port — and +// carry nothing else: no user info, no path beyond "/", no query, no fragment. A +// redirect changes where a request goes and nothing about what it says; the path, +// headers and body stay the plugin's to change through the existing APIs. +// +// It sets Scheme and Host to the target's. Host following the redirect is the point: +// the session events, usage, the cost ledger and modelled pricing all key on it, and +// they must describe where the bytes went. Plugins that run after this one and key on +// Host see the new host for the same reason. The host the client named stays +// available as RequestedHost. The validated target is also kept privately, and that +// copy is what the listener applies (see RedirectTarget): Scheme and Host are any +// plugin's to write, so a later write to them cannot move the request. +// +// The framework records the redirect as a modify/redirected Invocation with "from" +// and "to" in Details, as SetBody records a body rewrite: a redirect moves the request +// and its credentials to another host, so the timeline shows it even if the plugin +// records nothing of its own. Under on_error: observe nothing moves and the +// Invocation is marked Shadow, so plugin code looks the same under enforce and observe. +// +// What a redirect does not do. The request's headers — credentials included — go to +// the new host unchanged. And it does not gate the plugin's own header writes: under +// observe nothing moves while those still apply, so a plugin that attaches +// credentials meant for the target must do so only when the request actually goes +// there — check Redirected after the call, which stays false under observe and on a +// refusal. Redirected is the only safe gate: pctx.Host names what the client asked for +// and, on a TLS-bridged request, is the client's own Host header, which need not match +// the address the proxy dials. A plugin that attaches credentials meant for a server +// should always Redirect to that server, even when pctx.Host already names it, so the +// request is dialed there. Plugins earlier in the chain made their decisions on the +// requested host and are not run again. +// +// Refused, with nothing changed and nothing recorded, when the calling plugin does +// not declare WritesDestination, when the listener did not mark the context +// redirectable, when the target is malformed, and outside OnRequest. +func (c *Context) Redirect(target *url.URL) error { + if c.inFinish { + slog.Warn("pipeline: plugin called pctx.Redirect during OnFinish — refused (the response is already sent)", + "plugin", c.currentPlugin) + return errors.New("pipeline: Redirect refused in OnFinish: the request has already been sent") + } + if c.currentPhase != InvocationPhaseRequest { + slog.Warn("pipeline: plugin called pctx.Redirect outside OnRequest — refused", + "plugin", c.currentPlugin, "phase", c.currentPhase) + return fmt.Errorf("pipeline: Redirect refused in phase %q: it is accepted only from OnRequest", c.currentPhase) + } + if !c.currentMayRedirect { + return fmt.Errorf("pipeline: plugin %q called Redirect without declaring WritesDestination", c.currentPlugin) + } + if !c.redirectable { + return errors.New("pipeline: this request cannot be redirected: the listener does not re-originate it (a CONNECT, or a transparently redirected connection)") + } + if err := checkRedirectTarget(target); err != nil { + return err + } + details := map[string]string{"from": c.Host, "to": target.Host} + if c.currentPolicy == ErrorPolicyObserve { + c.Record(Invocation{Action: ActionModify, Reason: "redirected", Shadow: true, Details: details}) + return nil + } + if !c.redirected { + c.requestedHost = c.Host + c.redirected = true + } + c.redirectScheme, c.redirectHost = target.Scheme, target.Host + c.Scheme, c.Host = target.Scheme, target.Host + c.Record(Invocation{Action: ActionModify, Reason: "redirected", Details: details}) + return nil +} + +// Redirected reports whether a Redirect took effect on this request, and is the ok +// RedirectTarget reports. The listener keys on that, not on RequestedHost: a redirect +// that changed only the scheme, or that pointed back at the host the client named, +// still has to be applied. +// +// It is also the only safe gate for a plugin that attaches credentials meant for the +// target. The request's headers go wherever the request goes, and Redirect returns nil +// under on_error: observe without moving anything, so a key set on the strength of that +// nil alone would reach the host the client named. Redirected stays false under observe +// and after a refusal. Comparing pctx.Host with the target is unsafe: on a TLS-bridged +// request, pctx.Host is the client's own Host header, not necessarily the address the +// proxy will dial. +func (c *Context) Redirected() bool { return c.redirected } + +// RedirectTarget is the scheme and host the last accepted Redirect validated, with ok +// exactly when Redirected. Listeners apply this, never the exported Scheme and Host: +// any plugin may write those, declared or not, so a listener that read them back would +// let a later plugin steer a redirected request somewhere no WritesDestination plugin +// chose, and the modify/redirected record would name a host the bytes never went to. +func (c *Context) RedirectTarget() (scheme, host string, ok bool) { + if !c.redirected { + return "", "", false + } + return c.redirectScheme, c.redirectHost, true +} + +// RequestedHost is the host the client named when a redirect sent the request +// somewhere else, and "" otherwise — including when the redirects ended back at +// that host. It is what a session event records beside Host. It compares against +// the target RedirectTarget reports, not the exported Host, so a later write to Host +// can neither hide a redirect nor invent one. +func (c *Context) RequestedHost() string { + if !c.redirected || strings.EqualFold(c.requestedHost, c.redirectHost) { + return "" + } + return c.requestedHost +} + +// checkRedirectTarget enforces Redirect's shape rule. Errors quote no part of the +// target: a refused target is exactly where a key sits — as user info, in the +// query, in a path segment — and Redacted masks only a password. +func checkRedirectTarget(u *url.URL) error { + if u == nil { + return errors.New("pipeline: Redirect target is nil") + } + switch { + case u.Scheme != "http" && u.Scheme != "https": + return errors.New("pipeline: Redirect target's scheme must be http or https") + case u.Opaque != "" || u.Hostname() == "": + return errors.New("pipeline: Redirect target has no host") + case u.User != nil: + return errors.New("pipeline: Redirect target carries user info; credentials belong in headers") + case u.Path != "" && u.Path != "/": + return errors.New("pipeline: Redirect target has a path; a redirect changes only the scheme and host") + case u.RawQuery != "" || u.ForceQuery || u.Fragment != "": + return errors.New("pipeline: Redirect target has a query or fragment") + } + return nil +} diff --git a/core/pipeline/destination_test.go b/core/pipeline/destination_test.go new file mode 100644 index 000000000..7b90bd610 --- /dev/null +++ b/core/pipeline/destination_test.go @@ -0,0 +1,342 @@ +package pipeline + +import ( + "context" + "net/url" + "strings" + "testing" +) + +// capPlugin declares a capability set and does nothing, for build-time rules. +type capPlugin struct { + name string + caps PluginCapabilities +} + +func (p *capPlugin) Name() string { return p.name } +func (p *capPlugin) Capabilities() PluginCapabilities { return p.caps } +func (p *capPlugin) OnRequest(context.Context, *Context) Action { + return Action{Type: Continue} +} +func (p *capPlugin) OnResponse(context.Context, *Context) Action { + return Action{Type: Continue} +} + +func destWriter(name string) *capPlugin { + return &capPlugin{name: name, caps: PluginCapabilities{WritesDestination: true, Description: "test"}} +} + +func TestNew_RejectsTwoDestinationWriters(t *testing.T) { + _, err := New([]Plugin{destWriter("first"), destWriter("second")}) + if err == nil { + t.Fatal("New accepted two WritesDestination plugins; the request can go to only one place") + } + for _, want := range []string{"WritesDestination", `"first"`, `"second"`} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not mention %s", err, want) + } + } +} + +func TestNew_AcceptsOneDestinationWriter(t *testing.T) { + bystander := &capPlugin{name: "bystander", caps: PluginCapabilities{Description: "test"}} + if _, err := New([]Plugin{destWriter("router"), bystander}); err != nil { + t.Fatalf("New rejected a single WritesDestination plugin: %v", err) + } +} + +// A plugin that rewrites a body AND chooses a destination is still one destination +// writer: the mutator branch of validateCapabilities ends in a continue, and the +// destination check must not sit behind it. +func TestNew_CountsADestinationWriterThatAlsoMutates(t *testing.T) { + both := &capPlugin{name: "both", caps: PluginCapabilities{WritesDestination: true, WritesRequestBody: true, Description: "test"}} + if _, err := New([]Plugin{both, destWriter("second")}); err == nil { + t.Fatal("New accepted a second WritesDestination plugin after one that also writes the body") + } +} + +// fnPlugin runs closures from its hooks, so each test states exactly what the +// plugin does and from where. +type fnPlugin struct { + name string + caps PluginCapabilities + onRequest func(*Context) + onResponse func(*Context) +} + +func (p *fnPlugin) Name() string { return p.name } +func (p *fnPlugin) Capabilities() PluginCapabilities { return p.caps } +func (p *fnPlugin) OnRequest(_ context.Context, pctx *Context) Action { + if p.onRequest != nil { + p.onRequest(pctx) + } + return Action{Type: Continue} +} +func (p *fnPlugin) OnResponse(_ context.Context, pctx *Context) Action { + if p.onResponse != nil { + p.onResponse(pctx) + } + return Action{Type: Continue} +} + +// finishPlugin adds an OnFinish hook to fnPlugin. +type finishPlugin struct { + fnPlugin + onFinish func(*Context) +} + +func (p *finishPlugin) OnFinish(_ context.Context, pctx *Context) { p.onFinish(pctx) } + +// redirector is a plugin whose OnRequest runs do; declares sets WritesDestination. +func redirector(declares bool, do func(*Context)) *fnPlugin { + return &fnPlugin{ + name: "router", + caps: PluginCapabilities{WritesDestination: declares, Description: "test"}, + onRequest: do, + } +} + +// mustURL parses s. Every malformed target these tests use parses as a URL; it is +// Redirect that must refuse it. +func mustURL(s string) *url.URL { + u, err := url.Parse(s) + if err != nil { + panic(err) + } + return u +} + +// runRequest runs a one-plugin pipeline's request pass over an outbound context for +// http://a.example, marked redirectable when redirectable is true. +func runRequest(t *testing.T, plugin Plugin, redirectable bool, opts ...Option) (*Pipeline, *Context) { + t.Helper() + p, err := New([]Plugin{plugin}, opts...) + if err != nil { + t.Fatalf("New: %v", err) + } + pctx := &Context{Direction: Outbound, Scheme: "http", Host: "a.example"} + if redirectable { + pctx.MarkRedirectable() + } + p.Run(context.Background(), pctx) + return p, pctx +} + +func outbound(pctx *Context) []Invocation { + if pctx.Extensions.Invocations == nil { + return nil + } + return pctx.Extensions.Invocations.Outbound +} + +// assertUntouched fails unless pctx still points where the client asked. +func assertUntouched(t *testing.T, pctx *Context) { + t.Helper() + if pctx.Scheme != "http" || pctx.Host != "a.example" { + t.Errorf("destination = %s://%s, want the client's http://a.example", pctx.Scheme, pctx.Host) + } + if pctx.Redirected() || pctx.RequestedHost() != "" { + t.Errorf("Redirected() = %v, RequestedHost() = %q; want false and empty", pctx.Redirected(), pctx.RequestedHost()) + } + if scheme, host, ok := pctx.RedirectTarget(); ok || scheme != "" || host != "" { + t.Errorf("RedirectTarget() = %q, %q, %v; want nothing for the listener to apply", scheme, host, ok) + } +} + +func TestRedirect_SendsTheRequestToTarget(t *testing.T) { + var err error + _, pctx := runRequest(t, redirector(true, func(c *Context) { + err = c.Redirect(mustURL("https://b.example:8443")) + }), true) + + if err != nil { + t.Fatalf("Redirect: %v", err) + } + if pctx.Scheme != "https" || pctx.Host != "b.example:8443" { + t.Errorf("destination = %s://%s, want https://b.example:8443", pctx.Scheme, pctx.Host) + } + if !pctx.Redirected() { + t.Error("Redirected() = false after a redirect took effect") + } + if got := pctx.RequestedHost(); got != "a.example" { + t.Errorf("RequestedHost() = %q, want a.example", got) + } + invs := outbound(pctx) + if len(invs) != 1 { + t.Fatalf("invocations = %+v, want exactly the framework's redirect record", invs) + } + inv := invs[0] + if inv.Plugin != "router" || inv.Action != ActionModify || inv.Reason != "redirected" || inv.Shadow || + inv.Details["from"] != "a.example" || inv.Details["to"] != "b.example:8443" { + t.Errorf("invocation = %+v, want router modify/redirected from a.example to b.example:8443", inv) + } +} + +func TestRedirect_ASecondRedirectKeepsTheFirstRequestedHost(t *testing.T) { + _, pctx := runRequest(t, redirector(true, func(c *Context) { + _ = c.Redirect(mustURL("https://b.example")) + _ = c.Redirect(mustURL("https://c.example")) + }), true) + if pctx.Host != "c.example" { + t.Errorf("Host = %q, want the last target c.example", pctx.Host) + } + if got := pctx.RequestedHost(); got != "a.example" { + t.Errorf("RequestedHost() = %q, want the host the CLIENT named, a.example", got) + } +} + +func TestRedirect_BackToTheRequestedHostReportsNoRequestedHost(t *testing.T) { + _, pctx := runRequest(t, redirector(true, func(c *Context) { + _ = c.Redirect(mustURL("https://b.example")) + _ = c.Redirect(mustURL("http://a.example")) + }), true) + if !pctx.Redirected() { + t.Error("Redirected() = false; the listener must still apply the final scheme and host") + } + if scheme, host, ok := pctx.RedirectTarget(); !ok || scheme != "http" || host != "a.example" { + t.Errorf("RedirectTarget() = %q, %q, %v; want the final target http, a.example", scheme, host, ok) + } + if got := pctx.RequestedHost(); got != "" { + t.Errorf("RequestedHost() = %q; the request goes where the client asked, so there is nothing to record", got) + } +} + +func TestRedirectTarget_NothingToApplyWithoutARedirect(t *testing.T) { + _, pctx := runRequest(t, redirector(true, nil), true) + assertUntouched(t, pctx) +} + +// Scheme and Host are exported and any later plugin can write them, declared or not. +// The listener applies RedirectTarget, so such a write must not move the request, +// and RequestedHost must keep naming the host the client asked for. +func TestRedirectTarget_IgnoresALaterWriteToTheExportedFields(t *testing.T) { + steer := &fnPlugin{ + name: "steer", + caps: PluginCapabilities{Description: "test"}, + onRequest: func(c *Context) { c.Scheme, c.Host = "http", "a.example" }, + } + route := redirector(true, func(c *Context) { _ = c.Redirect(mustURL("https://b.example:8443")) }) + p, err := New([]Plugin{route, steer}) + if err != nil { + t.Fatalf("New: %v", err) + } + pctx := &Context{Direction: Outbound, Scheme: "http", Host: "a.example"} + pctx.MarkRedirectable() + p.Run(context.Background(), pctx) + + if scheme, host, ok := pctx.RedirectTarget(); !ok || scheme != "https" || host != "b.example:8443" { + t.Errorf("RedirectTarget() = %q, %q, %v; want the validated target https, b.example:8443", scheme, host, ok) + } + if got := pctx.RequestedHost(); got != "a.example" { + t.Errorf("RequestedHost() = %q, want a.example: the request still goes to b.example:8443", got) + } +} + +func TestRedirect_RefusesMalformedTargets(t *testing.T) { + for _, target := range []string{ + "ftp://sk-secret.b.example", // scheme + "sk-secret.b.example", // no scheme: parses as a path + "https://", // no host + "https:sk-secret@b.example", // opaque, no host + "https://sk-secret@b.example", // credentials belong in headers + "https://user:sk-secret@b.example", // and so does a password + "https://b.example/v1/sk-secret", // a redirect moves the host, never the path + "https://b.example?key=sk-secret", // query + "https://b.example#sk-secret", // fragment + } { + t.Run(target, func(t *testing.T) { + var err error + _, pctx := runRequest(t, redirector(true, func(c *Context) { + err = c.Redirect(mustURL(target)) + }), true) + if err == nil { + t.Fatal("Redirect accepted a malformed target") + } + // A refused target is where a key sits; the error must quote none of it. + if strings.Contains(err.Error(), "sk-secret") || strings.Contains(err.Error(), "b.example") { + t.Errorf("the error quotes the target: %v", err) + } + assertUntouched(t, pctx) + if invs := outbound(pctx); len(invs) != 0 { + t.Errorf("a refused redirect recorded %+v", invs) + } + }) + } +} + +func TestRedirect_RefusesANilTarget(t *testing.T) { + var err error + _, pctx := runRequest(t, redirector(true, func(c *Context) { err = c.Redirect(nil) }), true) + if err == nil { + t.Fatal("Redirect(nil) succeeded") + } + assertUntouched(t, pctx) +} + +func TestRedirect_RefusedWithoutTheCapability(t *testing.T) { + var err error + _, pctx := runRequest(t, redirector(false, func(c *Context) { + err = c.Redirect(mustURL("https://b.example")) + }), true) + if err == nil || !strings.Contains(err.Error(), "WritesDestination") { + t.Fatalf("Redirect from an undeclared plugin = %v, want a refusal naming WritesDestination", err) + } + assertUntouched(t, pctx) +} + +func TestRedirect_RefusedWhereTheListenerCannotHonorIt(t *testing.T) { + var err error + var redirectable bool + _, pctx := runRequest(t, redirector(true, func(c *Context) { + redirectable = c.Redirectable() + err = c.Redirect(mustURL("https://b.example")) + }), false) + if redirectable { + t.Error("Redirectable() = true on a context no listener marked") + } + if err == nil || !strings.Contains(err.Error(), "cannot be redirected") { + t.Fatalf("Redirect on an unmarked context = %v, want a refusal", err) + } + assertUntouched(t, pctx) +} + +func TestRedirect_UnderObserveRecordsAShadowAndChangesNothing(t *testing.T) { + var err error + _, pctx := runRequest(t, redirector(true, func(c *Context) { + err = c.Redirect(mustURL("https://b.example")) + }), true, WithPolicies(ErrorPolicyObserve)) + if err != nil { + t.Fatalf("Redirect under observe: %v; plugin code must not see a difference", err) + } + assertUntouched(t, pctx) + invs := outbound(pctx) + if len(invs) != 1 || invs[0].Reason != "redirected" || !invs[0].Shadow { + t.Errorf("invocations = %+v, want one shadow modify/redirected", invs) + } +} + +func TestRedirect_RefusedInTheResponsePass(t *testing.T) { + var err error + plug := redirector(true, nil) + plug.onResponse = func(c *Context) { err = c.Redirect(mustURL("https://b.example")) } + p, pctx := runRequest(t, plug, true) + p.RunResponse(context.Background(), pctx) + if err == nil || !strings.Contains(err.Error(), "OnRequest") { + t.Fatalf("Redirect from OnResponse = %v, want a refusal", err) + } + assertUntouched(t, pctx) +} + +func TestRedirect_RefusedInOnFinish(t *testing.T) { + var err error + plug := &finishPlugin{ + fnPlugin: *redirector(true, nil), + onFinish: func(c *Context) { err = c.Redirect(mustURL("https://b.example")) }, + } + p, pctx := runRequest(t, plug, true) + p.RunFinish(context.Background(), pctx, Outcome{}) + if err == nil || !strings.Contains(err.Error(), "OnFinish") { + t.Fatalf("Redirect from OnFinish = %v, want a refusal", err) + } + assertUntouched(t, pctx) +} diff --git a/core/pipeline/listener_support.go b/core/pipeline/listener_support.go new file mode 100644 index 000000000..aa88ecc51 --- /dev/null +++ b/core/pipeline/listener_support.go @@ -0,0 +1,41 @@ +package pipeline + +// ListenerSupport says which listener-dependent capabilities the listener a pipeline +// is built for can honor. +// +// Some capabilities are promises only a listener can keep. A plugin may ask for a +// redirect, but only a listener that builds the upstream request itself can send it +// somewhere else; anywhere else the plugin would run, record its decision, and change +// nothing. plugins.BuildWithDeps refuses such a plugin at build time, on startup and on +// reload, so a configuration that cannot work fails before traffic arrives. +// +// Each listener package says what it honors (forwardproxy.Support, reverseproxy.Support, +// extproc.Support) and each binary passes that into its builds. The zero value honors +// none of these capabilities, so a listener that has not considered one refuses the +// plugins that need it rather than silently ignoring them. Adding a capability of this +// kind means adding a field here and a case to Unsupported. +type ListenerSupport struct { + // Listener names the listener in the build error, e.g. "forward proxy". + Listener string + + // Destination: the listener re-originates requests from pctx after the pipeline + // and honors pctx.Redirect. Required by WritesDestination plugins. + Destination bool +} + +// Unsupported returns the name of the first capability caps declares that s cannot +// honor, or "" when it can honor every one. +func (s ListenerSupport) Unsupported(caps PluginCapabilities) string { + if caps.WritesDestination && !s.Destination { + return "WritesDestination" + } + return "" +} + +// Name is Listener, or a description of the listener for a build that named none. +func (s ListenerSupport) Name() string { + if s.Listener == "" { + return "listener this pipeline is built for" + } + return s.Listener +} diff --git a/core/pipeline/listener_support_test.go b/core/pipeline/listener_support_test.go new file mode 100644 index 000000000..9f87c8f2e --- /dev/null +++ b/core/pipeline/listener_support_test.go @@ -0,0 +1,33 @@ +package pipeline + +import "testing" + +func TestListenerSupport_ZeroValueRefusesADestinationWriter(t *testing.T) { + got := ListenerSupport{}.Unsupported(PluginCapabilities{WritesDestination: true}) + if got != "WritesDestination" { + t.Errorf("Unsupported = %q, want WritesDestination: a listener that has not declared support must refuse, not ignore", got) + } +} + +func TestListenerSupport_DestinationHonored(t *testing.T) { + got := ListenerSupport{Destination: true}.Unsupported(PluginCapabilities{WritesDestination: true}) + if got != "" { + t.Errorf("Unsupported = %q with Destination: true, want none", got) + } +} + +func TestListenerSupport_OtherCapabilitiesNeedNoSupport(t *testing.T) { + caps := PluginCapabilities{ReadsBody: true, WritesRequestBody: true, WritesResponseBody: true} + if got := (ListenerSupport{}).Unsupported(caps); got != "" { + t.Errorf("Unsupported = %q for body capabilities, which every listener handles", got) + } +} + +func TestListenerSupport_NameFallsBackForAnUnnamedListener(t *testing.T) { + if got := (ListenerSupport{Listener: "ext_proc listener"}).Name(); got != "ext_proc listener" { + t.Errorf("Name() = %q", got) + } + if got := (ListenerSupport{}).Name(); got == "" { + t.Error("Name() is empty for an unnamed listener; the build error would read badly") + } +} diff --git a/core/pipeline/pipeline.go b/core/pipeline/pipeline.go index 02ef28936..43a884f88 100644 --- a/core/pipeline/pipeline.go +++ b/core/pipeline/pipeline.go @@ -26,6 +26,9 @@ type Pipeline struct { plugins []Plugin policies []ErrorPolicy finishTimeout time.Duration + // redirects[i] is plugins[i]'s WritesDestination, read once at New so Run does + // not ask every plugin for its capabilities on every request. + redirects []bool } // Option configures pipeline construction. @@ -78,7 +81,11 @@ func New(plugins []Plugin, opts ...Option) (*Pipeline, error) { if finishTimeout <= 0 { finishTimeout = DefaultFinishTimeout } - return &Pipeline{plugins: plugins, policies: policies, finishTimeout: finishTimeout}, nil + redirects := make([]bool, len(plugins)) + for i, plugin := range plugins { + redirects[i] = plugin.Capabilities().WritesDestination + } + return &Pipeline{plugins: plugins, policies: policies, finishTimeout: finishTimeout, redirects: redirects}, nil } // Run executes the request phase of the pipeline sequentially. @@ -112,6 +119,7 @@ func (p *Pipeline) Run(ctx context.Context, pctx *Context) Action { return Deny("pipeline.cancelled", "request cancelled") } pctx.setCurrent(plugin.Name(), InvocationPhaseRequest, policy) + pctx.currentMayRedirect = p.redirectsAt(i) pctx.dispatched = append(pctx.dispatched, i) action := plugin.OnRequest(ctx, pctx) pctx.clearCurrent() @@ -248,6 +256,12 @@ func (p *Pipeline) policyAt(i int) ErrorPolicy { return ErrorPolicyEnforce } +// redirectsAt reports whether plugins[i] declares WritesDestination. Bounds-safe, +// like policyAt, so a Pipeline not built by New never panics. +func (p *Pipeline) redirectsAt(i int) bool { + return i < len(p.redirects) && p.redirects[i] +} + // markShadowAndLog records the would-have-denied Invocation as // Shadow=true and emits a WARN log. If the plugin already appended a // deny Invocation (typical for gate plugins that call @@ -598,20 +612,28 @@ func (p *Pipeline) dispatchFinish(parent context.Context, name string, f Finishe f.OnFinish(ctx, pctx) } -// validateCapabilities enforces body-mutation ordering rules: +// validateCapabilities enforces body-mutation ordering rules and the destination rule: // - At most one WritesRequestBody plugin per pipeline — mutation ordering would // otherwise be ambiguous; downstream readers can't tell which version // they're seeing. // - A body reader (ReadsBody) must not follow a body mutator (WritesRequestBody) — // the reader would silently see mutated bytes instead of the originals. +// - At most one WritesDestination plugin per pipeline — a request goes to one +// place, and a second redirect would silently override the first. func validateCapabilities(plugins []Plugin) error { // Each direction admits at most one mutator. The rules are per-direction // because ordering is only ambiguous between two plugins rewriting the // same bytes; a request mutator and a response mutator never collide. - var requestMutator, responseMutator string + var requestMutator, responseMutator, destinationWriter string var firstMutator, readerAfterMutator string for _, plugin := range plugins { caps := plugin.Capabilities().Normalize() + if caps.WritesDestination { + if destinationWriter != "" { + return fmt.Errorf("pipeline: two plugins declare WritesDestination: %q and %q — a request goes to one place; at most one destination writer per pipeline is allowed", destinationWriter, plugin.Name()) + } + destinationWriter = plugin.Name() + } if caps.WritesRequestBody { if requestMutator != "" { return fmt.Errorf("pipeline: two plugins declare WritesRequestBody: %q and %q — mutation ordering would be ambiguous; at most one request-body mutator per pipeline is allowed", requestMutator, plugin.Name()) diff --git a/core/pipeline/plugin.go b/core/pipeline/plugin.go index 380566c26..7fec3a7f8 100644 --- a/core/pipeline/plugin.go +++ b/core/pipeline/plugin.go @@ -52,6 +52,16 @@ type PluginCapabilities struct { // direction, for the same ordering reason as the request side. WritesResponseBody bool + // WritesDestination: the plugin may call pctx.Redirect to send the request to a + // different host than the one the client named. + // + // Only a listener that builds the upstream request itself can honor that, so + // plugins.BuildWithDeps refuses a plugin declaring it for a listener whose + // ListenerSupport says otherwise — on startup and on reload — rather than let + // it run, record a redirect, and change nothing. Pipeline.New admits at most + // one per pipeline: with two, the destination would be whichever ran last. + WritesDestination bool + // Requires names plugins that MUST be present in the same chain // AND appear earlier (lower index). Matches are case-sensitive // plugin Name() strings. A missing or misordered name causes diff --git a/core/pipeline/session.go b/core/pipeline/session.go index 369b8936e..1be05a4a5 100644 --- a/core/pipeline/session.go +++ b/core/pipeline/session.go @@ -156,6 +156,12 @@ type SessionEvent struct { // landed on. Empty when the listener didn't populate pctx.Host. Host string + // RequestedHost is the host the client asked for, present only when a plugin + // redirected the request (pctx.Redirect) and Host is therefore somewhere else. + // Host stays the host the bytes went to, because usage, the cost ledger and + // pricing key on it. + RequestedHost string + // Duration is the wall-clock time from request entry into the listener // to response recording. Zero on request-phase events. On response // events it's computed as now - matching-request.At. On denied events @@ -415,9 +421,12 @@ type sessionEventWire struct { StatusCode int `json:"statusCode,omitempty"` Error *EventError `json:"error,omitempty"` Host string `json:"host,omitempty"` - DurationMs int64 `json:"durationMs,omitempty"` - TLS *EventTLS `json:"tls,omitempty"` - Tunnel bool `json:"tunnel,omitempty"` + // omitempty for the same skew reason as TunnelReason: an old agentop ignores the + // key, and an event from a proxy that predates redirects decodes to "". + RequestedHost string `json:"requestedHost,omitempty"` + DurationMs int64 `json:"durationMs,omitempty"` + TLS *EventTLS `json:"tls,omitempty"` + Tunnel bool `json:"tunnel,omitempty"` // omitempty so both skew directions are safe: an old agentop ignores an unknown // key, and a new agentop against an old proxy sees "" and renders exactly what it // renders today. @@ -441,30 +450,31 @@ type sessionEventWire struct { func (e SessionEvent) MarshalJSON() ([]byte, error) { return json.Marshal(sessionEventWire{ - SessionID: e.SessionID, - Seq: e.Seq, - At: e.At, - Direction: e.Direction, - Phase: e.Phase, - RequestID: e.RequestID, - A2A: e.A2A, - MCP: e.MCP, - Inference: e.Inference, - Invocations: e.Invocations, - Plugins: e.Plugins, - Identity: e.Identity, - StatusCode: e.StatusCode, - Error: e.Error, - Host: e.Host, - DurationMs: e.Duration.Milliseconds(), - TLS: e.TLS, - Tunnel: e.Tunnel, - TunnelReason: e.TunnelReason, - BytesUp: e.BytesUp, - BytesDown: e.BytesDown, - HTTPMethod: e.HTTPMethod, - HTTPPath: e.HTTPPath, - Client: e.Client, + SessionID: e.SessionID, + Seq: e.Seq, + At: e.At, + Direction: e.Direction, + Phase: e.Phase, + RequestID: e.RequestID, + A2A: e.A2A, + MCP: e.MCP, + Inference: e.Inference, + Invocations: e.Invocations, + Plugins: e.Plugins, + Identity: e.Identity, + StatusCode: e.StatusCode, + Error: e.Error, + Host: e.Host, + RequestedHost: e.RequestedHost, + DurationMs: e.Duration.Milliseconds(), + TLS: e.TLS, + Tunnel: e.Tunnel, + TunnelReason: e.TunnelReason, + BytesUp: e.BytesUp, + BytesDown: e.BytesDown, + HTTPMethod: e.HTTPMethod, + HTTPPath: e.HTTPPath, + Client: e.Client, }) } @@ -477,30 +487,31 @@ func (e *SessionEvent) UnmarshalJSON(data []byte) error { return err } *e = SessionEvent{ - SessionID: w.SessionID, - Seq: w.Seq, - At: w.At, - Direction: w.Direction, - Phase: w.Phase, - RequestID: w.RequestID, - A2A: w.A2A, - MCP: w.MCP, - Inference: w.Inference, - Invocations: w.Invocations, - Plugins: w.Plugins, - Identity: w.Identity, - StatusCode: w.StatusCode, - Error: w.Error, - Host: w.Host, - Duration: time.Duration(w.DurationMs) * time.Millisecond, - TLS: w.TLS, - Tunnel: w.Tunnel, - TunnelReason: w.TunnelReason, - BytesUp: w.BytesUp, - BytesDown: w.BytesDown, - HTTPMethod: w.HTTPMethod, - HTTPPath: w.HTTPPath, - Client: w.Client, + SessionID: w.SessionID, + Seq: w.Seq, + At: w.At, + Direction: w.Direction, + Phase: w.Phase, + RequestID: w.RequestID, + A2A: w.A2A, + MCP: w.MCP, + Inference: w.Inference, + Invocations: w.Invocations, + Plugins: w.Plugins, + Identity: w.Identity, + StatusCode: w.StatusCode, + Error: w.Error, + Host: w.Host, + RequestedHost: w.RequestedHost, + Duration: time.Duration(w.DurationMs) * time.Millisecond, + TLS: w.TLS, + Tunnel: w.Tunnel, + TunnelReason: w.TunnelReason, + BytesUp: w.BytesUp, + BytesDown: w.BytesDown, + HTTPMethod: w.HTTPMethod, + HTTPPath: w.HTTPPath, + Client: w.Client, } return nil } diff --git a/core/pipeline/session_test.go b/core/pipeline/session_test.go index 7e8152057..c4691f0c8 100644 --- a/core/pipeline/session_test.go +++ b/core/pipeline/session_test.go @@ -117,7 +117,7 @@ func TestSessionEvent_MarshalJSON_OmitsEmpty(t *testing.T) { // The omitempty half of the version-skew contract: a new proxy that // recorded none of these must not emit the keys at all, so a client // predating them sees the same bytes it always did. - "tunnel", "tunnelReason", "httpMethod", "httpPath", + "tunnel", "tunnelReason", "httpMethod", "httpPath", "requestedHost", } { if strings.Contains(s, `"`+field+`":`) { t.Errorf("expected %q omitted when zero: %s", field, s) @@ -291,7 +291,7 @@ func TestSessionEventWire_HasEveryDomainField(t *testing.T) { func TestSessionEventWire_EveryFieldSerializes(t *testing.T) { ev := SessionEvent{ SessionID: "s", At: time.Now(), Direction: Outbound, Phase: SessionRequest, - RequestID: "r", Host: "h:443", StatusCode: 200, Duration: time.Second, + RequestID: "r", Host: "h:443", RequestedHost: "a:443", StatusCode: 200, Duration: time.Second, Tunnel: true, TunnelReason: TunnelClientRejectedCA, HTTPMethod: "GET", HTTPPath: "/v1/models", } @@ -305,7 +305,7 @@ func TestSessionEventWire_EveryFieldSerializes(t *testing.T) { } for _, key := range []string{ "sessionId", "at", "direction", "phase", "requestId", - "host", "statusCode", "durationMs", "tunnel", "tunnelReason", + "host", "requestedHost", "statusCode", "durationMs", "tunnel", "tunnelReason", "httpMethod", "httpPath", } { if _, ok := got[key]; !ok { @@ -473,3 +473,24 @@ func TestSessionEvent_TunnelBytesRoundTrip(t *testing.T) { t.Errorf("an event with no byte counts emitted the keys: %s", plain) } } + +func TestSessionEvent_RequestedHostRoundTrips(t *testing.T) { + in := SessionEvent{ + At: time.Now().UTC().Truncate(time.Millisecond), Direction: Outbound, Phase: SessionRequest, + Host: "glm-litellm.example.com", RequestedHost: "ete-litellm.example.com", + } + b, err := json.Marshal(in) + if err != nil { + t.Fatalf("Marshal: %v", err) + } + if !strings.Contains(string(b), `"requestedHost":"ete-litellm.example.com"`) { + t.Errorf("wire form %s lacks requestedHost", b) + } + var out SessionEvent + if err := json.Unmarshal(b, &out); err != nil { + t.Fatalf("Unmarshal: %v", err) + } + if out.RequestedHost != in.RequestedHost || out.Host != in.Host { + t.Errorf("round trip = host %q requested %q, want %q and %q", out.Host, out.RequestedHost, in.Host, in.RequestedHost) + } +} diff --git a/core/plugins/README.md b/core/plugins/README.md index 517facfcc..18d0b11d4 100644 --- a/core/plugins/README.md +++ b/core/plugins/README.md @@ -16,6 +16,7 @@ Built-in plugins and the open plugin registry. Plugin authoring docs live under | `a2a-parser` | Parse Agent-to-Agent JSON-RPC traffic into `Extensions.A2A` | | `mcp-parser` | Parse Model Context Protocol traffic into `Extensions.MCP` | | `inference-parser` | Parse OpenAI-style / Ollama inference traffic into `Extensions.Inference` | +| [`inference-router`](../../docs/plugin-catalog.md#inference-router) | Send each coding agent's new sessions to the inference server chosen for it; a session stays on the server it started on | | [`ibac`](../../docs/ibac-plugin.md) | Outbound Intent-Based Access Control: LLM judge denies outbound HTTP that doesn't align with the user's most recent intent. Catches prompt-injection / data-exfiltration attempts | | `opa` | OPA policy evaluation on inbound and outbound requests via bundle download | @@ -50,7 +51,7 @@ named profiles in `scripts/profile-tags`: | Profile | Carries | |---------|---------| | `local` | the three parsers + `tool-prune` | -| `full` | all thirteen non-optional plugins | +| `full` | all fourteen non-optional plugins | | `lite` | jwt-validation, token-exchange, litellm-budget-track, static-inject | | `envoy`, `cpex` | the sets those binaries shipped historically | diff --git a/core/plugins/deps.go b/core/plugins/deps.go index 2049ec3af..2e7238454 100644 --- a/core/plugins/deps.go +++ b/core/plugins/deps.go @@ -18,7 +18,8 @@ import ( // parameter list: BuildWithSPIFFE existed because SPIFFE needed injecting, and // adding pricing the same way would have meant a third near-identical builder. // -// Every field is optional. A zero Deps builds exactly what Build always did. +// Every field is optional. A zero Deps builds exactly what Build always did for every +// plugin that declares no listener-dependent capability (see pipeline.ListenerSupport). type Deps struct { // SPIFFE is injected into plugins implementing spiffe.ProviderConsumer. SPIFFE *spiffe.Provider @@ -27,6 +28,11 @@ type Deps struct { // Long-lived: the reloader rebuilds pipelines but the resolver is swapped in // place, so a plugin from any build sees current rates. See pricing.Registry. Pricing *pricing.Registry + + // Listener says which listener-dependent capabilities the listener this pipeline + // serves can honor; a plugin declaring one it cannot is refused. The zero value + // honors none of them, so a build that names no listener admits no such plugin. + Listener pipeline.ListenerSupport } // BuildWithDeps constructs a pipeline from an ordered list of plugin entries, @@ -71,6 +77,12 @@ func BuildWithDeps(entries []config.PluginEntry, deps Deps, opts ...pipeline.Opt } p := factory() + // Before anything is injected or configured: a plugin the listener cannot honor + // is refused before it can build state or start work. + if c := deps.Listener.Unsupported(p.Capabilities()); c != "" { + return nil, fmt.Errorf("plugin %q declares %s, which the %s does not honor", e.Name, c, deps.Listener.Name()) + } + if c, ok := p.(spiffe.ProviderConsumer); ok && deps.SPIFFE != nil { c.SetSPIFFEProvider(deps.SPIFFE) } diff --git a/core/plugins/deps_test.go b/core/plugins/deps_test.go index c25e86cb1..0fb227dc6 100644 --- a/core/plugins/deps_test.go +++ b/core/plugins/deps_test.go @@ -164,3 +164,62 @@ func TestPricingConsumerPlugins_TracksTheCostOwner(t *testing.T) { t.Errorf("litellm-budget-track is reported as a pricing consumer; it bills a settled figure and computes nothing: %v", consumers) } } + +// destinationPlugin declares WritesDestination and does nothing else. +type destinationPlugin struct{ name string } + +func (p *destinationPlugin) Name() string { return p.name } +func (p *destinationPlugin) Capabilities() pipeline.PluginCapabilities { + return pipeline.PluginCapabilities{WritesDestination: true, Description: "test"} +} +func (p *destinationPlugin) OnRequest(context.Context, *pipeline.Context) pipeline.Action { + return pipeline.Action{Type: pipeline.Continue} +} +func (p *destinationPlugin) OnResponse(context.Context, *pipeline.Context) pipeline.Action { + return pipeline.Action{Type: pipeline.Continue} +} + +func registerDestination(t *testing.T, name string) []config.PluginEntry { + t.Helper() + RegisterPlugin(name, func() pipeline.Plugin { return &destinationPlugin{name: name} }) + t.Cleanup(func() { UnregisterPlugin(name) }) + return []config.PluginEntry{{Name: name}} +} + +func TestBuildWithDeps_RefusesADestinationWriterTheListenerCannotHonor(t *testing.T) { + entries := registerDestination(t, "test-dest-refused") + _, err := BuildWithDeps(entries, Deps{Listener: pipeline.ListenerSupport{Listener: "reverse proxy"}}) + if err == nil { + t.Fatal("BuildWithDeps admitted a WritesDestination plugin for a listener that cannot redirect") + } + for _, want := range []string{`"test-dest-refused"`, "WritesDestination", "reverse proxy"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not mention %s", err, want) + } + } +} + +func TestBuildWithDeps_ZeroDepsRefusesADestinationWriter(t *testing.T) { + entries := registerDestination(t, "test-dest-zero") + if _, err := BuildWithDeps(entries, Deps{}); err == nil { + t.Fatal("a build that names no listener admitted a WritesDestination plugin") + } +} + +func TestBuildWithDeps_AdmitsADestinationWriterWhereHonored(t *testing.T) { + entries := registerDestination(t, "test-dest-ok") + deps := Deps{Listener: pipeline.ListenerSupport{Listener: "forward proxy", Destination: true}} + if _, err := BuildWithDeps(entries, deps); err != nil { + t.Fatalf("BuildWithDeps refused a WritesDestination plugin the listener honors: %v", err) + } +} + +// An entry under on_error: off is dropped before it is built, so a plugin the +// listener cannot honor is no obstacle while it is switched off. +func TestBuildWithDeps_DoesNotCheckAnEntryThatIsOff(t *testing.T) { + entries := registerDestination(t, "test-dest-off") + entries[0].OnError = pipeline.ErrorPolicyOff + if _, err := BuildWithDeps(entries, Deps{}); err != nil { + t.Fatalf("BuildWithDeps checked an entry that is off: %v", err) + } +} diff --git a/core/plugins/inferencerouter/plugin.go b/core/plugins/inferencerouter/plugin.go new file mode 100644 index 000000000..8fff2c34a --- /dev/null +++ b/core/plugins/inferencerouter/plugin.go @@ -0,0 +1,406 @@ +// Package inferencerouter is the inference-router plugin: it sends a coding agent's +// new sessions to the inference server chosen for that agent, and keeps every +// session on the server it started on. +// +// The choice is plugin config — servers by name, and agents by name to a server — +// changed by editing it and letting the proxy hot-reload. Sessions are pinned in the +// process-scoped store (pctx.Shared), which a reload does not replace, so changing +// an agent's server moves only the sessions it has not started yet. +// +// Place it last in the outbound chain, which is where agentop puts it. A redirect +// moves pctx.Host, so the plugins before it decide on the host the client asked for +// and any plugin after it would see the server's host instead. Nothing should follow +// it that keys on the host. +// +// A routed request is always redirected to its server's own scheme and host, even +// when it already names that host. pctx.Host is the request's Host header, the +// client's word, and on a TLS-bridged request the forward proxy dials the CONNECT +// authority, which need not be that host. A router that skipped the redirect because +// the Host header already named the server would hand the server's key to whatever +// the client CONNECTed to. With the redirect the listener dials RedirectTarget, the +// server, whatever the Host header or the CONNECT said. +// +// The server's key replaces the client's only when pctx.Redirected() reports that the +// redirect took effect. Under on_error: observe, Redirect returns nil and moves +// nothing, so a key set on the strength of that nil would go to the host the client +// named. +// +// A session is pinned by where its first request went, not by the choice made for +// it: to the server when the request was redirected there; to "not routed" when it +// stayed where the client sent it, because the agent is not routed or because the +// router runs under on_error: observe; and not at all when the redirect failed, so +// the session's next request decides again. A pin by choice would hold a session +// started under observe to a server it never used, and turning enforce on would then +// move it there mid-conversation, which is the switch the pin exists to prevent. +// +// The first request the router sees is not always a session's first. A session +// already running when routing is first configured, quiet while it was, has no pin, +// and neither has one whose pin lapsed. The session store's history tells such a +// session from a new one: on a pin miss for a routed agent, the latest earlier +// inference request that agent sent in the session to a server's host keeps the +// session on that server, and only a session with no such request is new and goes +// to its agent's current server. Without this, the documented first setup — add +// the servers, then route Claude Code — would move every conversation that sent +// nothing between the router's arrival and the route to the new server on its next +// turn. A request to any other host is no evidence: the router never routes one, so +// it says nothing about which server the session is on. Reading it as "not routed" +// would leave an agent that switches providers inside a session, as OpenCode does, +// unrouted for good once its first request went elsewhere, still sending its own +// key to the server it then addresses. Only a non-empty history is evidence: a +// view is empty for a session the store has recorded nothing of yet. After a +// restart the store holds nothing from before it, and the pins, which live in +// memory, are gone too, so a running session's next request then looks new and can +// move; so can a quiet one the store has evicted before the router pinned it. +// +// A pin is its agent's. The session id is the listener's answer, and another +// agent's request can be filed under it — by process attribution, which files a +// command an agent runs under that agent's session and is on by default on a +// laptop; by the ActiveSession fallback with client affinity off; or by a header id +// two clients share. Such a request is decided as if the session were unpinned, from +// its own agent's history and choice, and leaves the pin alone. Honouring the pin for +// it would hand it the session's server and key. +// +// The listener's synthetic sessions are never pinned: the default bucket, and the +// pending: buckets an agent's calls collect in before its session is known. +// Each holds many conversations rather than one, so a pin would hold every later +// conversation filed there to the first one's server. Their requests follow the +// agent's current server, unpinned, as a request with no session does. +package inferencerouter + +import ( + "context" + "encoding/json" + "fmt" + "log/slog" + "net/http" + "net/url" + "strings" + "sync" + "time" + + "github.com/rossoctl/cortex/core/pipeline" + "github.com/rossoctl/cortex/core/plugins" + "github.com/rossoctl/cortex/core/plugins/inferencerouter/routerconfig" + "github.com/rossoctl/cortex/core/session" +) + +// Name is the plugin's registered name. +const Name = "inference-router" + +const ( + // pinPrefix namespaces the plugin's keys in the shared store. + pinPrefix = Name + "/pin/" + + // pinTTL is how long a pin outlives a session's last request. It slides: every + // request renews it, so only a session idle this long forgets its server. + pinTTL = 30 * 24 * time.Hour + + // codeUnavailable maps to 503 in pipeline's code table. A session whose server + // cannot be reached through the router is the server being unavailable to it, + // not the client's mistake. + codeUnavailable = "upstream.unreachable" +) + +// The pin detail on every record that resolved a server. +const ( + pinNew = "new" // this request pinned the session + pinExisting = "existing" // the session was already pinned + pinNone = "none" // nothing was pinned: no session, a synthetic one, no store, a failed redirect, or another agent's pin +) + +// route is one configured server, ready to redirect to. +type route struct { + endpoint routerconfig.Endpoint + key string +} + +// target is the server's own scheme and host, where every request routed to it is +// redirected. +func (r route) target() *url.URL { + return &url.URL{Scheme: r.endpoint.Scheme, Host: r.endpoint.Host} +} + +// Router is the plugin. Built by Configure; the zero value routes nothing. +type Router struct { + servers map[string]route // by server name + byHost map[string]string // every server's Endpoint.Hostname, to its name + agents map[string]string // agent name to server name + + // noStore makes the "pins are off" warning once per instance rather than once + // per request. + noStore sync.Once +} + +// New constructs an unconfigured plugin. +func New() *Router { return &Router{} } + +func init() { + plugins.RegisterPlugin(Name, func() pipeline.Plugin { return New() }) +} + +func (p *Router) Name() string { return Name } + +func (p *Router) Capabilities() pipeline.PluginCapabilities { + return pipeline.PluginCapabilities{ + WritesDestination: true, + Description: "Sends each agent's new sessions to its chosen inference server.", + } +} + +// ConfigSchema implements pipeline.SchemaProvider. servers and agents are maps, which +// pipeline.SchemaOf renders as type "unknown". +func (p *Router) ConfigSchema() []pipeline.FieldSchema { + return pipeline.SchemaOf(routerconfig.Config{}) +} + +// Configure decodes and validates the config, then builds the routing tables. A +// failed Configure leaves the previous tables in place; BuildWithDeps discards the +// instance anyway. +func (p *Router) Configure(raw json.RawMessage) error { + c, err := routerconfig.Decode(raw) + if err != nil { + return fmt.Errorf("inference-router config: %w", err) + } + servers := make(map[string]route, len(c.Servers)) + byHost := make(map[string]string, len(c.Servers)) // one name per host: Decode refuses two servers on one + for name, s := range c.Servers { + ep, err := routerconfig.ParseURL(s.URL) + if err != nil { // Decode already accepted it; kept so a drift between the two fails here + return fmt.Errorf("inference-router config: servers.%s.url: %w", name, err) + } + if ep.PlaintextRemote() { + slog.Warn("inference-router: server is plain http on another machine, so every request routed to it "+ + "crosses the network decrypted, its key and prompt included; use https unless the network is trusted", + "server", name, "url", ep.URL()) + } + servers[name] = route{endpoint: ep, key: s.Key} + byHost[ep.Hostname] = name + } + p.servers, p.byHost, p.agents = servers, byHost, c.Agents + return nil +} + +// OnRequest routes one request. See the package doc for what it decides and why. +func (p *Router) OnRequest(_ context.Context, pctx *pipeline.Context) pipeline.Action { + cont := pipeline.Action{Type: pipeline.Continue} + + // A CONNECT or a transparent connection is dialed where the client chose, + // whatever a plugin does. Pinning on one would pin a session on a request that + // cannot be routed, so it is left before anything is decided. + if !pctx.Redirectable() { + pctx.Skip("not_redirectable") + return cont + } + // Every path on a server's host is handled — /v1/messages, count_tokens, + // /v1/models — because they all belong to the server the session is on. + if _, ok := p.byHost[routerconfig.Hostname(pctx.Host)]; !ok { + pctx.Skip("not_an_inference_server") + return cont + } + + name, pin := p.serverFor(pctx) + if name == "" { + // Not routed: the request, its key included, stays exactly as the client sent it. + pin.settle("") + pctx.Record(pipeline.Invocation{Action: pipeline.ActionSkip, Reason: "not_routed", + Details: map[string]string{"pin": pin.state}}) + return cont + } + // Built where each record is made, since a failed redirect changes the pin. + details := func() map[string]string { return map[string]string{"server": name, "pin": pin.state} } + srv, ok := p.servers[name] + if !ok { + // Removed since the session was pinned, by agentop server remove or a hand edit. + // Moving the conversation to whatever the agent uses now is the switch this + // plugin exists to prevent. + pctx.Record(pipeline.Invocation{Action: pipeline.ActionDeny, Reason: "pinned_server_removed", Details: details()}) + return pipeline.Deny(codeUnavailable, fmt.Sprintf( + "this session is pinned to inference server %q, which is no longer configured; add it back, or start a new session", name)) + } + + // Redirected even when the Host header already names the server: the listener + // dials the redirect target, and without one it dials what the client chose, + // which on a bridged request is the CONNECT authority and not the Host header. + if err := pctx.Redirect(srv.target()); err != nil { + // The request went nowhere, so a first request pins nothing: pinning the server + // would hold the session to one it never reached, and the next request can + // decide again. + pin.forgo() + pctx.Record(pipeline.Invocation{Action: pipeline.ActionDeny, Reason: "redirect_failed", Details: details()}) + return pipeline.Deny(codeUnavailable, fmt.Sprintf("inference-router could not send this request to %q: %v", name, err)) + } + // Redirect returns nil under on_error: observe without moving anything. The + // server's key must then stay off the request, which still goes where the client + // sent it. Redirected is the signal that it moved, and it is this router's + // redirect: a pipeline admits one WritesDestination plugin. + if !pctx.Redirected() { + // The request stayed put, so that is what a first request pins: a session that + // started under observe stays put once enforce is on. + pin.settle("") + pctx.Record(pipeline.Invocation{Action: pipeline.ActionObserve, Reason: "would_route", Details: details()}) + return cont + } + pin.settle(name) + setKey(pctx, srv.key) + pctx.Record(pipeline.Invocation{Action: pipeline.ActionModify, Reason: "routed", Details: details()}) + return cont +} + +func (p *Router) OnResponse(_ context.Context, _ *pipeline.Context) pipeline.Action { + return pipeline.Action{Type: pipeline.Continue} +} + +// pin is a session's pin as the shared store holds it: the agent whose session it +// is, and the server the session is on, "" for not routed. +type pin struct { + agent string + server string +} + +// pinning is one request's session pin: how it was decided, and for a +// session's first request where to store the outcome once OnRequest knows it. +type pinning struct { + state string // pinNew, pinExisting or pinNone + store pipeline.SharedStore + key string + agent string +} + +// settle pins a session on its first request to where that request went: server +// when it was redirected there, "" when it stayed where the client sent it. A +// session already pinned keeps its pin, which serverFor renewed. +func (pn *pinning) settle(server string) { + if pn.state == pinNew { + pn.store.Put(pn.key, pin{agent: pn.agent, server: server}, pinTTL) + } +} + +// forgo leaves a session unpinned after a first request that went nowhere, so its +// next request decides again. A session already pinned keeps its pin. +func (pn *pinning) forgo() { + if pn.state == pinNew { + pn.state = pinNone + } +} + +// serverFor is the server the request's session uses, "" for not routed, and the +// session's pin. +// +// A session its agent already pinned uses its pin, "not routed" included, so routing +// an agent later does not move the sessions it already has running; every request +// renews the pin. Otherwise the session is decided by unpinned, and the pin comes +// back pinNew for OnRequest to settle by where the request went (see the package +// doc) — except when the pin is another agent's, which this request must neither +// follow nor overwrite, so it comes back pinNone. Without a session to pin, whether +// none, a synthetic one or no store, the agent's choice applies and nothing is +// pinned: a synthetic session's history is many conversations', not this one's. +// +// Two first requests of one session racing can both miss and both store; they store +// the same outcome unless a reload lands between them, which is the case the pin +// cannot rule out and does not need to. +func (p *Router) serverFor(pctx *pipeline.Context) (string, pinning) { + agent := agentOf(pctx) + choice := p.agents[agent] + if pctx.Session == nil || pctx.Session.ID == "" || synthetic(pctx.Session.ID) { + return choice, pinning{state: pinNone} + } + if pctx.Shared == nil { + p.noStore.Do(func() { + slog.Warn("inference-router: this binary wires no process store, so sessions are not pinned: " + + "each request follows its agent's current server, and changing it moves running sessions") + }) + return choice, pinning{state: pinNone} + } + key := pinPrefix + pctx.Session.ID + if v, ok := pctx.Shared.Get(key); ok { + if pn, ok := v.(pin); ok { + if pn.agent != agent { + return p.unpinned(pctx.Session, agent, choice), pinning{state: pinNone} + } + pctx.Shared.Put(key, pn, pinTTL) + return pn.server, pinning{state: pinExisting} + } + } + return p.unpinned(pctx.Session, agent, choice), pinning{state: pinNew, store: pctx.Shared, key: key, agent: agent} +} + +// unpinned is the server for a request in a session its agent holds no pin on: the +// server the session's history shows it already uses, or for a new session — no +// earlier request to any server — the agent's current choice. An agent that is not +// routed is left alone, whatever its history. +func (p *Router) unpinned(s *pipeline.SessionView, agent, choice string) string { + if choice == "" { + return "" + } + if server, ok := p.wentTo(s.Events, agent); ok { + return server + } + return choice +} + +// wentTo is the server the latest earlier inference request agent sent in events +// to a server's host went to; ok is false when there is no such request, which is +// what makes a session new. +// +// Only an outbound request row a parser read as inference counts, and only one that +// reached a server. A tunnel row, a count_tokens or a /v1/models request no parser +// claimed, and a denied request, which went nowhere, say nothing about where the +// conversation is; nor does a request to any other host, which the router never +// routes (see the package doc). Another agent's row says nothing about this agent's +// conversation, and following it would hand this request that agent's server and +// key. A row's Host is where the bytes went, the server's host for a routed request, +// because the listener records it after the redirect; RequestedHost, where the +// client asked to go, is not where the session is. +func (p *Router) wentTo(events []pipeline.SessionEvent, agent string) (server string, ok bool) { + for i := len(events) - 1; i >= 0; i-- { + e := &events[i] + if e.Direction != pipeline.Outbound || e.Phase != pipeline.SessionRequest || e.Inference == nil || + pipeline.AgentName(e.Client.Label()) != agent { + continue + } + if server, ok := p.byHost[routerconfig.Hostname(e.Host)]; ok { + return server, true + } + } + return "", false +} + +// synthetic reports a session id the listener files traffic under when it knows no +// one conversation: the default bucket, or an agent's pending bucket. See the +// package doc for why those are never pinned. +func synthetic(id string) bool { + return id == session.DefaultSessionID || strings.HasPrefix(id, session.PendingPrefix) +} + +// agentOf is the request's agent as the session store and agentop name it, or "" +// when the request carried no User-Agent. +func agentOf(pctx *pipeline.Context) string { + agent := pipeline.AgentName(pctx.ClientInfo().Label()) + if agent == pipeline.UnknownClientLabel { + return "" + } + return agent +} + +// setKey puts key in the header the client authenticated with: X-Api-Key when it +// sent one, Authorization otherwise, both when it sent both. That keeps the header +// Claude Code uses whichever of ANTHROPIC_API_KEY or ANTHROPIC_AUTH_TOKEN set it up. +func setKey(pctx *pipeline.Context, key string) { + if pctx.Headers == nil { + pctx.Headers = http.Header{} + } + h := pctx.Headers + apiKey := len(h.Values("X-Api-Key")) > 0 + if apiKey { + h.Set("X-Api-Key", key) + } + if !apiKey || len(h.Values("Authorization")) > 0 { + h.Set("Authorization", "Bearer "+key) + } +} + +var ( + _ pipeline.Plugin = (*Router)(nil) + _ pipeline.Configurable = (*Router)(nil) + _ pipeline.SchemaProvider = (*Router)(nil) +) diff --git a/core/plugins/inferencerouter/plugin_test.go b/core/plugins/inferencerouter/plugin_test.go new file mode 100644 index 000000000..3f7824899 --- /dev/null +++ b/core/plugins/inferencerouter/plugin_test.go @@ -0,0 +1,748 @@ +package inferencerouter + +import ( + "bytes" + "context" + "encoding/json" + "log/slog" + "net/http" + "strings" + "testing" + + "github.com/rossoctl/cortex/core/memstore" + "github.com/rossoctl/cortex/core/pipeline" + "github.com/rossoctl/cortex/core/plugins" + "github.com/rossoctl/cortex/core/plugins/inferencerouter/routerconfig" + "github.com/rossoctl/cortex/core/session" +) + +const ( + claudeUA = "claude-cli/2.1.286 (external, cli)" + opencodeUA = "opencode/latest/2.0.21/cli" + + eteHost = "ete.example.com" + glmHost = "glm.example.com:8443" +) + +// routerConfig is two servers and the agents block given, which may be empty. +func routerConfig(agents string) string { + return `{ + "servers": { + "ete": {"url": "https://ete.example.com", "key": "ete-key"}, + "glm": {"url": "https://glm.example.com:8443", "key": "glm-key"} + }, + "agents": {` + agents + `} + }` +} + +// build configures a router and wraps it in a one-plugin pipeline. A redirect is +// accepted only from inside Pipeline.Run, so OnRequest is never called directly. +func build(t *testing.T, config string, opts ...pipeline.Option) *pipeline.Pipeline { + t.Helper() + r := New() + if err := r.Configure(json.RawMessage(config)); err != nil { + t.Fatalf("Configure: %v", err) + } + p, err := pipeline.New([]pipeline.Plugin{r}, opts...) + if err != nil { + t.Fatalf("pipeline.New: %v", err) + } + return p +} + +func newStore(t *testing.T) *memstore.Store { + t.Helper() + s := memstore.New() + t.Cleanup(s.Close) + return s +} + +// request is a context the forward proxy would build for a decrypted request to +// host from an agent with User-Agent ua, in session (none when ""), carrying the +// client's own key as a bearer token. +func request(store pipeline.SharedStore, host, ua, session string) *pipeline.Context { + pctx := &pipeline.Context{ + Direction: pipeline.Outbound, + Method: http.MethodPost, + Scheme: "https", + Host: host, + Path: "/v1/messages", + Headers: http.Header{"User-Agent": {ua}, "Authorization": {"Bearer client-key"}}, + } + if store != nil { + pctx.Shared = store + } + if session != "" { + pctx.Session = &pipeline.SessionView{ID: session} + } + pctx.ResolveClient() + pctx.MarkRedirectable() + return pctx +} + +func run(t *testing.T, p *pipeline.Pipeline, pctx *pipeline.Context) pipeline.Action { + t.Helper() + return p.Run(context.Background(), pctx) +} + +// routerRecord is the router's own invocation: the last one on the outbound pass, +// after the framework's modify/redirected when there was a redirect. +func routerRecord(t *testing.T, pctx *pipeline.Context) pipeline.Invocation { + t.Helper() + if pctx.Extensions.Invocations == nil || len(pctx.Extensions.Invocations.Outbound) == 0 { + t.Fatal("the router recorded nothing") + } + invs := pctx.Extensions.Invocations.Outbound + return invs[len(invs)-1] +} + +func assertRecord(t *testing.T, pctx *pipeline.Context, action pipeline.InvocationAction, reason string, details map[string]string) { + t.Helper() + inv := routerRecord(t, pctx) + if inv.Plugin != Name || inv.Action != action || inv.Reason != reason { + t.Errorf("record = %s %s/%s, want %s %s/%s", inv.Plugin, inv.Action, inv.Reason, Name, action, reason) + } + for k, want := range details { + if got := inv.Details[k]; got != want { + t.Errorf("Details[%q] = %q, want %q (all: %v)", k, got, want, inv.Details) + } + } +} + +// assertUntouched fails unless pctx still goes to host with the client's own key. +func assertUntouched(t *testing.T, pctx *pipeline.Context, host string) { + t.Helper() + if pctx.Host != host || pctx.Redirected() { + t.Errorf("Host = %q, Redirected = %v; want %q and no redirect", pctx.Host, pctx.Redirected(), host) + } + if got := pctx.Headers.Get("Authorization"); got != "Bearer client-key" { + t.Errorf("Authorization = %q, want the client's own key", got) + } +} + +// assertRouted fails unless pctx goes to host with key as its bearer token. +func assertRouted(t *testing.T, pctx *pipeline.Context, host, key string) { + t.Helper() + if pctx.Host != host { + t.Errorf("Host = %q, want %q", pctx.Host, host) + } + if got := pctx.Headers.Get("Authorization"); got != "Bearer "+key { + t.Errorf("Authorization = %q, want Bearer %s", got, key) + } +} + +// pinOf is the server session is pinned to, "" for not routed, and whether it is +// pinned at all. A pin is its agent's, and every pin these tests make is +// claude-code's but where a test checks the agent itself. +func pinOf(t *testing.T, store *memstore.Store, session string) (string, bool) { + t.Helper() + v, ok := store.Get(pinPrefix + session) + if !ok { + return "", false + } + pn, ok := v.(pin) + if !ok { + t.Fatalf("pin for %s is a %T, want a pin", session, v) + } + return pn.server, true +} + +func TestRouter_IgnoresAHostThatIsNoServer(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(store, "api.anthropic.com", claudeUA, "s1") + run(t, p, pctx) + + assertUntouched(t, pctx, "api.anthropic.com") + assertRecord(t, pctx, pipeline.ActionSkip, "not_an_inference_server", nil) + if _, ok := pinOf(t, store, "s1"); ok { + t.Error("a request to no server pinned its session") + } +} + +// A CONNECT is dialed where the client chose. Pinning on it would pin a session on +// a request that cannot be routed. +func TestRouter_IgnoresAContextTheListenerCannotRedirect(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := &pipeline.Context{ + Direction: pipeline.Outbound, Method: http.MethodConnect, Scheme: "tcp", Host: eteHost + ":443", + Headers: http.Header{"User-Agent": {claudeUA}}, Shared: store, Session: &pipeline.SessionView{ID: "s1"}, + } + run(t, p, pctx) + + assertRecord(t, pctx, pipeline.ActionSkip, "not_redirectable", nil) + if _, ok := pinOf(t, store, "s1"); ok { + t.Error("a CONNECT pinned its session") + } +} + +func TestRouter_LeavesAnUnlistedAgentAloneAndPinsItAsNotRouted(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(store, eteHost, opencodeUA, "s1") + run(t, p, pctx) + + assertUntouched(t, pctx, eteHost) + assertRecord(t, pctx, pipeline.ActionSkip, "not_routed", map[string]string{"pin": pinNew}) + if pin, ok := pinOf(t, store, "s1"); !ok || pin != "" { + t.Errorf("pin = %q, %v; want the session pinned as not routed (\"\")", pin, ok) + } +} + +func TestRouter_DoesNotRouteARequestWithNoUserAgent(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(newStore(t), eteHost, "", "s1") + run(t, p, pctx) + + assertUntouched(t, pctx, eteHost) + assertRecord(t, pctx, pipeline.ActionSkip, "not_routed", nil) +} + +func TestRouter_RoutesAListedAgentsSessionToItsServer(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(store, eteHost, claudeUA, "s1") + if a := run(t, p, pctx); a.Type != pipeline.Continue { + t.Fatalf("action = %+v, want Continue", a) + } + + assertRouted(t, pctx, glmHost, "glm-key") + if !pctx.Redirected() || pctx.RequestedHost() != eteHost { + t.Errorf("Redirected = %v, RequestedHost = %q; want a redirect from %s", pctx.Redirected(), pctx.RequestedHost(), eteHost) + } + assertRecord(t, pctx, pipeline.ActionModify, "routed", map[string]string{"server": "glm", "pin": pinNew}) + if pin, _ := pinOf(t, store, "s1"); pin != "glm" { + t.Errorf("pin = %q, want glm", pin) + } +} + +// Every path on a server's host belongs to the session's server, not only the +// inference call. +func TestRouter_RoutesEveryPathOnAServersHost(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(newStore(t), eteHost, claudeUA, "s1") + pctx.Method, pctx.Path = http.MethodGet, "/v1/models" + run(t, p, pctx) + + assertRouted(t, pctx, glmHost, "glm-key") +} + +func TestRouter_PutsTheKeyInTheHeaderTheClientSent(t *testing.T) { + for _, tc := range []struct { + name string + sent http.Header + wantAPIKey, wantAu string + }{ + {"x-api-key only", http.Header{"X-Api-Key": {"client-key"}}, "glm-key", ""}, + {"authorization only", http.Header{"Authorization": {"Bearer client-key"}}, "", "Bearer glm-key"}, + {"both", http.Header{"X-Api-Key": {"client-key"}, "Authorization": {"Bearer client-key"}}, "glm-key", "Bearer glm-key"}, + {"neither", http.Header{}, "", "Bearer glm-key"}, + } { + t.Run(tc.name, func(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(newStore(t), eteHost, claudeUA, "s1") + pctx.Headers = tc.sent + pctx.Headers.Set("User-Agent", claudeUA) + run(t, p, pctx) + + if got := pctx.Headers.Get("X-Api-Key"); got != tc.wantAPIKey { + t.Errorf("X-Api-Key = %q, want %q", got, tc.wantAPIKey) + } + if got := pctx.Headers.Get("Authorization"); got != tc.wantAu { + t.Errorf("Authorization = %q, want %q", got, tc.wantAu) + } + }) + } +} + +// The switching story rests on these two: a choice made now applies only to +// sessions that have not started. +func TestRouter_RoutingAnAgentLaterDoesNotMoveASessionPinnedAsNotRouted(t *testing.T) { + store := newStore(t) + before := build(t, routerConfig(``)) + run(t, before, request(store, eteHost, claudeUA, "running")) + + after := build(t, routerConfig(`"claude-code": "glm"`)) + running := request(store, eteHost, claudeUA, "running") + run(t, after, running) + assertUntouched(t, running, eteHost) + assertRecord(t, running, pipeline.ActionSkip, "not_routed", map[string]string{"pin": pinExisting}) + + fresh := request(store, eteHost, claudeUA, "fresh") + run(t, after, fresh) + assertRouted(t, fresh, glmHost, "glm-key") +} + +func TestRouter_MovingAnAgentDoesNotMoveASessionPinnedToTheFirstServer(t *testing.T) { + store := newStore(t) + before := build(t, routerConfig(`"claude-code": "glm"`)) + run(t, before, request(store, eteHost, claudeUA, "running")) + + after := build(t, routerConfig(`"claude-code": "ete"`)) + running := request(store, eteHost, claudeUA, "running") + run(t, after, running) + assertRouted(t, running, glmHost, "glm-key") + assertRecord(t, running, pipeline.ActionModify, "routed", map[string]string{"server": "glm", "pin": pinExisting}) + + fresh := request(store, eteHost, claudeUA, "fresh") + run(t, after, fresh) + assertRouted(t, fresh, eteHost, "ete-key") +} + +func TestRouter_ASessionlessRequestFollowsTheCurrentChoiceUnpinned(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(store, eteHost, claudeUA, "") + run(t, p, pctx) + + assertRouted(t, pctx, glmHost, "glm-key") + assertRecord(t, pctx, pipeline.ActionModify, "routed", map[string]string{"pin": pinNone}) +} + +func TestRouter_APinToARemovedServerIsA503(t *testing.T) { + store := newStore(t) + store.Put(pinPrefix+"s1", pin{agent: "claude-code", server: "east"}, pinTTL) + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(store, eteHost, claudeUA, "s1") + a := run(t, p, pctx) + + if a.Type != pipeline.Reject { + t.Fatalf("action = %+v, want Reject", a) + } + status, _, body := a.Violation.Render() + if status != http.StatusServiceUnavailable { + t.Errorf("status = %d, want 503", status) + } + if want := `pinned to inference server \"east\", which is no longer configured`; !strings.Contains(string(body), want) { + t.Errorf("body = %s, want it to contain %s", body, want) + } + assertRecord(t, pctx, pipeline.ActionDeny, "pinned_server_removed", map[string]string{"server": "east", "pin": pinExisting}) + assertUntouched(t, pctx, eteHost) +} + +// assertRedirectedTo fails unless the listener will dial scheme://host for pctx: +// RedirectTarget is what it applies, whatever the Host header said. +func assertRedirectedTo(t *testing.T, pctx *pipeline.Context, scheme, host string) { + t.Helper() + gotScheme, gotHost, ok := pctx.RedirectTarget() + if !ok || gotScheme != scheme || gotHost != host { + t.Errorf("RedirectTarget = %q %q %v, want %q %q true", gotScheme, gotHost, ok, scheme, host) + } +} + +// Claude Code pointed at the server it is routed to is still redirected there. The +// Host header is the client's word: on a TLS-bridged request the listener otherwise +// dials the CONNECT authority, which need not be this host, and the server's key +// would go with it. Including when the request names the default port the server's +// URL leaves out. +func TestRouter_ARoutedRequestOnItsServersHostIsStillRedirected(t *testing.T) { + for _, host := range []string{eteHost, "ETE.example.com:443"} { + t.Run(host, func(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "ete"`)) + pctx := request(newStore(t), host, claudeUA, "s1") + run(t, p, pctx) + + if !pctx.Redirected() { + t.Fatal("Redirected = false: a routed request must be redirected even on its server's own host") + } + assertRedirectedTo(t, pctx, "https", eteHost) + if got := pctx.Headers.Get("Authorization"); got != "Bearer ete-key" { + t.Errorf("Authorization = %q, want Bearer ete-key", got) + } + assertRecord(t, pctx, pipeline.ActionModify, "routed", map[string]string{"server": "ete", "pin": pinNew}) + }) + } + // A redirect back to the host the client named records no requested host, so the + // session row does not show a move that did not happen, and the framework's record + // says it went from the server to the server. + pctx := request(newStore(t), eteHost, claudeUA, "s1") + run(t, build(t, routerConfig(`"claude-code": "ete"`)), pctx) + if got := pctx.RequestedHost(); got != "" { + t.Errorf("RequestedHost = %q, want \"\" for a request already on the server's host", got) + } + if invs := pctx.Extensions.Invocations.Outbound; len(invs) != 2 || invs[0].Reason != "redirected" || + invs[0].Details["from"] != eteHost || invs[0].Details["to"] != eteHost { + t.Errorf("invocations = %+v, want modify/redirected from %s to %s, then routed", invs, eteHost, eteHost) + } +} + +// A plain-http request naming an https server's host is moved to https before the +// key is set, so the key never crosses the network in plaintext. +func TestRouter_APlaintextRequestOnAnHTTPSServersHostIsRedirectedToHTTPS(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "ete"`)) + pctx := request(newStore(t), eteHost, claudeUA, "s1") + pctx.Scheme = "http" + run(t, p, pctx) + + assertRedirectedTo(t, pctx, "https", eteHost) + if pctx.Scheme != "https" { + t.Errorf("Scheme = %q, want https", pctx.Scheme) + } + if got := pctx.Headers.Get("Authorization"); got != "Bearer ete-key" { + t.Errorf("Authorization = %q, want Bearer ete-key", got) + } +} + +// A session is pinned by where its first request went, not by the choice. Under +// observe that request stays on the host the client named, so the session is pinned +// as not routed; turning enforce on later must not move it to a server it never +// used, which is the mid-conversation switch the pin exists to prevent. +func TestRouter_ASessionStartedUnderObserveStaysPutUnderEnforce(t *testing.T) { + store := newStore(t) + observe := build(t, routerConfig(`"claude-code": "glm"`), pipeline.WithPolicies(pipeline.ErrorPolicyObserve)) + first := request(store, eteHost, claudeUA, "s1") + run(t, observe, first) + assertRecord(t, first, pipeline.ActionObserve, "would_route", map[string]string{"server": "glm", "pin": pinNew}) + if pin, ok := pinOf(t, store, "s1"); !ok || pin != "" { + t.Fatalf("pin = %q, %v; want the session pinned as not routed (\"\"), since it went nowhere else", pin, ok) + } + + enforce := build(t, routerConfig(`"claude-code": "glm"`)) + next := request(store, eteHost, claudeUA, "s1") + run(t, enforce, next) + assertUntouched(t, next, eteHost) + assertRecord(t, next, pipeline.ActionSkip, "not_routed", map[string]string{"pin": pinExisting}) +} + +// undeclared is the router without WritesDestination, so the pipeline refuses its +// Redirect: the one way to make a redirect fail from a test. +type undeclared struct{ *Router } + +func (u undeclared) Capabilities() pipeline.PluginCapabilities { + caps := u.Router.Capabilities() + caps.WritesDestination = false + return caps +} + +// A first request whose redirect failed went nowhere, so it pins nothing: the next +// request decides again rather than inheriting a server the session never reached. +func TestRouter_AFailedRedirectPinsNothing(t *testing.T) { + store := newStore(t) + r := New() + if err := r.Configure(json.RawMessage(routerConfig(`"claude-code": "glm"`))); err != nil { + t.Fatalf("Configure: %v", err) + } + failing, err := pipeline.New([]pipeline.Plugin{undeclared{r}}) + if err != nil { + t.Fatalf("pipeline.New: %v", err) + } + pctx := request(store, eteHost, claudeUA, "s1") + a := run(t, failing, pctx) + if a.Type != pipeline.Reject { + t.Fatalf("action = %+v, want Reject", a) + } + if status, _, _ := a.Violation.Render(); status != http.StatusServiceUnavailable { + t.Errorf("status = %d, want 503", status) + } + assertRecord(t, pctx, pipeline.ActionDeny, "redirect_failed", map[string]string{"server": "glm", "pin": pinNone}) + assertUntouched(t, pctx, eteHost) + if pin, ok := pinOf(t, store, "s1"); ok { + t.Fatalf("pin = %q after a failed redirect, want none", pin) + } + + next := request(store, eteHost, claudeUA, "s1") + run(t, build(t, routerConfig(`"claude-code": "glm"`)), next) + assertRouted(t, next, glmHost, "glm-key") + assertRecord(t, next, pipeline.ActionModify, "routed", map[string]string{"server": "glm", "pin": pinNew}) +} + +// The listener's synthetic sessions are shared buckets, not conversations: default +// collects traffic it cannot attribute, and pending: an agent's calls before +// its session is known. Pinning one would hold every later conversation filed there +// to the first one's server, so they route by the agent's current choice, unpinned. +func TestRouter_NeverPinsTheListenersSyntheticSessions(t *testing.T) { + for _, id := range []string{session.DefaultSessionID, session.PendingPrefix + "claude-code"} { + t.Run(id, func(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := request(store, eteHost, claudeUA, id) + run(t, p, pctx) + + assertRouted(t, pctx, glmHost, "glm-key") + assertRecord(t, pctx, pipeline.ActionModify, "routed", map[string]string{"server": "glm", "pin": pinNone}) + if pin, ok := pinOf(t, store, id); ok { + t.Errorf("pin = %q, want no pin for the synthetic session %q", pin, id) + } + }) + } +} + +// Under observe, Redirect returns nil and moves nothing. The server's key must not +// reach the host the client named. +func TestRouter_UnderObserveLeavesTheClientsKeyAlone(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "glm"`), pipeline.WithPolicies(pipeline.ErrorPolicyObserve)) + pctx := request(newStore(t), eteHost, claudeUA, "s1") + run(t, p, pctx) + + assertUntouched(t, pctx, eteHost) + assertRecord(t, pctx, pipeline.ActionObserve, "would_route", map[string]string{"server": "glm"}) + invs := pctx.Extensions.Invocations.Outbound + if len(invs) != 2 || invs[0].Reason != "redirected" || !invs[0].Shadow { + t.Errorf("invocations = %+v, want the framework's shadow redirect, then would_route", invs) + } +} + +func TestRouter_WithoutAStoreRoutesUnpinnedAndWarnsOnce(t *testing.T) { + var logs bytes.Buffer + prev := slog.Default() + slog.SetDefault(slog.New(slog.NewTextHandler(&logs, nil))) + t.Cleanup(func() { slog.SetDefault(prev) }) + + p := build(t, routerConfig(`"claude-code": "glm"`)) + for range 2 { + pctx := request(nil, eteHost, claudeUA, "s1") + run(t, p, pctx) + assertRouted(t, pctx, glmHost, "glm-key") + assertRecord(t, pctx, pipeline.ActionModify, "routed", map[string]string{"pin": pinNone}) + } + if n := strings.Count(logs.String(), "sessions are not pinned"); n != 1 { + t.Errorf("warned %d times, want once:\n%s", n, logs.String()) + } +} + +func TestConfigure_WrapsErrorsInThePluginsName(t *testing.T) { + err := New().Configure(json.RawMessage(`{}`)) + if err == nil || !strings.HasPrefix(err.Error(), "inference-router config: ") { + t.Fatalf("err = %v, want it prefixed with inference-router config:", err) + } +} + +func TestConfigure_WarnsOnPlainHTTPToAnotherMachine(t *testing.T) { + var logs bytes.Buffer + prev := slog.Default() + slog.SetDefault(slog.New(slog.NewTextHandler(&logs, nil))) + t.Cleanup(func() { slog.SetDefault(prev) }) + + err := New().Configure(json.RawMessage(`{"servers": { + "lan": {"url": "http://10.0.0.5:4000", "key": "k"}, + "local": {"url": "http://localhost:4000", "key": "k"}}}`)) + if err != nil { + t.Fatalf("Configure: %v", err) + } + out := logs.String() + if !strings.Contains(out, "server=lan") || !strings.Contains(out, "plain http") { + t.Errorf("no plaintext warning for lan:\n%s", out) + } + if strings.Contains(out, "server=local") { + t.Errorf("warned about a loopback server:\n%s", out) + } +} + +func TestCapabilities_DeclareARedirectAndNoBody(t *testing.T) { + caps := New().Capabilities() + if !caps.WritesDestination || caps.ReadsBody || caps.WritesRequestBody || caps.WritesResponseBody { + t.Errorf("caps = %+v, want WritesDestination alone", caps) + } + if n := len(caps.Description); n == 0 || n > 80 { + t.Errorf("description is %d chars, want 1-80", n) + } +} + +// routerconfig copies the no-User-Agent label rather than importing pipeline. +func TestNoAgentIsThePipelinesUnknownLabel(t *testing.T) { + if routerconfig.NoAgent != pipeline.UnknownClientLabel { + t.Errorf("routerconfig.NoAgent = %q, pipeline.UnknownClientLabel = %q", routerconfig.NoAgent, pipeline.UnknownClientLabel) + } +} + +// A listener that cannot honor a redirect refuses the plugin at build time. +func TestRouter_IsRefusedWhereTheListenerCannotRedirect(t *testing.T) { + _, err := plugins.BuildWithDeps(entries(routerConfig(``)), plugins.Deps{}) + if err == nil || !strings.Contains(err.Error(), "WritesDestination") { + t.Fatalf("err = %v, want a refusal naming WritesDestination", err) + } +} + +// sent is a request row the forward proxy recorded in a session: an outbound +// inference request from the agent with User-Agent ua, which went to host. +func sent(host, ua string) pipeline.SessionEvent { + return pipeline.SessionEvent{ + Direction: pipeline.Outbound, Phase: pipeline.SessionRequest, Host: host, + Inference: &pipeline.InferenceExtension{Model: "claude-opus-5-5"}, Client: pipeline.ParseUserAgent(ua), + } +} + +// withHistory gives pctx's session the events the store recorded before it. +func withHistory(pctx *pipeline.Context, events ...pipeline.SessionEvent) *pipeline.Context { + pctx.Session.Events = events + return pctx +} + +// The reviewer's probe. A Claude Code session already running on ete, but quiet +// while routing was first configured, has no pin: the router never saw it. Its next +// turn is not a new session's first request, and the store's history says so: it +// keeps going where it already went, with ete's key, rather than moving to glm in +// the middle of a conversation. +func TestRouter_ARunningSessionTheRouterHasNotSeenStaysWhereItWent(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := withHistory(request(store, eteHost, claudeUA, "running"), sent(eteHost, claudeUA)) + run(t, p, pctx) + + assertRouted(t, pctx, eteHost, "ete-key") + assertRedirectedTo(t, pctx, "https", eteHost) + assertRecord(t, pctx, pipeline.ActionModify, "routed", map[string]string{"server": "ete", "pin": pinNew}) + if v, _ := store.Get(pinPrefix + "running"); v != (pin{agent: "claude-code", server: "ete"}) { + t.Errorf("pin = %#v, want claude-code's session pinned to ete", v) + } + + // Pinned now: the next request needs no history to stay. + next := request(store, eteHost, claudeUA, "running") + run(t, p, next) + assertRouted(t, next, eteHost, "ete-key") + assertRecord(t, next, pipeline.ActionModify, "routed", map[string]string{"server": "ete", "pin": pinExisting}) +} + +// Only a request to a server's host says which server a session is on: the router +// never routes a request to any other host, so one there says nothing about where +// the session lives. OpenCode switches providers inside a session, so its first +// requests can go to Zen or another provider before it addresses a server; that +// session is new to the router and goes to its agent's server, with the server's +// key. Treating it as not routed kept the client's own key on the server for the +// session's whole life, a 401 on every request. +func TestRouter_AnEarlierRequestToAHostNoServerHasIsNoEvidence(t *testing.T) { + for _, tc := range []struct{ name, ua, host string }{ + {"opencode after another provider", opencodeUA, "opencode.ai"}, + {"claude-code after api.anthropic.com", claudeUA, "api.anthropic.com"}, + } { + t.Run(tc.name, func(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm", "opencode": "glm"`)) + pctx := withHistory(request(store, eteHost, tc.ua, "running"), sent(tc.host, tc.ua)) + run(t, p, pctx) + + assertRouted(t, pctx, glmHost, "glm-key") + assertRecord(t, pctx, pipeline.ActionModify, "routed", map[string]string{"server": "glm", "pin": pinNew}) + if pin, _ := pinOf(t, store, "running"); pin != "glm" { + t.Errorf("pin = %q, want glm", pin) + } + }) + } +} + +// Only the latest earlier request to a server's host decides, so a session is held +// to the server it went to last; a request to any other host in between is passed +// over. +func TestRouter_TheLatestEarlierRequestToAServerDecides(t *testing.T) { + for _, tc := range []struct { + name string + history []pipeline.SessionEvent + want string // the server's host + }{ + {"elsewhere, then ete", []pipeline.SessionEvent{sent("api.anthropic.com", claudeUA), sent(eteHost, claudeUA)}, eteHost}, + {"ete, then elsewhere", []pipeline.SessionEvent{sent(eteHost, claudeUA), sent("api.anthropic.com", claudeUA)}, eteHost}, + {"glm, then ete with case and its default port", []pipeline.SessionEvent{sent(glmHost, claudeUA), sent("ETE.example.com:443", claudeUA)}, eteHost}, + {"ete, then glm with its port", []pipeline.SessionEvent{sent(eteHost, claudeUA), sent("GLM.example.com:8443", claudeUA)}, glmHost}, + {"only elsewhere: new, the current server", []pipeline.SessionEvent{sent("api.anthropic.com", claudeUA)}, glmHost}, + } { + t.Run(tc.name, func(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := withHistory(request(newStore(t), eteHost, claudeUA, "running"), tc.history...) + run(t, p, pctx) + assertRedirectedTo(t, pctx, "https", tc.want) + }) + } +} + +// A row's Host is where the request went, which for a routed request is the +// server it was redirected to; RequestedHost is where the client asked to go. The +// session is on the server the bytes reached, so Host decides. +func TestRouter_TheServerARequestWentToDecidesNotTheOneItAskedFor(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "ete"`)) + routed := sent(glmHost, claudeUA) + routed.RequestedHost = eteHost + pctx := withHistory(request(store, eteHost, claudeUA, "running"), routed) + run(t, p, pctx) + + assertRouted(t, pctx, glmHost, "glm-key") + if pin, _ := pinOf(t, store, "running"); pin != "glm" { + t.Errorf("pin = %q, want glm, the server the earlier request reached", pin) + } +} + +// A view can be empty, or hold no inference request, for a session that exists, +// so only an earlier inference request this agent sent and the proxy sent on is +// evidence of a running conversation. Each row here is not, and the session is new: +// it goes to the agent's current server. +func TestRouter_ASessionWithNoEarlierInferenceRequestIsNew(t *testing.T) { + tunnel := pipeline.SessionEvent{Direction: pipeline.Outbound, Phase: pipeline.SessionRequest, Host: eteHost + ":443", + Tunnel: true, HTTPMethod: http.MethodConnect, Client: pipeline.ParseUserAgent(claudeUA)} + denied := sent(eteHost, claudeUA) + denied.Phase = pipeline.SessionDenied + response := sent(eteHost, claudeUA) + response.Phase = pipeline.SessionResponse + inbound := sent(eteHost, claudeUA) + inbound.Direction = pipeline.Inbound + unparsed := sent(eteHost, claudeUA) + unparsed.Inference = nil + for _, tc := range []struct { + name string + history []pipeline.SessionEvent + }{ + {"no events", nil}, + {"only the bridged CONNECT's tunnel row", []pipeline.SessionEvent{tunnel}}, + {"a denied request", []pipeline.SessionEvent{denied}}, + {"a response row", []pipeline.SessionEvent{response}}, + {"an inbound request", []pipeline.SessionEvent{inbound}}, + {"a request no parser read as inference", []pipeline.SessionEvent{unparsed}}, + {"another agent's inference request", []pipeline.SessionEvent{sent(eteHost, opencodeUA)}}, + } { + t.Run(tc.name, func(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := withHistory(request(newStore(t), eteHost, claudeUA, "s1"), tc.history...) + run(t, p, pctx) + assertRouted(t, pctx, glmHost, "glm-key") + assertRecord(t, pctx, pipeline.ActionModify, "routed", map[string]string{"server": "glm", "pin": pinNew}) + }) + } +} + +// The history rule is for an agent that is routed. An unlisted agent is left +// alone whatever its session did. +func TestRouter_AnUnlistedAgentsHistoryRoutesNothing(t *testing.T) { + p := build(t, routerConfig(`"claude-code": "glm"`)) + pctx := withHistory(request(newStore(t), eteHost, opencodeUA, "s1"), sent(glmHost, opencodeUA)) + run(t, p, pctx) + assertUntouched(t, pctx, eteHost) + assertRecord(t, pctx, pipeline.ActionSkip, "not_routed", nil) +} + +// A session id is the listener's answer, and another agent's request can be filed +// under one: by process attribution (a command an agent runs is filed under its +// session), the ActiveSession fallback with client affinity off, or a header id +// two clients share. A pin is that agent's and holds for it alone. Another agent's +// request is decided as if the session were unpinned — by its own choice, or left +// alone when it is not listed — and never takes the session's server or key, nor +// disturbs its pin. +func TestRouter_APinHoldsOnlyForTheAgentItWasMadeFor(t *testing.T) { + store := newStore(t) + p := build(t, routerConfig(`"claude-code": "glm"`)) + run(t, p, request(store, eteHost, claudeUA, "s1")) + history := sent(glmHost, claudeUA) // where claude-code's request went + + unlisted := withHistory(request(store, eteHost, opencodeUA, "s1"), history) + run(t, p, unlisted) + assertUntouched(t, unlisted, eteHost) + assertRecord(t, unlisted, pipeline.ActionSkip, "not_routed", map[string]string{"pin": pinNone}) + + both := build(t, routerConfig(`"claude-code": "glm", "opencode": "ete"`)) + listed := withHistory(request(store, eteHost, opencodeUA, "s1"), history) + run(t, both, listed) + assertRouted(t, listed, eteHost, "ete-key") + assertRecord(t, listed, pipeline.ActionModify, "routed", map[string]string{"server": "ete", "pin": pinNone}) + + // A request with no User-Agent is no agent's, and no pin is its. + anonymous := withHistory(request(store, eteHost, "", "s1"), history) + run(t, both, anonymous) + assertUntouched(t, anonymous, eteHost) + + if v, _ := store.Get(pinPrefix + "s1"); v != (pin{agent: "claude-code", server: "glm"}) { + t.Errorf("pin = %#v after other agents' requests, want claude-code's glm pin untouched", v) + } + owner := request(store, eteHost, claudeUA, "s1") + run(t, both, owner) + assertRouted(t, owner, glmHost, "glm-key") + assertRecord(t, owner, pipeline.ActionModify, "routed", map[string]string{"server": "glm", "pin": pinExisting}) +} diff --git a/core/plugins/inferencerouter/reload_test.go b/core/plugins/inferencerouter/reload_test.go new file mode 100644 index 000000000..d64e16977 --- /dev/null +++ b/core/plugins/inferencerouter/reload_test.go @@ -0,0 +1,53 @@ +package inferencerouter + +import ( + "encoding/json" + "testing" + + "github.com/rossoctl/cortex/core/config" + "github.com/rossoctl/cortex/core/pipeline" + "github.com/rossoctl/cortex/core/plugins" +) + +// entries is the outbound chain a config file with only the router would build. +func entries(routerConfig string) []config.PluginEntry { + return []config.PluginEntry{{Name: Name, Config: json.RawMessage(routerConfig)}} +} + +// reloadBuild builds the chain the way cmd/cortex does on startup and on every +// reload: through the registry, for a listener that honors redirects. +func reloadBuild(t *testing.T, routerConfig string) *pipeline.Pipeline { + t.Helper() + p, err := plugins.BuildWithDeps(entries(routerConfig), + plugins.Deps{Listener: pipeline.ListenerSupport{Listener: "forward proxy", Destination: true}}) + if err != nil { + t.Fatalf("BuildWithDeps: %v", err) + } + return p +} + +// TestPinsSurviveAReload is the claim switching rests on. The reloader builds a new +// pipeline from the edited file and swaps it in; the process store the listener +// holds is not rebuilt, so the new router instance reads the old one's pins. A +// running session keeps its server, routed or not, and only a new session sees the +// edit. +func TestPinsSurviveAReload(t *testing.T) { + store := newStore(t) + before := reloadBuild(t, routerConfig(`"claude-code": "glm"`)) + run(t, before, request(store, eteHost, claudeUA, "routed")) + run(t, before, request(store, eteHost, opencodeUA, "unrouted")) + + after := reloadBuild(t, routerConfig(`"claude-code": "ete", "opencode": "glm"`)) + + routed := request(store, eteHost, claudeUA, "routed") + run(t, after, routed) + assertRouted(t, routed, glmHost, "glm-key") + + unrouted := request(store, eteHost, opencodeUA, "unrouted") + run(t, after, unrouted) + assertUntouched(t, unrouted, eteHost) + + fresh := request(store, eteHost, opencodeUA, "fresh") + run(t, after, fresh) + assertRouted(t, fresh, glmHost, "glm-key") +} diff --git a/core/plugins/inferencerouter/routerconfig/config.go b/core/plugins/inferencerouter/routerconfig/config.go new file mode 100644 index 000000000..8892bc4c9 --- /dev/null +++ b/core/plugins/inferencerouter/routerconfig/config.go @@ -0,0 +1,249 @@ +// Package routerconfig is the inference-router plugin's configuration: its shape, +// its validation, and the host rules the plugin routes by. +// +// It is a package of its own, importing only the standard library, so that agentop +// can check a config before writing it without linking the plugin. Linking the +// plugin would pull the plugin registry, and SPIFFE and gRPC with it, into a +// terminal UI. One copy of these rules is what keeps `agentop server` from writing a +// config the proxy then refuses on reload. +package routerconfig + +import ( + "bytes" + "encoding/json" + "errors" + "fmt" + "maps" + "net" + "net/url" + "regexp" + "slices" + "strconv" + "strings" +) + +// NoAgent is the agent name of a request that carried no User-Agent. It is +// pipeline.UnknownClientLabel, copied rather than imported to keep this package +// free of the pipeline; a test in the plugin package holds the two equal. +// +// It cannot be routed: requests under it come from no one agent, so routing it +// would send every unlabelled request — a health probe, a curl — to a server. +const NoAgent = "unknown" + +// Config is the plugin's config: block. +type Config struct { + Servers map[string]Server `json:"servers" required:"true" description:"Inference servers, by name."` + Agents map[string]string `json:"agents" description:"Agents to route, by agent name, to a server name. Unlisted agents are not routed."` +} + +// Server is one inference server. +type Server struct { + URL string `json:"url" required:"true" description:"scheme://host[:port] of the server; no path."` + Key string `json:"key" required:"true" description:"API key sent to this server in place of the client's."` + Opus string `json:"opus" description:"This server's model for Claude Code's opus requests (PR 4)."` + Sonnet string `json:"sonnet" description:"This server's model for Claude Code's sonnet requests (PR 4)."` + Haiku string `json:"haiku" description:"This server's model for Claude Code's haiku requests (PR 4)."` +} + +// Decode reads raw, the entry's config: block as JSON, refusing unknown fields, and +// validates it. Unknown fields are refused because the likeliest one is a guess at +// a feature this design leaves out on purpose — a `default:` server — and a +// silently ignored guess would route nothing while looking configured. +func Decode(raw json.RawMessage) (Config, error) { + var c Config + if len(raw) > 0 { + dec := json.NewDecoder(bytes.NewReader(raw)) + dec.DisallowUnknownFields() + if err := dec.Decode(&c); err != nil { + return Config{}, err + } + } + c.applyDefaults() + if err := c.Validate(); err != nil { + return Config{}, err + } + return c, nil +} + +func (c *Config) applyDefaults() { + if c.Agents == nil { + c.Agents = map[string]string{} + } +} + +// Validate checks every rule the plugin routes by. Servers and agents are checked +// in name order, so the same config always reports the same first error. +func (c Config) Validate() error { + if len(c.Servers) == 0 { + return errors.New("servers: at least one server is required") + } + byHost := map[string]string{} + for _, name := range slices.Sorted(maps.Keys(c.Servers)) { + s := c.Servers[name] + if err := CheckName(name); err != nil { + return fmt.Errorf("servers: %w", err) + } + ep, err := ParseURL(s.URL) + if err != nil { + return fmt.Errorf("servers.%s.url: %w", name, err) + } + // Port-blind on purpose. agentop names a session's server from the host its + // requests went to, and two servers on one host would make that ambiguous. + if other, ok := byHost[ep.Hostname]; ok { + return fmt.Errorf("servers %q and %q are both on %s: each server needs a host of its own, "+ + "because a session's server is named from the host its requests went to", other, name, ep.Hostname) + } + byHost[ep.Hostname] = name + if err := CheckKey(s.Key); err != nil { + return fmt.Errorf("servers.%s.key: %w", name, err) + } + if s.Opus != "" || s.Sonnet != "" || s.Haiku != "" { + return fmt.Errorf("servers.%s: opus, sonnet and haiku are not accepted yet: "+ + "model mapping needs chained body writers (PR 4)", name) + } + } + for _, agent := range slices.Sorted(maps.Keys(c.Agents)) { + if err := CheckAgent(agent); err != nil { + return fmt.Errorf("agents: %w", err) + } + server := c.Agents[agent] + if _, ok := c.Servers[server]; !ok { + return fmt.Errorf("agents.%s: %q is not a server listed under servers", agent, server) + } + } + return nil +} + +var namePattern = regexp.MustCompile(`^[a-z0-9._-]+$`) + +// CheckName reports whether name can name a server. +func CheckName(name string) error { + if !namePattern.MatchString(name) { + return fmt.Errorf("%q is not a server name: use lowercase letters, digits, '.', '_' and '-'", name) + } + return nil +} + +// CheckAgent reports whether name can be routed. Agent names are the ones +// pipeline.AgentName produces and agentop shows, such as claude-code and opencode. +func CheckAgent(name string) error { + if !namePattern.MatchString(name) { + return fmt.Errorf("%q is not an agent name: agent names are lowercase, as agentop shows them (claude-code, opencode)", name) + } + if name == NoAgent { + return fmt.Errorf("%q cannot be routed: it is the name for requests that carry no User-Agent, which belong to no one agent", name) + } + return nil +} + +// CheckKey reports whether key can be sent as a header value. The error never +// repeats the key. +func CheckKey(key string) error { + if key == "" { + return errors.New("the key is empty") + } + for i := 0; i < len(key); i++ { + if key[i] <= ' ' || key[i] > '~' { + return errors.New("the key must be printable ASCII with no spaces: it is sent as a header value") + } + } + return nil +} + +// Endpoint is a server URL reduced to what routing compares. +type Endpoint struct { + // Scheme is "http" or "https". + Scheme string + // Host is the lowercased host, with the port only when it is not the scheme's + // default. It is the redirect target's host. + Host string + // Hostname is the lowercased host without any port: what a request is matched + // to a server by, and what makes two servers the same host. + Hostname string +} + +// URL is the endpoint as scheme://host, the form agentop writes. +func (e Endpoint) URL() string { return e.Scheme + "://" + e.Host } + +// PlaintextRemote reports an http endpoint on another machine. A request routed +// there leaves the proxy decrypted, with its key and its prompt, across the network. +func (e Endpoint) PlaintextRemote() bool { + if e.Scheme != "http" || e.Hostname == "localhost" { + return false + } + ip := net.ParseIP(e.Hostname) + return ip == nil || !ip.IsLoopback() +} + +// ParseURL validates a server URL and returns its Endpoint. It accepts what +// pipeline.Context.Redirect accepts — http or https, a host, no user info, no path +// beyond "/", no query or fragment — and two things more: a port must be a number +// from 1 to 65535, and the scheme's default port is dropped, so a server at +// https://x:443 is the host a request for x names, and the redirect a routed request +// for x still gets records no requested host. +// +// Its errors are fixed text that names the problem and quotes no part of raw, not +// even redacted. They reach the logs and the unauthenticated /reload/status, and a +// key pasted into a mistyped URL can land anywhere in it: user info, query, +// fragment, a path segment, or the path or port url.Parse files it under when the +// "//" is wrong. Redacting shape by shape kept missing one. +func ParseURL(raw string) (Endpoint, error) { + u, err := url.Parse(raw) + if err != nil { + // Not even url.Parse's inner error, which quotes a bad port or escape. + return Endpoint{}, errors.New("not a valid URL") + } + switch { + case u.Scheme != "http" && u.Scheme != "https": + return Endpoint{}, errors.New("the URL's scheme must be http or https") + case u.Opaque != "" || u.Hostname() == "": + return Endpoint{}, errors.New("the URL has no host") + case u.User != nil: + return Endpoint{}, errors.New("the URL carries user info; the key is given separately, never in the URL") + case u.Path != "" && u.Path != "/": + return Endpoint{}, errors.New("the URL has a path; give scheme://host[:port] only, since a routed request keeps its own path") + case u.RawQuery != "" || u.ForceQuery || u.Fragment != "": + return Endpoint{}, errors.New("the URL has a query or fragment") + } + port := "" + if p := u.Port(); p != "" { + n, err := strconv.Atoi(p) + if err != nil || n < 1 || n > 65535 { + return Endpoint{}, errors.New("the URL's port must be a number from 1 to 65535") + } + port = strconv.Itoa(n) + } + hostname := strings.ToLower(u.Hostname()) + return Endpoint{Scheme: u.Scheme, Host: joinHost(hostname, withoutDefault(u.Scheme, port)), Hostname: hostname}, nil +} + +// Hostname is a request's host[:port], lowercased and without the port: what the +// request is matched to a server by. +func Hostname(hostport string) string { + h, _ := splitHost(hostport) + return h +} + +func splitHost(hostport string) (host, port string) { + if h, p, err := net.SplitHostPort(hostport); err == nil { + return strings.ToLower(h), p + } + return strings.ToLower(strings.TrimSuffix(strings.TrimPrefix(hostport, "["), "]")), "" +} + +func withoutDefault(scheme, port string) string { + if (scheme == "https" && port == "443") || (scheme == "http" && port == "80") { + return "" + } + return port +} + +func joinHost(host, port string) string { + if port != "" { + return net.JoinHostPort(host, port) + } + if strings.Contains(host, ":") { + return "[" + host + "]" + } + return host +} diff --git a/core/plugins/inferencerouter/routerconfig/config_test.go b/core/plugins/inferencerouter/routerconfig/config_test.go new file mode 100644 index 000000000..21fab38ea --- /dev/null +++ b/core/plugins/inferencerouter/routerconfig/config_test.go @@ -0,0 +1,206 @@ +package routerconfig + +import ( + "encoding/json" + "strings" + "testing" +) + +const twoServers = `{ + "servers": { + "ete": {"url": "https://ete.example.com", "key": "sk-ete"}, + "glm": {"url": "https://glm.example.com:8443", "key": "sk-glm"} + }, + "agents": {"claude-code": "glm"} +}` + +func TestDecode_AcceptsServersAndAgents(t *testing.T) { + c, err := Decode(json.RawMessage(twoServers)) + if err != nil { + t.Fatalf("Decode: %v", err) + } + if len(c.Servers) != 2 || c.Agents["claude-code"] != "glm" { + t.Errorf("decoded %+v", c) + } +} + +func TestDecode_NoAgentsIsAnEmptyMap(t *testing.T) { + c, err := Decode(json.RawMessage(`{"servers": {"ete": {"url": "https://ete.example.com", "key": "k"}}}`)) + if err != nil { + t.Fatalf("Decode: %v", err) + } + if c.Agents == nil { + t.Error("Agents is nil; an absent agents block routes nothing, as an empty one does") + } +} + +// A default server is the feature this design leaves out on purpose; a config that +// asks for one must be told, not silently route nothing. +func TestDecode_RefusesUnknownFields(t *testing.T) { + _, err := Decode(json.RawMessage(`{"servers": {"ete": {"url": "https://ete.example.com", "key": "k"}}, "default": "ete"}`)) + if err == nil || !strings.Contains(err.Error(), `"default"`) { + t.Fatalf("err = %v, want an unknown-field error naming default", err) + } +} + +func TestValidate_RefusesEachBrokenRule(t *testing.T) { + for _, tc := range []struct { + name, config, want string + }{ + {"no servers", `{"servers": {}}`, "at least one server"}, + {"absent servers", `{}`, "at least one server"}, + {"uppercase name", `{"servers": {"ETE": {"url": "https://e.example", "key": "k"}}}`, `"ETE" is not a server name`}, + {"name with a space", `{"servers": {"e te": {"url": "https://e.example", "key": "k"}}}`, `"e te" is not a server name`}, + {"ftp", `{"servers": {"e": {"url": "ftp://e.example", "key": "k"}}}`, "scheme must be http or https"}, + {"no scheme", `{"servers": {"e": {"url": "e.example", "key": "k"}}}`, "scheme must be http or https"}, + {"no host", `{"servers": {"e": {"url": "https://", "key": "k"}}}`, "has no host"}, + {"path", `{"servers": {"e": {"url": "https://e.example/v1", "key": "k"}}}`, "has a path"}, + {"query", `{"servers": {"e": {"url": "https://e.example?x=1", "key": "k"}}}`, "query or fragment"}, + {"fragment", `{"servers": {"e": {"url": "https://e.example#x", "key": "k"}}}`, "query or fragment"}, + {"user info", `{"servers": {"e": {"url": "https://u:p@e.example", "key": "k"}}}`, "user info"}, + {"non-numeric port", `{"servers": {"e": {"url": "https://e.example:abc", "key": "k"}}}`, "servers.e.url: not a valid URL"}, + {"port zero", `{"servers": {"e": {"url": "https://e.example:0", "key": "k"}}}`, "port must be a number from 1 to 65535"}, + {"port too high", `{"servers": {"e": {"url": "https://e.example:65536", "key": "k"}}}`, "port must be a number from 1 to 65535"}, + {"shared host, port and case ignored", `{"servers": { + "a": {"url": "https://gw.example", "key": "k"}, + "b": {"url": "http://GW.example:4000", "key": "k"}}}`, `"a" and "b" are both on gw.example`}, + {"empty key", `{"servers": {"e": {"url": "https://e.example", "key": ""}}}`, "the key is empty"}, + {"key with a space", `{"servers": {"e": {"url": "https://e.example", "key": "sk 1"}}}`, "printable ASCII with no spaces"}, + {"agent naming no server", `{"servers": {"e": {"url": "https://e.example", "key": "k"}}, "agents": {"claude-code": "glm"}}`, `"glm" is not a server listed under servers`}, + {"agent name as a label", `{"servers": {"e": {"url": "https://e.example", "key": "k"}}, "agents": {"Claude Code": "e"}}`, `"Claude Code" is not an agent name`}, + {"the no-User-Agent bucket", `{"servers": {"e": {"url": "https://e.example", "key": "k"}}, "agents": {"unknown": "e"}}`, `"unknown" cannot be routed`}, + {"opus", `{"servers": {"e": {"url": "https://e.example", "key": "k", "opus": "glm-5.3"}}}`, "model mapping needs chained body writers (PR 4)"}, + {"haiku", `{"servers": {"e": {"url": "https://e.example", "key": "k", "haiku": "glm-5.3"}}}`, "model mapping needs chained body writers (PR 4)"}, + } { + t.Run(tc.name, func(t *testing.T) { + _, err := Decode(json.RawMessage(tc.config)) + if err == nil { + t.Fatal("Decode accepted it") + } + if !strings.Contains(err.Error(), tc.want) { + t.Errorf("err = %q, want it to contain %q", err, tc.want) + } + }) + } +} + +// The key is a secret; an error about it must not print it. +func TestCheckKey_NeverRepeatsTheKey(t *testing.T) { + err := CheckKey("sk-secret value") + if err == nil || strings.Contains(err.Error(), "sk-secret") { + t.Errorf("err = %v; want an error that does not contain the key", err) + } +} + +func TestParseURL_Normalises(t *testing.T) { + for _, tc := range []struct { + raw, url, host, hostname string + }{ + {"https://ete.example.com", "https://ete.example.com", "ete.example.com", "ete.example.com"}, + {"https://ETE.Example.com:443/", "https://ete.example.com", "ete.example.com", "ete.example.com"}, + {"http://localhost:80", "http://localhost", "localhost", "localhost"}, + {"https://glm.example.com:8443", "https://glm.example.com:8443", "glm.example.com:8443", "glm.example.com"}, + {"https://x.example:0443", "https://x.example", "x.example", "x.example"}, + {"http://[::1]:4000", "http://[::1]:4000", "[::1]:4000", "::1"}, + {"https://[::1]:443", "https://[::1]", "[::1]", "::1"}, + } { + t.Run(tc.raw, func(t *testing.T) { + ep, err := ParseURL(tc.raw) + if err != nil { + t.Fatalf("ParseURL: %v", err) + } + if ep.URL() != tc.url || ep.Host != tc.host || ep.Hostname != tc.hostname { + t.Errorf("got URL %q Host %q Hostname %q, want %q %q %q", ep.URL(), ep.Host, ep.Hostname, tc.url, tc.host, tc.hostname) + } + }) + } +} + +func TestHostname_StripsThePortAndCase(t *testing.T) { + for in, want := range map[string]string{ + "ete.example.com": "ete.example.com", + "ETE.example.com:8443": "ete.example.com", + "[::1]:4000": "::1", + "[::1]": "::1", + } { + if got := Hostname(in); got != want { + t.Errorf("Hostname(%q) = %q, want %q", in, got, want) + } + } +} + +func TestPlaintextRemote(t *testing.T) { + for raw, want := range map[string]bool{ + "https://gw.example": false, + "http://localhost:4000": false, + "http://127.0.0.1:4000": false, + "http://[::1]:4000": false, + "http://10.0.0.5:4000": true, + "http://gateway.lan": true, + "http://LOCALHOST:4000/": false, + } { + ep, err := ParseURL(raw) + if err != nil { + t.Fatalf("ParseURL(%q): %v", raw, err) + } + if got := ep.PlaintextRemote(); got != want { + t.Errorf("PlaintextRemote(%q) = %v, want %v", raw, got, want) + } + } +} + +// No ParseURL error quotes any part of the URL. Errors reach the logs and the +// unauthenticated /reload/status, and a key can land anywhere in a mistyped URL — +// user info, query, fragment, a path segment, or wherever url.Parse files it when +// the "//" is wrong — so redacting shape by shape kept missing one. Each error is +// fixed text naming the problem; this table is every input that once leaked. +func TestParseURL_QuotesNoPartOfTheURL(t *testing.T) { + const ( + invalid = "not a valid URL" + scheme = "the URL's scheme must be http or https" + noHost = "the URL has no host" + user = "the URL carries user info" + path = "the URL has a path" + query = "the URL has a query or fragment" + ) + for _, tc := range []struct{ url, want string }{ + // User info, query and fragment. + {"https://sk-SECRET@h.example.com", user}, + {"https://u:sk-SECRET@h.example.com", user}, + {"ftp://u:sk-SECRET@h.example.com", scheme}, + {"https://u:sk-SECRET@", noHost}, + {"https:u:sk-SECRET@h.example.com", noHost}, + {"https://h.example.com/?key=sk-SECRET", query}, + {"https://h.example.com/#sk-SECRET", query}, + {"https://h.example.com/?a=1#sk-SECRET", query}, + {"ftp://h.example.com/?key=sk-SECRET", scheme}, + // A mistyped "//" files the key under the path. + {"https:/u:sk-SECRET@h.example.com", noHost}, + {"https:///u:sk-SECRET@h.example.com", noHost}, + {"https//u:sk-SECRET@h.example.com", scheme}, + {"sk-SECRET@h.example.com", scheme}, + // url.Parse's own errors quote the bad port, or the escape. + {"https://u:sk-SECRET/x@h.example.com", invalid}, + {"https://u:sk-SECRET#x@h.example.com", invalid}, + {"https://u:sk-SECRET?x@h.example.com", invalid}, + {"https://h.example.com:sk-SECRET", invalid}, + {"https://[::1]:sk-SECRET", invalid}, + {"https://u:sk-SECRET@h.example.com:abc", invalid}, + {"https://u:sk-SECRET%ZZ@h.example.com", invalid}, + // A key as a path segment. + {"https://h.example.com/sk-SECRET", path}, + } { + _, err := ParseURL(tc.url) + if err == nil { + t.Errorf("ParseURL(%q) accepted it", tc.url) + continue + } + msg := err.Error() + if strings.Contains(msg, "sk-SECRET") || strings.Contains(msg, "u:") || strings.Contains(msg, "example.com") || strings.Contains(msg, "::1") { + t.Errorf("ParseURL(%q) error quotes the URL: %q", tc.url, msg) + } + if !strings.Contains(msg, tc.want) { + t.Errorf("ParseURL(%q) error = %q, want %q", tc.url, msg, tc.want) + } + } +} diff --git a/core/reloader/reloader.go b/core/reloader/reloader.go index 86c609647..b15739aa2 100644 --- a/core/reloader/reloader.go +++ b/core/reloader/reloader.go @@ -422,6 +422,15 @@ func validateReloadable(active, next *config.Config) error { if !reflect.DeepEqual(active.Session, next.Session) { diffs = append(diffs, "session.*") } + // mtls.* also needs a restart: the TLS dialer is built once in main and handed to + // the forward proxy listener. On an mTLS proxy, a reload that removes the mtls block + // while adding a WritesDestination plugin would let that plugin through, violating + // the TLS-in-TLS constraint forwardproxy.Support is built to enforce. Like cost_ledger + // and session, mtls.* is unreloadable in any direction (present↔absent or field change). + // Pointer-to-struct, so DeepEqual covers nil↔present as well as field drift. + if !reflect.DeepEqual(active.MTLS, next.MTLS) { + diffs = append(diffs, "mtls.*") + } if len(diffs) > 0 { return fmt.Errorf("unreloadable field changed, pod restart required: %v", diffs) } diff --git a/core/reloader/reloader_test.go b/core/reloader/reloader_test.go index e438027db..6b7465cc0 100644 --- a/core/reloader/reloader_test.go +++ b/core/reloader/reloader_test.go @@ -1,6 +1,7 @@ package reloader import ( + "bytes" "context" "errors" "os" @@ -648,6 +649,45 @@ func TestReloader_AnAcceptedReloadCommitsOnce(t *testing.T) { } } +// mtls needs a restart: the TLS dialer is constructed once in main and handed to the +// forward proxy listener, and nothing re-reads it. More critically, a reload that removes +// the mtls block while adding a WritesDestination plugin would let that plugin through on +// an mTLS proxy that was built with the opposite constraint — so validateReloadable blocks +// any change to mtls.* (present↔absent or field change), exactly like cost_ledger and session. +func TestValidateReloadable_RefusesMTLSEdit(t *testing.T) { + // Test 1: adding an mtls block to a config that had none + active := &config.Config{MTLS: nil} + next := &config.Config{MTLS: &config.MTLSConfig{Mode: "strict"}} + if err := validateReloadable(active, next); err == nil { + t.Fatal("adding an mtls block was accepted") + } else if !bytes.Contains([]byte(err.Error()), []byte("mtls.*")) { + t.Errorf("error doesn't mention mtls.*: %v", err) + } + + // Test 2: removing an mtls block from a config that had one + active = &config.Config{MTLS: &config.MTLSConfig{Mode: "strict"}} + next = &config.Config{MTLS: nil} + if err := validateReloadable(active, next); err == nil { + t.Fatal("removing an mtls block was accepted") + } else if !bytes.Contains([]byte(err.Error()), []byte("mtls.*")) { + t.Errorf("error doesn't mention mtls.*: %v", err) + } + + // Test 3: unchanged mtls block should be accepted + active = &config.Config{MTLS: &config.MTLSConfig{Mode: "strict"}} + next = &config.Config{MTLS: &config.MTLSConfig{Mode: "strict"}} + if err := validateReloadable(active, next); err != nil { + t.Errorf("unchanged mtls block was refused: %v", err) + } + + // Test 4: nil mtls in both should be accepted + active = &config.Config{MTLS: nil} + next = &config.Config{MTLS: nil} + if err := validateReloadable(active, next); err != nil { + t.Errorf("nil mtls in both was refused: %v", err) + } +} + // session.archive needs a restart like the rest of `session`: the archive is constructed once // in main and registered on the store, and nothing re-reads it. validateReloadable already // refuses any change under Session; this pins that the archive's block is inside that net. diff --git a/core/sessionapi/summary_test.go b/core/sessionapi/summary_test.go index 9485742cd..932841737 100644 --- a/core/sessionapi/summary_test.go +++ b/core/sessionapi/summary_test.go @@ -72,6 +72,9 @@ func fullEvent() pipeline.SessionEvent { Result: map[string]any{"big": strings.Repeat("r", 1024)}, Err: &pipeline.MCPError{Code: -1, Message: "boom"}, }, + // Set only when a plugin redirected the request; differs from Host on purpose, + // so an assertion cannot pass by reading the wrong one. + RequestedHost: "requested.example.com", } } @@ -121,6 +124,13 @@ func TestSummarizeEvent_DropsPayloadsKeepsTimelineFields(t *testing.T) { if got.Seq != full.Seq || got.At != full.At || got.Host != full.Host { t.Error("identity/time/host fields did not survive") } + // requestedHost is kept although only the detail pane reads it: agentop draws its + // redirected: line from the projected event before the full one arrives, and never + // fetches the full one for an event with no protocol extension. + if got.RequestedHost != full.RequestedHost { + t.Errorf("requestedHost = %q, want %q — the detail pane's redirected: line reads it", + got.RequestedHost, full.RequestedHost) + } if got.Direction != full.Direction || got.Phase != full.Phase { t.Error("direction/phase did not survive") } @@ -305,9 +315,10 @@ func TestSummarizeEvent_ShapeIsGuarded(t *testing.T) { typ reflect.Type want int }{ - // 24 since BytesUp/BytesDown, which are TIMELINE data: agentop's BYTES column - // renders them. Scalars, so the struct copy keeps them; asserted above. - {"SessionEvent", reflect.TypeOf(pipeline.SessionEvent{}), 24}, + // 25 since RequestedHost, which is kept: agentop's detail pane draws its + // redirected: line from the projected event before the full one arrives. A + // scalar, so the struct copy keeps it; asserted above. + {"SessionEvent", reflect.TypeOf(pipeline.SessionEvent{}), 25}, // 25 since AgentRole, which is TIMELINE data: agentop's CONTEXT gauge reads it on every // row the timeline serves. It needs no assertion of its own in the projection test // beyond the equality one there — a scalar survives the struct copy, unlike the two diff --git a/docs/agents/claude-code.md b/docs/agents/claude-code.md index 6e475e1eb..a042ddac6 100644 --- a/docs/agents/claude-code.md +++ b/docs/agents/claude-code.md @@ -121,6 +121,83 @@ proposes the tools you have not called, and writes them to the `tool-prune` plug list. Claude Code is the only agent it can build a list for. See [Cut token cost](../laptop-token-savings.md). +## Switching inference servers + +With more than one LiteLLM server, `agentop server` chooses which one Claude +Code's **new** sessions use, with one Claude Code configuration and without +restarting anything. A running conversation stays on the server it started on, +with the exceptions below. +See [Choosing an inference server](../../cmd/agentop/README.md#choosing-an-inference-server-agentop-server). + +That one configuration is: + +- **`ANTHROPIC_BASE_URL` at one of the configured servers.** Cortex routes only + requests addressed to a configured server, so Claude Code pointed anywhere else + is not routed. An `https` base URL is routed only when the TLS bridge decrypts + it, which needs its port in `tls_bridge.ports`, 443 and 8443 unless set. On + another port, such as `:4000`, the request stays an opaque `CONNECT` tunnel and + is not routed, although `agentop server`'s check, which matches by host alone, + shows ✓. + An `http` base URL reaches the router only if Claude Code sends it through the + proxy, and `agentop configure claude-code enable` sets `HTTPS_PROXY` only. The + server named must be up even for sessions routed elsewhere: the proxy dials the + `CONNECT` target before it sees the request inside. And a routed request keeps + its path, so a base URL with a path needs that same path on every server. +- **No model settings**, or only Claude's own names in them: the top-level + `model` setting and the variables `ANTHROPIC_MODEL`, + `ANTHROPIC_DEFAULT_OPUS_MODEL`, `ANTHROPIC_DEFAULT_SONNET_MODEL`, + `ANTHROPIC_DEFAULT_HAIKU_MODEL`, `ANTHROPIC_DEFAULT_FABLE_MODEL`, + `ANTHROPIC_SMALL_FAST_MODEL` and `CLAUDE_CODE_SUBAGENT_MODEL`. Claude's own names + are its aliases — `default`, `best`, `opus`, `opusplan`, `sonnet`, `haiku` and + `fable`, with or without a suffix such as `[1m]` — and any model id with `claude` + in it. Claude Code then asks for Claude's model names whichever server it talks + to — the conversation, auto mode's classifier and the background calls each + under its own family's name. A configuration that names one server model for + every family leaves nothing to tell them apart, and Claude Code shapes its + requests for the model it believes it is using. + +`agentop server` checks both. For now a server must serve Claude's model names, as +a LiteLLM gateway in front of Claude does; mapping them to another server's models +is not built yet. + +The key is the server's: on a routed request the router puts the configured +server's key in the header Claude Code sent its own in, `X-Api-Key` for +`ANTHROPIC_API_KEY` and `Authorization` for `ANTHROPIC_AUTH_TOKEN`, so either can +stay as it is. Those two headers are all it replaces: a credential in any other +header, such as one set through `ANTHROPIC_CUSTOM_HEADERS`, reaches the server +unchanged. + +Claude Code keeps naming the model it asked for, so agentop is where the server +shows: the events table's host is where each request went, and the detail pane's +`redirected:` line names the host Claude Code asked for when it was another. + +A session is pinned on the first request the router sees from it, so switch +**before** `/clear` or a new `claude`, not after. A conversation already running +when you first route Claude Code keeps the server its requests were going to, even +if it sent nothing while you ran `agentop server`: Cortex reads which server its +last request reached from the session's history, and from then on sends it there +with that server's key. That history is in the proxy's memory, which keeps the most +recently used sessions (100 by default), so a conversation quiet long enough to be +dropped before you route Claude Code is taken for a new session. A proxy restart +forgets the pins and that history, so a running conversation's next request after a +restart is taken for a new session's and goes to Claude Code's current server; see +[Known issues](#known-issues). A conversation +carried to a new session id is pinned as new, to its agent's current server, which +happens in three ways: Claude Code's continued-in hand-off copies a live +conversation to a new id; `claude daemon` starts background jobs with +`--session-id --fork-session --resume `; and `/clear` with background +subagents running moves their traffic to the new id. + +Nothing fails over. When a session's server is down its requests fail, with a 502 +from the proxy when the server cannot be reached; route the agent to another +server, and start a new session there. + +**`/model` may list a server's own names.** If Claude Code asks the gateway for its +model list, that request is routed like the rest, so the picker can show, say, +`glm-5.3`. Routing is unaffected, since Claude Code keeps sending its own names, +but choosing such a name there breaks the rule above. If Claude Code saves that +choice as `model` in `~/.claude/settings.json`, `agentop server` reports it. + ## Verified depth Tested live on 2026-10-05 with Claude Code 2.1.286 on macOS 26.6.2 (arm64), Cortex built @@ -141,7 +218,9 @@ Sonnet 5 and Claude Haiku 4.5. This was exercised: Not exercised: Linux; `install.sh --claude-code`; Claude Code talking to `api.anthropic.com` directly rather than through a gateway; Amazon Bedrock and Google -Vertex AI. +Vertex AI; routing through `inference-router` with a live Claude Code, which so far is +covered by unit and listener tests and by scratch-`HOME` runs of `agentop server` +against a fake stats server. ## Known issues @@ -154,6 +233,14 @@ Vertex AI. - **`disable` discards hand-edits.** If you change one of the seven keys by hand after `enable`, `disable` still puts back the value from before `enable`, without a warning ([#1289](https://github.com/rossoctl/cortex/issues/1289)). +- **A proxy restart can move a running conversation to another server.** Cortex + keeps a conversation on its server by a pin and by the session's history, and + both live in the proxy's memory; the session archive on disk is not read for it. + After a restart, a conversation's next request looks like a new session's and + goes to the server Claude Code is routed to now, so one started on a different + server moves, mid-conversation. To keep it where it was, route Claude Code back + to that server with `agentop server use --agent claude-code` before + continuing it, and switch again once it has sent a request, which pins it there. - **A continued conversation can show twice.** Claude Code sometimes moves a live conversation to a new session id. agentop then shows two rows with the same title, and the older row keeps the copied title. Cortex does not yet read Claude Code's record of diff --git a/docs/agents/opencode.md b/docs/agents/opencode.md index 5cd64a532..0efff0390 100644 --- a/docs/agents/opencode.md +++ b/docs/agents/opencode.md @@ -269,6 +269,21 @@ without being recorded once the service has named a session. Before that, they a recorded. See [How traffic is grouped into sessions](../laptop-service.md#how-traffic-is-grouped-into-sessions). +**Switching inference servers is OpenCode's own job.** `opencode.json` defines +providers, each with its own base URL and key, and OpenCode switches provider and +model inside a session. Cortex's `inference-router` therefore leaves OpenCode +alone unless `agentop server use --agent opencode` lists it. Listing it +sends OpenCode's new sessions that address one of the configured servers to the +chosen one, whatever provider was picked in OpenCode, so it works only between +servers that serve the model names OpenCode asks for. A session that used another +provider first is still new to the router when it addresses a server: only a +request to a configured server says which server a session is on. The reason to do it is to +keep keys out of `opencode.json`: on a routed request the router puts the +configured server's key in the `X-Api-Key` or `Authorization` header OpenCode +sent, and when OpenCode sent neither, it adds `Authorization: Bearer `. No +other header is replaced, so a credential OpenCode sends in any other header +reaches the server unchanged. + ## Verified depth Tested live with OpenCode 2.0.21 on macOS 26.6 (arm64) and Cortex built from the change @@ -315,6 +330,9 @@ Not tested live: [#941](https://github.com/rossoctl/cortex/issues/941) has one user's report of a LiteLLM run, started with `agentop exec` under its earlier name, in which token counts showed. +- **Routing through `inference-router`** with a live OpenCode. It is covered so far by + unit and listener tests and by scratch-`HOME` runs of `agentop server` against a fake + stats server. ## Known issues diff --git a/docs/architecture.md b/docs/architecture.md index 437e5ffaf..3f192239d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -458,7 +458,7 @@ declarative profile per shipped artifact and emits its tags: | Profile | Artifact | Plugins | |---------|----------|---------| | `local` | desktop `cortex` | the three parsers + `tool-prune` | -| `full` | `authbridge` image, Kubernetes proxy-sidecar | all thirteen | +| `full` | `authbridge` image, Kubernetes proxy-sidecar | all fourteen | | `lite` | `authbridge-lite` image | sidecar minimum: jwt-validation, token-exchange, litellm-budget-track, static-inject | | `envoy` | `authbridge-envoy` image | envoy-sidecar set | | `cpex` | `authbridge-cpex` image | cpex set | diff --git a/docs/framework-architecture.md b/docs/framework-architecture.md index ce975039d..9e661e78c 100644 --- a/docs/framework-architecture.md +++ b/docs/framework-architecture.md @@ -755,7 +755,7 @@ curl http://localhost:9093/config # now-active config 2. Debounce 250 ms so a symlink-swap's REMOVE+CREATE+CHMOD burst fires one reload, not three. 3. SHA-256 dedup — identical bytes are ignored (mtime-only touches don't trigger rebuilds). 4. `PipelineBuilder` runs: `config.Load` → mode override → `ApplyPreset` → `Validate` → `plugins.Build`. Any failure records the error in `Status.LastError` and leaves the active pipeline untouched. -5. Compare the new config to the active one on unreloadable fields (`Mode`, `Listener.*`, `Session.*`, `CostLedger.*`); refuse if they differ (see below). +5. Compare the new config to the active one on unreloadable fields (`Mode`, `Listener.*`, `Session.*`, `CostLedger.*`, `MTLS.*`); refuse if they differ (see below). 6. `Start` the new pipelines with a 60 s budget. On Start failure, `Stop` any partially-started pipelines so their goroutines don't leak. 7. `inboundH.Store(newIn)` + `outboundH.Store(newOut)` — new requests now route to the new pipelines. 8. Background goroutine: `time.Sleep(drainWindow)` (default 30 s) → `oldPipeline.Stop(ctx)` with a 15 s budget. In-flight requests that already Loaded the old pipeline finish against it. @@ -771,6 +771,7 @@ curl http://localhost:9093/config # now-active config | `listener.*` (ports) | ❌ | Bound sockets; refuse reload | | `session.*` (TTL, MaxEvents, MaxSessions, ID headers) | ❌ | Every consumer reads the block once at startup — `session.New(...)` in each `cmd` main, `forwardproxy.Server.SessionIDHeaders` assigned before `ListenAndServe`. There is no live object to reach; refuse reload | | `cost_ledger.*` (`enabled`, `dir`, `retention_days`) | ❌ | The ledger is a `*ledger.Writer` opened once at startup and handed to the session store as a `Recorder`. Refuse reload | +| `mtls.*` (the block's presence and `mode`, the only field it carries) | ❌ | The inbound TLS listener and, under `strict`, the forward proxy's mTLS dialer are built once at startup, and the outbound chain is built for `forwardproxy.Support` of that state; refuse reload. Legacy `cert_file` / `key_file` / `bundle_file` keys are dropped at load, so an edit to them is neither applied nor refused | **Why `session.*` and `cost_ledger.*` are refused rather than "reloaded but ineffective".** Both used to be accepted: the reload succeeded, `ReloadsOK` incremented, @@ -786,7 +787,7 @@ For the unreloadable cases, `Status.LastError` names the field(s) that changed a **Validation guarantee: bad YAML never takes the pod down.** Any failure during Load / Validate / Build / Start results in the status being updated and the active pipeline continuing to serve traffic on the previous config. Only a successful end-to-end reload swaps the holders. -**Non-reloadable choices elsewhere.** The in-memory session store, the cost ledger, the stat server, the session API server, and the reloader itself are all process-scoped — they live from startup to shutdown. A change to `session.enabled`, `cost_ledger.enabled`, `stats.address`, `listener.session_api_addr`, or the reloader's own knobs (drain window, debounce) requires a pod restart. `session.*`, `cost_ledger.*` and `listener.*` are refused outright, per the table above; `stats.address` and the reloader's own knobs are read once at construction, so an edit there is simply not seen. +**Non-reloadable choices elsewhere.** The in-memory session store, the cost ledger, the stat server, the session API server, and the reloader itself are all process-scoped — they live from startup to shutdown. A change to `session.enabled`, `cost_ledger.enabled`, `stats.address`, `listener.session_api_addr`, or the reloader's own knobs (drain window, debounce) requires a pod restart. `session.*`, `cost_ledger.*`, `listener.*` and `mtls.*` are refused outright, per the table above; `stats.address` and the reloader's own knobs are read once at construction, so an edit there is simply not seen. --- diff --git a/docs/plugin-catalog.md b/docs/plugin-catalog.md index 36a326711..a7095443e 100644 --- a/docs/plugin-catalog.md +++ b/docs/plugin-catalog.md @@ -27,6 +27,7 @@ AuthBridge pipeline YAML, not whether it is compiled into the binary | [`cpex`](#cpex) | APL DSL + named [CPEX](https://github.com/contextforge-org/cpex) plugins (Cedar, PII, audit, …) over a single chain step. | Opt-in | Outbound | No | | [`ibac`](#ibac) | LLM-judge intent-based access control for outbound tool calls. | Alpha | Outbound | No | | [`inference-parser`](#inference-parser) | Parses LLM completions into `pctx.Extensions.Inference`. | Alpha | Outbound | No | +| [`inference-router`](#inference-router) | Sends each coding agent's new sessions to the inference server chosen for it; a session stays on the server it started on. | Alpha | Outbound | No | | [`jwt-validation`](#jwt-validation) | Inbound JWT validation (signature, issuer, audience) against JWKS. | Ready | Inbound | YES | | [`lineage-telemetry`](#lineage-telemetry) | Emits two facts-only OTel lineage spans per HTTP exchange, parented across pods through one `tracestate` member. | Alpha | Both | No | | [`litellm-budget-track`](#litellm-budget-track) | Tracks `x-litellm-response-cost` (with `-original` fallback) and enforces a daily budget limit. Place on whichever chain carries LLM traffic — inbound when fronting the LLM endpoint, outbound when hosting an agent via `authbridge exec`. | Alpha | Both | No | @@ -145,6 +146,104 @@ No configuration — no config struct, does not implement `Configurable`. Rates injection from the top-level [`pricing:`](pricing.md#overriding-in-config) section; with none configured the parser still parses and reports the traffic as unpriced. +## `inference-router` + +Sends a coding agent's new sessions to the inference server chosen for it — a +LiteLLM gateway, say, with its own URL and key — and keeps every session on the +server it started on. Routing is opt-in per agent: an agent not listed under +`agents` is not routed, and its requests, keys included, pass untouched. Manage it +with [`agentop server`](../cmd/agentop/README.md#choosing-an-inference-server-agentop-server); +the config stays hand-editable. Built for the laptop: it needs a listener that +honors a redirect, so `cortex-envoy` and a config with an `mtls:` block refuse it +at startup, and in-cluster use has not been examined. + +- `servers` (map, required) — inference servers by name: lowercase letters, digits, `.`, `_` and `-`. Each has: + - `url` (string, required) — `scheme://host[:port]`, `http` or `https`, no path, query or fragment. A port must be 1–65535; the scheme's default is dropped, so `https://x:443` is `x`. No two servers may share a host, compared without port or case: a session's server is named from the host its requests went to. A plain-`http` server on another machine logs a WARN at load, since routed requests would cross the network decrypted. + - `key` (string, required) — the API key sent to this server in place of the client's, in the header the client used: `X-Api-Key` when it sent one, `Authorization: Bearer` otherwise, both when it sent both. Those two are the only headers replaced: a credential the client sends in any other header, such as one Claude Code's `ANTHROPIC_CUSTOM_HEADERS` sets, reaches the server unchanged. Printable ASCII, no spaces. `/config` and `/v1/pipeline` redact it. + - `opus`, `sonnet`, `haiku` — reserved for mapping Claude Code's model families to a server's own models, and refused until that lands. +- `agents` (map) — agent name, as agentop shows it (`claude-code`, `opencode`), to a server name. Each value must name a listed server. `unknown`, the name for requests with no User-Agent, cannot be routed. + +```yaml + - name: inference-router + config: + servers: + ete: + url: https://ete-litellm.example.com + key: sk-… + glm: + url: https://glm-litellm.example.com + key: sk-… + agents: + claude-code: glm # only agents listed here are routed +``` + +**What it does to a request.** Only a request the forward proxy re-sends — a plain +proxied one, or one decrypted by the TLS bridge — can be routed. A `CONNECT` or a +transparently redirected connection is dialed where the client chose, and is +`skip/not_redirectable`. A request to a host no server has is +`skip/not_an_inference_server`. On a server's host every path is handled, +`/v1/models` and `count_tokens` included, and the session decides: + +- The first request the router sees from a session pins it, in the process store, + to where that request went: a server when the redirect took effect, or "not + routed" when the request stayed where the client sent it — because the agent is + not routed, or because the router runs under `on_error: observe`, even for an + agent with a server. A session started under observe therefore stays unrouted + after a switch to enforce. A first request whose redirect failed pins nothing, + and the next one decides again. Every request renews the pin, which lapses 30 + days after the last. +- Where that request goes is decided by the session's history, for a routed + agent: a session already running when routing was first configured, quiet while + it was, has no pin but is not new. The latest earlier inference request that + agent sent in the session to a server's host, as the session store holds it, + keeps the session on that server, from then with that server's key. A session + with no such request is new and goes to its agent's current server. A request to + any other host is no evidence, since the router never routes one: an OpenCode + session that used another provider first is new when it addresses a server. + The history is read from the `inference` record `inference-parser` puts on each + request, so the rule needs `inference-parser` in the outbound chain; without it, a + session the router has not pinned is always treated as new. +- A pin is its agent's. A request from another agent filed under the same session + id — by process attribution, which files a command an agent runs under that + agent's session and is on by default on a laptop; by the active-session fallback + with client affinity off; or by a header id two clients share — is decided as if + the session were unpinned, from that agent's own history and server, and leaves + the pin alone. +- Changing `agents` therefore moves only new sessions while the proxy runs: the + pins and the history survive a hot reload. A restart loses both — the router reads + the store's history in memory, not the session archive on disk — so a running + session's next request after a restart is treated as a new session's, and moves + if its agent's server is another. The store also drops quiet sessions while the + proxy runs — `session.max_sessions` (100 by default) evicts the least recently + used, and a configured `session.ttl` expires them — and a session dropped before + the router pinned it is treated as new too; a pin outlives that. A request with no session follows its agent's + current server, unpinned, and so does one filed under a synthetic session — the + `default` bucket or a `pending:` id — since each holds many conversations, + not one. +- Not routed: `skip/not_routed`, and the request is left as the client sent it. +- Pinned to a server since removed: `deny/pinned_server_removed`, a 503 asking for + a new session. The conversation is never moved to another server. +- Otherwise the request is redirected to the server, and the key is replaced only + once the redirect has taken effect, so it goes to the server and nowhere else. + Every routed request is redirected, even one that already names the server's + host: the `Host` header is the client's word, and a TLS-bridged request is + otherwise dialed to the host the client `CONNECT`ed to, which need not be the + same. Each therefore carries the framework's `modify/redirected` record — with + `from` equal to `to` when the agent's base URL (`ANTHROPIC_BASE_URL`, for Claude + Code) already names the server — followed by the router's `modify/routed` with + `server` and `pin` (`new`, `existing` or `none`). +- A redirect the listener refuses: `deny/redirect_failed`, a 503. +- Under `on_error: observe` nothing moves and the client's key stays. The timeline + shows two rows: the framework's shadow `modify/redirected`, and the router's + `observe/would_route`. + +**Put it last in the outbound chain**, where `agentop server add` puts it. A +redirect moves `pctx.Host`, so the plugins before the router decide on the host +the client asked for, and a plugin after it would see the server's instead; put +nothing after it that keys on the host. Session events, usage and cost follow the +server's host, and each event's `requestedHost` keeps the one asked for when it +differs. + ## `jwt-validation` Validates inbound JWTs: signature via JWKS, issuer, and audience. diff --git a/docs/plugin-reference.md b/docs/plugin-reference.md index 864f1f806..e1b38f72e 100644 --- a/docs/plugin-reference.md +++ b/docs/plugin-reference.md @@ -219,6 +219,7 @@ type PluginCapabilities struct { ReadsBody bool WritesRequestBody bool WritesResponseBody bool + WritesDestination bool // plugin may call pctx.Redirect Requires []string // ALL must be present + earlier (hard) RequiresAny []string // AT LEAST ONE must be present + run after it (hard) @@ -814,6 +815,79 @@ session store is unauthenticated. Plugin-private debug logs may include body bytes at DEBUG level, but never publish them to the session stream or Custom map. +## Redirecting a request + +A plugin that decides where a request goes — a router, a failover — declares +`WritesDestination: true` and calls `pctx.Redirect` from `OnRequest`: + +```go +if pctx.Redirectable() { + if err := pctx.Redirect(&url.URL{Scheme: "https", Host: "glm-litellm.example.com"}); err != nil { + return pctx.DenyAndRecord("redirect_failed", "router.redirect", err.Error()) + } + // Only when the request really goes there: under on_error: observe, Redirect + // returns nil and moves nothing, but this header write would still apply. + if pctx.Redirected() { + pctx.Headers.Set("Authorization", "Bearer "+glmKey) + } +} +``` + +| Call | Effect | +|---|---| +| `pctx.Redirect(target)` | Sends the request to `target`'s scheme and host. `target` carries nothing else — no user info, no path beyond `/`, no query, no fragment. | +| `pctx.Redirectable()` | Whether the listener will honor a redirect on this request. False on a `CONNECT` and on a transparently redirected connection, which are dialed where the client chose. True on the TLS-bridged requests decrypted from either, which the forward proxy re-originates — that is how HTTPS inference traffic arrives. | +| `pctx.Redirected()` | Whether a redirect took effect. False under `on_error: observe` and after a refusal. | +| `pctx.RedirectTarget()` | The scheme and host the last accepted redirect validated, and whether there was one. Listeners apply this. | +| `pctx.RequestedHost()` | The host the client named, or `""` unless the request now goes elsewhere. | + +**`pctx.Host` follows the redirect.** The session events, usage, the cost ledger +and modelled pricing all key on it, so they describe where the bytes went. A +plugin that runs after the redirect and keys on the host sees the new one. Each +session event records the requested host beside it as `requestedHost`. Writing +`pctx.Host` does not move a request: the listener applies `RedirectTarget`, the +copy `Redirect` validated, and resets `pctx.Host` to match before recording. + +The framework records every redirect as `modify/redirected` with `from` and `to` +in `Details`. Under `on_error: observe` nothing moves and the record is a shadow. + +**What a redirect does not do.** + +- **It does not touch the headers.** The client's headers — credentials + included — go to the new host unchanged. A plugin that must not forward them + removes or replaces them itself. +- **It does not gate your header writes.** Under `on_error: observe` nothing + moves while header writes still apply, so a plugin that attaches credentials + meant for the target must do so only when the request actually goes there: + check `pctx.Redirected()` after the call, as above — it stays false under + observe and on a refusal. Redirected is the only safe gate. pctx.Host names + what the client asked for, and on a TLS-bridged request is the client's own + Host header, which need not match the address the proxy dials. A plugin that + attaches credentials meant for a server should always Redirect to that server, + even when pctx.Host already names it, so the request is dialed there. Otherwise + the target's key goes to the host the client named. +- **It does not re-run earlier plugins.** Plugins before the redirecting one in + the chain made their decisions on the requested host. + +`Redirect` is refused, with nothing changed, from a plugin that did not declare +the capability, outside `OnRequest`, for a malformed target, and on a context the +listener did not mark redirectable. + +**Build-time rules.** At most one `WritesDestination` plugin per pipeline. And +`plugins.BuildWithDeps` refuses one for a listener that cannot honor it — on +startup and on reload — using the `pipeline.ListenerSupport` each listener +package supplies: + +| Listener | Honors a redirect | +|---|---| +| forward proxy — the outbound chain in `cortex` and `cortex-cpex` | yes, unless `mtls:` is configured | +| reverse proxy — the inbound chain in `cortex` and `cortex-cpex` | no | +| ext_proc — both chains in `cortex-envoy` | no | + +A capability some listener cannot honor is added the same way: a field on +`pipeline.ListenerSupport`, a case in its `Unsupported`, and each listener's +`Support` saying whether it can. + ## Finishing requests (stateful plugins) Plugins that **reserve per-request state** in `OnRequest` — rate-limiter diff --git a/scripts/profile-tags/profiles.go b/scripts/profile-tags/profiles.go index 13ab96c20..798deacab 100644 --- a/scripts/profile-tags/profiles.go +++ b/scripts/profile-tags/profiles.go @@ -37,7 +37,8 @@ var allProfiles = []ProfileName{ // Artifact contents: // // local desktop cortex — the three parsers + tool-prune -// full authbridge image, Kubernetes proxy-sidecar — all thirteen +// full authbridge image, Kubernetes proxy-sidecar, release tarballs, +// make dev-install — all fourteen // lite authbridge-lite image — sidecar minimum // envoy authbridge-envoy image // cpex authbridge-cpex image @@ -53,6 +54,12 @@ var membership = map[PluginName][]ProfileName{ "lineage": {ProfileFull, ProfileEnvoy}, "opa": {ProfileFull, ProfileEnvoy}, + // Full only. Its redirect needs the forward proxy, so cortex-envoy would refuse + // it at build time, and only cmd/cortex registers it. Full is what the release + // tarballs and make dev-install build, so it reaches laptops; it changes no + // traffic until an agent is routed. + "inferencerouter": {ProfileFull}, + // The credential and identity plugins lite exists for, plus budget tracking. "jwtvalidation": {ProfileFull, ProfileLite, ProfileEnvoy, ProfileCpex}, "tokenexchange": {ProfileFull, ProfileLite, ProfileEnvoy, ProfileCpex}, diff --git a/scripts/profile-tags/profiles_test.go b/scripts/profile-tags/profiles_test.go index 5ddb455fc..3b5cab003 100644 --- a/scripts/profile-tags/profiles_test.go +++ b/scripts/profile-tags/profiles_test.go @@ -25,7 +25,7 @@ func TestTags_Local(t *testing.T) { // TestTags_ReproduceTodaysDefaults is the invariant for this refactor: the // convention change must not alter what any published artifact contains. Each -// want list is the set that binary links today — proxy's 13 default-on tagged +// want list is the set that binary links today — proxy's 14 default-on tagged // plugins, and envoy's and cpex's tagged-plus-unconditional imports. If one of // these drifts, a shipped artifact silently gained or lost a plugin. func TestTags_ReproduceTodaysDefaults(t *testing.T) { @@ -34,7 +34,7 @@ func TestTags_ReproduceTodaysDefaults(t *testing.T) { want []string }{ {"full", []string{ - "a2aparser", "ibac", "inferenceparser", "jwtvalidation", "lineage", + "a2aparser", "ibac", "inferenceparser", "inferencerouter", "jwtvalidation", "lineage", "litellm_budgettrack", "mcpparser", "opa", "sparc", "staticinject", "tokenbroker", "tokenexchange", "toolprune", }},