Repository navigation
feat(server): route hydra_config through PREFILL and HTTP decode paths (#487) - #62
Merged
Merged
Conversation
hydra_vortex#487 Factor a shared hydra_apply_config() helper from the CONFIGURE handler that classifies config keys (T1/T2/T3) and either stages or synchronously applies them based on a sync flag. This eliminates the duplicated classify-apply-stage logic. PREFILL (0x42): Parse optional hydra_config object from the request body. When present with model_path, drive the model swap directly via the shared helper (sync mode), bypassing the preset alias lookup. Keep the existing bare model alias field working for simple swaps without topology changes. HTTP decode (/v1/chat/completions): Extract hydra_config from the raw body, attach it to the server_task via a new hydra_config_json field, and apply it synchronously on the task-queue thread at the start of COMPLETION processing. This avoids the httplib worker thread race that caused the previous attempt (b831c23) to be reverted (eebbe11). rpc_servers T3: Add rpc_servers (JSON string array) to T3 classification. Stage pending servers in a file-scope static via hydra_apply_t3_mutators(), then register new endpoints via a ggml backend helper in apply_t3_rebuild() before load_model(). Tracks already-registered endpoints to avoid unsafe repeated registration. Also adds model_path to T3 classification and hydra_apply_t3_mutators() so the flat hydra_config key drives model swap through the standard T3 rebuild path.
2 of 3 tasks
1 of 2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes hydra_vortex#487 (fork side). Hydra.Core already sends a
hydra_configobject (
split_mode/tensor_split/model_path/rpc_servers/etc, fromEngineConfig.ToHydraConfigDict()) on both the PREFILL (0x42) RPC body andthe HTTP
/v1/chat/completionsdecode body — but until now nothing on thisside read it. CONFIGURE (0x40) already has a working, tested classify →
apply-T1 → stage-T2/T3 pipeline; this PR gives PREFILL and the HTTP decode
path the same capability, applied synchronously since both already own
the slot/task-queue-thread context when they run (CONFIGURE's stage-and-defer
behavior is unchanged).
hydra_apply_config()helper out of the CONFIGURE handler(sync flag: true for PREFILL/decode, false for CONFIGURE).
hydra_config, routes it through thehelper.
hydra_config.model_pathtakes priority over the legacy baremodelalias field when present./v1/chat/completions): extractshydra_configbeforeoaicompat_chat_params_parse()consumes the body, attaches it to theserver_task, applies it on the task-queue thread at the start of thattask's processing — not the httplib worker thread (an earlier attempt at
this,
b831c237c, was reverted for exactly that race —eebbe1116).rpc_serversto T3 classification; new endpoints are registered intothe global ggml RPC backend registry (same primitive
--rpc/--rpc-enginestartup parsing uses) before the model reload, deduped against
already-registered endpoints.
Does not touch binary DECODE (0x43) — Hydra.Core never calls it. Does not
touch
load_model()'s VRAM-release ordering (ggml-org#486, already merged).Test plan
cmake --preset hydra-dev && cmake --build build-hydra-dev --target llama-engine— clean build, verified independently (not just agent self-report).dense-27b-combinedrequest, confirmsplit_mode/tensor_split/rpc_serversactually land via INFO (0x41)layer_split/peer_addr/combined_head_attached, confirm no SIGKILL loop across MoE↔Dense swaps.🤖 Generated with a Paseo subagent (opencode-go/mimo-v2.5), reviewed and build-verified by Claude Code acting as lead.