[novkien]

Overview

The native server currently lets completed prompts enter token generation while other slots are still prefilling, and a legacy DFlash draft can reject image-following text positions from an M-RoPE target. This change adds a scheduler phase barrier for admitted prompts and infers the missing temporal-only M-RoPE metadata for compatible legacy drafts.

Addresses #5
Addresses #6

Additional information

For phase scheduling, compatible requests may join the PP cohort until its first member completes PP. That completion seals membership; early finishers retain their final logits and wait. TG starts only after every cohort member completes or terminates, and later PP waits until that TG cohort drains. The policy applies to streaming and non-streaming generation. This keeps the phase boundary explicit while allowing a short prompt to wait behind a longer one; an unlimited TG cohort can therefore delay later prefill.

For legacy DFlash, the binary infers temporal-only M-RoPE sections from a target that declares M-RoPE sections only when the draft has no explicit sections. Explicit draft metadata, text-only targets, and DSV4 remain unchanged. Invalid rotary dimensions fail closed before draft-context initialization. No model tensor or GGUF artifact is modified.

Focused validation passed on the combined GCC 13 build: llama-server, allocator (53 cases/345 assertions), RoPE and sampler checks; five phase/barrier cases; four n_probs variants; speculative parity with and without a draft; per-request and incompatible-group LoRA cases; four borrowed-snapshot cases; stream/non-stream equivalence; and a cache/no-cache continuation control. The cache/no-cache case was unskipped only in an out-of-source test wrapper. Python syntax and git diff --check passed. These CPU/local checks do not establish paired V100/RPC runtime behavior.

The remaining acceptance is to install the matching host and RPC binaries at the proxy-selected paths and verify live unequal-prompt PP/TG scheduling plus the Nex legacy-DFlash cold image, media/cache continuation, and text behavior through the current route. Issues remain open until that deployed behavior passes.

The implementation and focused tests were assisted by Codex. The owner authorized and reviewed the integrated candidate and remains responsible for the submitted changes; see AGENTS.md and CONTRIBUTING.md.

Requirements

• I have read and agree with the contributing guidelines.
• AI usage disclosure: YES — Codex assisted with implementation, tests, and PR preparation; the owner reviewed the integration and accepts responsibility for it.

[novkien]

Pre-merge handoff — reviewed head 2664ff5f1b1a83386a7678dcba8b1e8d768845a7

Owner and scope: Codex root task, fork issues #5/#6. Combined PR carries the reviewed 14-file candidate from base e58b770f3c2346deab036d9e4c3b2ea61ca0bbc5.

Contribution: #5 adds compatible concurrent PP cohort admission, seals membership when the first member completes, holds completed members' logits until every member completes or terminates, then drains the cohort through TG before later PP. #6 infers temporal-only M-RoPE sections for legacy DFlash drafts only when metadata is absent and the target declares M-RoPE sections; invalid dimensions fail closed.

Validation: On the GCC 13 native build, focused CPU checks passed: allocator (53 cases/345 assertions), RoPE/text parity and sampler; five phase/barrier cases; four n_probs variants; speculative parity with and without a draft; per-request and incompatible-group LoRA cases; four borrowed-snapshot cases; stream/non-stream equivalence; and the cache/no-cache control (unskipped only in an out-of-source wrapper). Python syntax and git diff --check passed. Independent source review approved this head.

Current PR checks: GitHub's python type-check run failed on two new assertions that pass their message as the second positional argument: tools/server/tests/unit/test_completion.py:587:21 and tools/server/tests/unit/test_lora.py:110:21. The check expects that positional argument to be a boolean; use the msg= keyword or an accepted assertion form. The other CI jobs are still running. The PR has not been merged.

Remaining: Correct and re-review the test assertions, rerun CI on the resulting exact head, merge only after review and enforced checks permit it, then build/install matching host/RPC binaries and run live scheduler plus Nex DFlash image/cache/text acceptance. Both issues stay open through deployment and behavior verification.

Next: Root coordinator owns the CI repair/review, merge decision, paired installation and all native/proxy acceptance. No production binary has been replaced by this PR delegate.

[novkien]

Work report — type-check failure corrected at current PR head

Owner and scope: Codex root task, combined PR #10 for issues #5/#6. Exact published head: 6dbd31a697ea6c402dd47f476a92e912103f4712 (parent 2664ff5f1b1a83386a7678dcba8b1e8d768845a7).

Why: GitHub python type-check had interpreted the second positional string to pytest.fail(...) in two new test failure paths as the helper's boolean strict argument.

Done and evidence: Replaced those two explicit failure paths with raise AssertionError(...) in test_completion.py and test_lora.py. Root ran the repository CI's ty 0.0.84 check against both affected files and the latest ty; both passed. git diff --check passed. No native server or library source changed in this correction, so the reviewed C++ build/test evidence remains applicable. The exact new head is pushed and PR checks have restarted; GitHub type-check and other workflow jobs are pending/running.

Remaining: Wait for the relevant current-head checks and review gate, then merge if permitted. Paired host/RPC installation and live native/proxy acceptance for PP/TG and Nex DFlash image/cache/text remain pending; both issues stay open.

Next: Root coordinator retains merge, build/deployment, and behavior acceptance.

[novkien]

Pre-merge handoff — exact head 6dbd31a697ea6c402dd47f476a92e912103f4712

Owner and scope: Codex root task for fork issues #5/#6. PR #10 is based on e58b770f3c2346deab036d9e4c3b2ea61ca0bbc5, contains the independently reviewed 14-file scheduler and DFlash candidate plus the two-line type-check correction, and links both open issues.

Validated: The reviewed native GCC 13 build and focused CPU/server tests passed (allocator 53 cases/345 assertions; RoPE and sampling; five PP/TG barrier cases; four n_probs variants; speculative parity with/without draft; two LoRA cases; borrowed-snapshot, stream/non-stream, and cache/no-cache continuation controls). git diff --check and Python syntax checks passed. The new exact head's GitHub python type-check passed. Source review approved the final integration.

Checks at handoff: The GitHub Server Linux and CI CPU workflows are still running/queued; branch protection and repository rulesets are absent, and those broad matrices add no enforced merge gate. The previous type-check failure was corrected before this head and current type-check is green.

Remaining acceptance: Merge this reviewed head, then install matching host/RPC binaries into the paths used by the proxy; verify manifests and active processes; run live unequal-prompt PP/TG scheduling and Nex DFlash cold-image, media/cache continuation, and text checks through the current route. These production build/deployment/model checks remain owned by the root coordinator. Issues #5 and #6 remain open until that behavior is verified.

[novkien]

Delivery report — merged; paired runtime acceptance remains open

Owner and scope: Codex root task, issues #5/#6. PR #10 merged at reviewed head 6dbd31a697ea6c402dd47f476a92e912103f4712 as squash commit 98b3e092d34b99222d9144f90e388767f02ec117 on 2026-10-01 08:30:13 UTC. GitHub readback confirms PR MERGED; both issues remain OPEN.

Evidence: The focused GCC 13 native build/server checks, CPU allocator/RoPE/sampler and server regressions, cache/text controls, independent source review, git diff --check, and exact-head GitHub Python type-check passed. Other optional GitHub CI jobs are still running/queued at merge; no branch-protection or repository-ruleset gate is configured.

Remaining: No installed native binaries were changed by this PR delivery step. The root coordinator still owns paired host/RPC build and installation at the proxy-selected paths, manifest/process verification, and live unequal-prompt PP/TG plus Nex DFlash cold-image, media/cache continuation, and text acceptance. Do not close #5 or #6 before those deployed checks pass.

Next: Root coordinator continues paired deployment and model/proxy acceptance on the merged source revision.