server, speculative-simple: wire dspark tap capture - #67
Conversation
There was a problem hiding this comment.
Pull request overview
Wires dspark capture-based speculative decoding into the server and speculative example.
Changes:
- Reads dspark metadata and enables target-layer capture.
- Stages prompt and verification capture rows.
- Adjusts draft context sizing and disables prompt-prefix reuse.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
tools/server/server-context.cpp |
Integrates dspark capture into server request processing. |
examples/speculative-simple/speculative-simple.cpp |
Adds a dspark-specific speculative decoding flow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if (cparams.n_batch < cparams.n_ctx) { | ||
| SRV_INF("draft-dspark: raising draft ctx n_batch %u -> %u (full-context staging)\n", | ||
| cparams.n_batch, cparams.n_ctx); | ||
| cparams.n_batch = cparams.n_ctx; |
There was a problem hiding this comment.
Fixed in 2fc57d2 — draft ctx batch is now sized n_ctx + block_size (block_size from the drafter GGUF; +64 fallback if the KV is missing), so a full round fits even at the context limit.
| if (spec_dspark) { | ||
| cparams.n_batch = std::max(cparams.n_batch, cparams.n_ctx); | ||
| cparams.n_ubatch = std::max(cparams.n_ubatch, cparams.n_batch); |
There was a problem hiding this comment.
Fixed in 2fc57d2 — same n_ctx + block_size sizing here, using a local block_size reader.
| const bool spec_dspark = std::find(params_base.speculative.types.begin(), | ||
| params_base.speculative.types.end(), | ||
| COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK) != params_base.speculative.types.end(); |
There was a problem hiding this comment.
Fixed in 2fc57d2 — ctx_shift is now force-disabled when capture-mode speculation initializes (same pattern as the existing mtmd/context checks), with a warning. Proper shift support (reset + recapture) left as future work.
| // capture-type drafters already ran begin() before prompt decode (see | ||
| // SLOT_STATE_STARTED) -- running it again here would wipe the prompt's | ||
| // staged capture rows. | ||
| if (slot.can_speculate() && !common_speculative_need_embd_capture(spec.get())) { |
There was a problem hiding this comment.
Fixed in 2fc57d2 — added server_slot::spec_disabled, set at launch for is_child() slots when the spec needs capture; can_speculate() respects it, so n_cmpl children run plain AR instead of erroring every round. Copying per-sequence speculative state on clone left as future work.
| const uint32_t block_size = server_read_dspark_block_size(params_spec.mparams.path); | ||
| if (block_size > 0) { | ||
| const uint32_t n_out_dspark = params_base.n_parallel * (1 + block_size); |
There was a problem hiding this comment.
Fixed in 2fc57d2 — init now validates --spec-draft-n-max == block_size (read from the drafter GGUF) and fails model load with a clear error on mismatch. Verified: --spec-draft-n-max 6 against a block-4 drafter now exits with draft-dspark: --spec-draft-n-max (6) must equal the drafter's block_size (4).
…ctx-shift and child-slot gating
Wire dspark tap capture into
tools/serverandexamples/speculative-simpleso--spec-type draft-dsparkworks outside the test harness.llama_set_capture_layers(..., masked=false); layer ids read from the drafter GGUFcommon_speculative_begin()before prompt decode; capture rows staged viacommon_speculative_process()on prefill/verify batchesn_outputs_max,n_batch >= n_ctx)--spec-draft-n-maxmust equal the drafter'sblock_size. Validated single-slot; greedy output identical with/without the drafter.