Repository navigation
Fix shared native runtime scheduling across provider aliases - #1860
Closed
chris-beckman wants to merge 6 commits into
Closed
chris-beckman wants to merge 6 commits into
chris-beckman wants to merge 6 commits into
Conversation
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
This was referenced Nov 5, 2026
Contributor
Author
|
Delivered on main through #1987 ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Provider aliases and decorators can share a native runtime while scheduling independently. Give each runtime a lifetime capacity identity, preserve it through debug capture and registry decoration, and use it for curation deferral and foreground/background admission. Titles yield to foreground requests before exact prompt-budget measurement as well as streaming; cancellation propagates through blocking and streaming utility calls without making the runtime unavailable.
Deterministic coverage exercises alias identity, independent runtimes, registry/debug wrappers, curation deferral, measurement preemption, cancellation and estimate fallback. The measurement correction passes 1,068 focused results at tree cb68539e7f563392ddf337118aa3a7d652494801 (commit e0d2748). The later combined runtime regression passes 1,196 Go results; the composed native suite passes 15 Python and 70 Rust tests. Combined frontend typechecking and budgets pass. Scoped receipts do not replace protected CI on the eventual landing composition.
On M1 8GB, all four full-prompt application turns settled and the host exited cleanly. Foreground measurement canceled the title request: preparation took 320 ms versus 23,759 ms before the correction. First-turn settlement was 100.156 seconds, with first model output still 80.272 seconds after request start. This qualifies scheduling behavior, not acceptable cold-prompt latency or model quality. Prompts and precision are unchanged.
Depends on #1734 → #1811 → #1812 → (#1735 + #1737) → #1736 → #1858. Final application composition also includes #1846 → #1761 → #1762 and independent reference harness #1859. Lands after other open prerequisite work. Keep draft under the user’s hold.