Skip to content

Fix shared native runtime scheduling across provider aliases - #1860

Closed
chris-beckman wants to merge 6 commits into
fix/intel-decision-numericsfrom
fix/local-runtime-capacity
Closed

chris-beckman wants to merge 6 commits into
fix/intel-decision-numericsfrom
fix/local-runtime-capacity

Conversation

@chris-beckman

@chris-beckman chris-beckman commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Provider aliases and decorators can share a native runtime while scheduling independently. Give each runtime a lifetime capacity identity, preserve it through debug capture and registry decoration, and use it for curation deferral and foreground/background admission. Titles yield to foreground requests before exact prompt-budget measurement as well as streaming; cancellation propagates through blocking and streaming utility calls without making the runtime unavailable.

Deterministic coverage exercises alias identity, independent runtimes, registry/debug wrappers, curation deferral, measurement preemption, cancellation and estimate fallback. The measurement correction passes 1,068 focused results at tree cb68539e7f563392ddf337118aa3a7d652494801 (commit e0d2748). The later combined runtime regression passes 1,196 Go results; the composed native suite passes 15 Python and 70 Rust tests. Combined frontend typechecking and budgets pass. Scoped receipts do not replace protected CI on the eventual landing composition.

On M1 8GB, all four full-prompt application turns settled and the host exited cleanly. Foreground measurement canceled the title request: preparation took 320 ms versus 23,759 ms before the correction. First-turn settlement was 100.156 seconds, with first model output still 80.272 seconds after request start. This qualifies scheduling behavior, not acceptable cold-prompt latency or model quality. Prompts and precision are unchanged.

Depends on #1734 → #1811 → #1812 → (#1735 + #1737) → #1736 → #1858. Final application composition also includes #1846 → #1761 → #1762 and independent reference harness #1859. Lands after other open prerequisite work. Keep draft under the user’s hold.

Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
@chris-beckman

Copy link
Copy Markdown
Contributor Author

Delivered on main through #1987 (ad462d8629) from this exact source head e0d2748db4d6. The source branch is retained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant