Skip to content

bench: RTX 5090 Laptop GPU follow-up for 0.1.40.2 and 0.1.40.3 - #1462

Closed
wolffahrer wants to merge 1 commit into
Niko1221:mainfrom
wolffahrer:community-bench-rtx-5090-laptop-0.1.40.3
Closed

wolffahrer wants to merge 1 commit into
Niko1221:mainfrom
wolffahrer:community-bench-rtx-5090-laptop-0.1.40.3

Conversation

@wolffahrer

Copy link
Copy Markdown
Contributor

Follow-up to #1263 (now in bench/results/2026-10-06-community-rtx-5090-laptop): same laptop, model files, configuration and scripts, only the Strata version changed. Results only, no engine changes.

New folder: bench/results/2026-10-08-community-rtx-5090-laptop-engine-0.1.40.3/

Median decode / prompt tok/s (Q2_0, 131K context, greedy, reasoning off, 256-token cap):

Prompt tokens 0.1.40.1 0.1.40.2 0.1.40.3
4,096 136.9 / 1,952.5 139.3 / 1,960.0 138.1 / 1,949.7
32,768 137.7 / 2,877.8 141.6 / 2,890.0 144.4 / 2,893.3
128,000 134.6 / 2,764.1 137.2 / 2,764.1 131.0 / 2,780.1
  • No measurable speed change on this hardware and no regression; differences are inside single-run min–max ranges.
  • Needles 6/6 on both versions. Coding check 10/10 on 0.1.40.3.
  • The 0.1.40.2 coding check scored 0/10. Re-runs with raw responses show the same greedy reasoning loop in 0.1.40.1 (first request after a fresh start, identical text in both versions, ended by reasoning_loop_recovery: "stop"), so we don't think it is a 0.1.40.2 regression. Details are in the README.
  • Raw data is included for 0.1.40.3 only. telemetry.jsonl is left out, memory-summary.json is kept.

🤖 Generated with Claude Code

Same laptop, model files, configuration and scripts as the 0.1.40.1
entry (2026-10-06-community-rtx-5090-laptop, Niko1221#1263); only the Strata
version changed. Raw data for 0.1.40.3, medians for 0.1.40.2 in the
README. Decode and prompt throughput unchanged within run-to-run
variation, all needles found, coding check 10/10 on 0.1.40.3. A 0/10
coding check on 0.1.40.2 is traced to a greedy reasoning loop that
occurs in 0.1.40.1 as well (re-runs documented). Results only, no
engine changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Niko1221

Copy link
Copy Markdown
Owner

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

  1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
  2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
  3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
  4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

@Niko1221 Niko1221 closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants