Skip to content

bench: fix multi-step Flash fixture padding - #14

Merged
novkien merged 1 commit into
masterfrom
codex/flash9-paddingfix
Oct 3, 2026
Merged

novkien merged 1 commit into
masterfrom
codex/flash9-paddingfix

Conversation

@novkien

@novkien novkien commented Oct 3, 2026 •

Copy link
Copy Markdown
Owner

Overview

Fixes generated fixture preparation when more than one padding step is needed. Padding is inserted before the final question, so it must be tracked directly; slicing the rendered content after the base prompt length can take text from the question instead of the accumulated padding. The regression test covers the multi-step case.

Addresses #9

Validation

  • python3 -m unittest -v tests.test_bench_flash_next (14 tests passed)
  • Python 3.11 flake8 with flake8-no-print on the two changed files
  • Python compilation and git diff --check
  • Live native token preparation passed for all 18 combinations: three fixture families, two lanes, and target lengths 2,048, 8,192, and 32,768, seed 731.

This is a correctness fix for benchmark preparation and makes no inference performance claim.

Requirements

  • Contributing guidelines reviewed for this submission.
  • AI usage disclosure: YES - Codex implemented and reviewed this change under the owner-approved Flash Next plan.

Track the inserted padding directly so follow-up token probes extend the pad instead of slicing from the question text. Add a regression for prompts that require multiple padding steps.

Assisted-by: Codex
@novkien

novkien commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Validation update for e1959f0abf370813d5fdc7ff60697fe5c3e8dcd9

The multi-step padding fix and regression are on the reviewed head. CI Lint passed (run 37096328003); editorconfig, labeler, model-naming, and one Python type-check job passed. The second type-check job was still pending at this observation. Local 14-test suite, Python compilation, scoped type-check, and git diff --check passed. Live native token preparation passed all 18 combinations: three families, two lanes, and 2,048/8,192/32,768 tokens at seed 731.

This corrects benchmark fixture generation only; it makes no inference speedup claim.

@novkien
novkien merged commit 668bee9 into master Oct 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants