Repository navigation
[CUDA] Guard FP8 XQA test by compute capability - #32559
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The capability check must target the same CUDA device used by the test.
Pull request overview
Guards FP8 XQA tests to run only on GPUs with SM89+ capability, preventing failures on SM86 while retaining UINT8/INT8 coverage.
Changes:
- Added capability detection and simulated SM86/SM89/SM90 tests.
- Conditionally includes FP8 test cases.
File summaries
| File | Summary |
|---|---|
onnxruntime/test/python/transformers/test_paged_attention_int4.py |
Adds the capability guard; the device queried should match the CUDA device used by run_case. |
Review details
Suppressed comments (1)
onnxruntime/test/python/transformers/test_paged_attention_int4.py:31
- This reads PyTorch's current device, but
run_casebinds every ORT tensor/session to CUDA device 0 (lines 302-324). If another test leaves a different device current, the guard can skip FP8 coverage on device 0 or run it on an SM<89 device; query the same device that the test executes on.
major, minor = torch.cuda.get_device_capability()
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Hariharan Seshadri (hariharans29)
approved these changes
Sep 11, 2026
adrastogi
pushed a commit
that referenced
this pull request
Oct 23, 2026
### Description The Windows CUDA CI runner for PR #32515 uses an SM86 A10-4Q. `PagedAttention` intentionally enables FP8 XQA only on SM89 or SM90+, so the FP8 cases in `test_xqa_large_attention_scale_and_k_scale` fell back to portable kernels and failed the XQA dispatch assertion. Mirror the runtime capability predicate in the test and include the FP8 variant only on supported GPUs. UINT8 and INT8 remain covered on SM80 and newer devices. This is a capability guard, not a memory guard: the failed job completed pytest normally and reported two assertion failures rather than a CUDA allocation failure or process crash. Failed job: https://github.com/microsoft/onnxruntime/actions/runs/34435175610/job/102757939916?pr=32515 ### Motivation and Context Fixes the Windows GPU CUDA CI regression introduced by #32515 without skipping valid XQA coverage on SM86. ### Testing - `TestPagedAttentionInt4Helpers`: 4 tests passed, including simulated SM86, SM89, and SM90 capability cases. - `lintrunner onnxruntime/test/python/transformers/test_paged_attention_int4.py` - `python -m py_compile onnxruntime/test/python/transformers/test_paged_attention_int4.py` - `git diff --check` (cherry picked from commit 8ec7981) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3eb3fc9b-6147-4706-b7ad-47bb3476377b
adrastogi
added a commit
that referenced
this pull request
Oct 27, 2026
### Description <!-- Describe your changes. --> This cherry-picks the following commits for the release: * #32610 * #32633 * #32641 * #32611 * #32609 * #32607 * #32679 * #32681 * #32666 CI fixes: * #32559 (`8ec7981a7439546a54196b9cbc777ae5de708b6b`): guard FP8 XQA test cases by GPU compute capability. The Windows CUDA runner is an SM86 A10, while FP8 XQA requires SM89 or SM90+. INT4/INT8 XQA coverage remains enabled on supported SM80+ devices. This fixes the two FP8 dispatch assertions without changing product behavior. * Raise the Android minimal baseline binary-size threshold by 1 KiB, from 1,589,248 to 1,590,272 bytes (release branch only). CI reports a section total of 1,590,022 bytes, exceeding the previous threshold by 774 bytes. The adjusted threshold leaves 250 bytes of headroom. * #32676, then #33489: rename the old `A10-Test` image and remove the unnecessary image override, in upstream order. * #33492: backport the scale-set runner migration across the release branch's 41 affected workflows. The Linux CUDA, CUDA Plugin, and TensorRT test jobs now use `ort-github-linux-6-a10-scus-ss`. Their previous retries were cancelled by `1es-resource-management[bot]` before any steps executed. The first ten cherry-picks, plus #32676 and #33489, applied cleanly from the original mainline commits, with source commit provenance preserved. #32607 includes the full mainline fix and regression tests rather than the reduced 1.28.3 backport. #33492 required conflict resolution because the release branch has an older workflow layout: * Preserve the existing Linux CI and Linux WebGPU job sets; do not introduce main-only no-exceptions or static-plugin jobs. * Leave `windows_arm64_python.yml` absent. It was introduced after 1.30 branched by #32791; its runner migration does not require adding that workflow to this release. * Preserve Windows WebGPU's combined build/test jobs and their check names, keeping them on the new GPU scale set rather than importing main's split build/test topology. The build-only minimal Edge job uses the new CPU scale set. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Bring the product fixes from #32806 into ORT 1.30.1, address the Windows CUDA test and Android size-gate failures, and migrate CI runner selection to unblock the cancelled GPU jobs. The merged EP API version correction from #33525 was brought in by merging the updated `rel-1.30.1` base. The `uuid-dev` CI fix in #32491 was already present in the 1.30 base and is not duplicated. The OpenVINO changes in #33112 and #33327 are intentionally excluded: the namespace regression fixed by #33327 was introduced by #33112, and neither change is present in the 1.30 base. Validation: * All nine original product cherry-pick patch IDs were verified against their mainline commits; the additional #32559 patch ID also matches upstream. * The upstream `test_fp8_xqa_cuda_capability` regression passed in an isolated standard-library harness using mocked SM86, SM89, and SM90 capabilities. Additional no-CUDA and external-runner cases passed. * Python syntax and Android threshold consistency/arithmetic checks passed. * `git diff --check 3b0f46f HEAD` passed. * All 41 migrated workflow YAML files parse. Structural comparisons against the pre-migration release workflows confirm that only runner labels, caller pool names, and the reusable pool-input description changed; job sets, steps, triggers, permissions, matrices, and build flags are preserved. * All 13 callers of the reusable Linux build workflow use scale-set pool names; all three previously cancelled GPU test jobs use the new A10 scale set. * The Android release-specific size threshold remains 1,590,272 bytes. * The full helper pytest invocation was not runnable because pytest is not installed in the local virtual environment. No local GPU execution or Android rebuild was performed; CI will validate the integrated changes. --------- Co-authored-by: Wei Wang <wei4.wang@intel.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: shiyi <shiyi.zou@intel.com> Co-authored-by: Bin Miao <bin.miao@intel.com> Co-authored-by: Bryan B <bryan.bernhart@intel.com> Co-authored-by: Xiaofei Han <xiaofeihan@microsoft.com> Co-authored-by: Aditya Rastogi <adityar@ntdev.microsoft.com> Co-authored-by: Tianlei Wu <tlwu@microsoft.com> Co-authored-by: eserscor <erscor@microsoft.com> Copilot-Session: 3eb3fc9b-6147-4706-b7ad-47bb3476377b
This was referenced Oct 30, 2026
This was referenced Nov 2, 2026
Open
This was referenced Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The Windows CUDA CI runner for PR #32515 uses an SM86 A10-4Q.
PagedAttentionintentionally enables FP8 XQA only on SM89 or SM90+, so the FP8 cases intest_xqa_large_attention_scale_and_k_scalefell back to portable kernels and failed the XQA dispatch assertion.Mirror the runtime capability predicate in the test and include the FP8 variant only on supported GPUs. UINT8 and INT8 remain covered on SM80 and newer devices. This is a capability guard, not a memory guard: the failed job completed pytest normally and reported two assertion failures rather than a CUDA allocation failure or process crash.
Failed job: https://github.com/microsoft/onnxruntime/actions/runs/34435175610/job/102757939916?pr=32515
Motivation and Context
Fixes the Windows GPU CUDA CI regression introduced by #32515 without skipping valid XQA coverage on SM86.
Testing
TestPagedAttentionInt4Helpers: 4 tests passed, including simulated SM86, SM89, and SM90 capability cases.lintrunner onnxruntime/test/python/transformers/test_paged_attention_int4.pypython -m py_compile onnxruntime/test/python/transformers/test_paged_attention_int4.pygit diff --check