Repository navigation
[None][test] Unwaive test_llm_multi_gpu_pytorch gpu2 tests for release/1.3 - #20716
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
WalkthroughThe integration waiver list removes two skipped entries for multi-GPU PyTorch LLM API tests. ChangesIntegration test waivers
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~2 minutes Change: Other Suggested reviewers: Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation |
|---|---|---|
| Title check |  |
The title clearly identifies the test waiver removal, the affected test, and the release branch. |
| Description check | |
The description includes the required Description, Test Coverage, and PR Checklist sections. It explains the removed waiver entries, verification stages, rationale, and remaining waived entries. |
| Linked Issues check | |
Directly linked issue #19847 is closed and completed. It provides historical context only. No active directly linked issue imposes coding or automated-test requirements. The diff removes the two waive… |
| Out of Scope Changes check | |
The whole-PR diff changes only tests/integration/test_lists/waives.txt. It removes the two waiver entries stated in the current PR description. No unrelated change is present. The older objective su… |
| Docstring Coverage | |
No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0… |
Full details: Linked Issues check
-
Explanation
Directly linked issue
#19847is closed and completed. It provides historical context only. No active directly linked issue imposes coding or automated-test requirements. The diff removes the two waiver entries for NVBugs 6428092 and 6432826.
Full details: Out of Scope Changes check
-
Explanation
The whole-PR diff changes only
tests/integration/test_lists/waives.txt. It removes the two waiver entries stated in the current PR description. No unrelated change is present. The older objective summary mentions four additional entries, but the current PR description explicitly keeps those entries waived.
Full details: Docstring Coverage
-
Explanation
No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files.
✨ Finishing Touches
-
🧪 Generate unit tests (beta)
-
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
|
/bot run --extra-stage "DGX_H100-2_GPUs-PyTorch-Others-*, DGX_H100-2_GPUs-PyTorch-Ray-1" |
|
Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements. |
|
PR_Github #79425 [ run ] triggered by Bot. Commit: |
|
PR_Github #79425 [ run ] completed with state
|
|
/bot run --extra-stage "DGX_H100-2_GPUs-PyTorch-Others-*, DGX_H100-2_GPUs-PyTorch-Ray-1" |
|
PR_Github #79495 [ run ] triggered by Bot. Commit: |
|
PR_Github #79495 [ run ] completed with state
|
|
/bot run --extra-stage "DGX_H100-2_GPUs-PyTorch-Others-*, DGX_H100-2_GPUs-PyTorch-Ray-1" |
|
PR_Github #79541 [ run ] triggered by Bot. Commit: |
|
Add Carrot credits or activate Agent usage billing to use Autofix |
|
PR_Github #79541 [ run ] completed with state
|
|
/bot run --extra-stage "DGX_H100-2_GPUs-PyTorch-Others-*, DGX_H100-2_GPUs-PyTorch-Ray-1" --disable-fail-fast |
…e/1.3 Remove the two release/1.3 waive entries for closed NVBugs 6428092 and 6432826, a pipeline-parallel TorchSampler IndexError fixed by NVIDIA#16163, which is on release/1.3. These are the only closed-bug entries left whose tests run in pre-merge CI on the waived platform (DGX_H100 2-GPU stages). The 6428092 entry waives the whole `-m "gpu2"` selection of test_llm_multi_gpu_pytorch.py, which also holds test_llm_rpc_tp2, test_llm_rpc_streaming_tp2 and test_llm_return_logprobs_streaming_tp2. main waives those three under 6859934 because it does not have the fix in NVIDIA#19847 yet; release/1.3 has it. The other closed-bug entries (6435097, 6432818, 6695515, 6812288) only run in post-merge stages on other GPUs or in QA lists, so they stay waived until QA verifies them on the waived platform. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
02e0332 to
b7c6c49
Compare
QiJune
changed the title
|
/bot run --extra-stage "DGX_H100-2_GPUs-PyTorch-Others-*, DGX_H100-2_GPUs-PyTorch-Ray-1" --disable-fail-fast |
|
PR_Github #79728 [ run ] triggered by Bot. Commit: |
|
PR_Github #79731 [ run ] triggered by Bot. Commit: |
|
PR_Github #79728 [ run ] completed with state |
|
/bot run --stage-list "DGX_H100-2_GPUs-PyTorch-Others-*, DGX_H100-2_GPUs-PyTorch-Ray-1" --disable-fail-fast |
|
PR_Github #79734 [ run ] triggered by Bot. Commit: |
|
PR_Github #79731 [ run ] completed with state |
|
PR_Github #79734 [ run ] completed with state |
|
/bot skip --comment "Waives-only change. Both removed entries (nvbugs 6428092, 6432826) verified on b7c6c49 by PR_Github #79734 (release-1.3 L0_MergeRequest_PR #484, --stage-list DGX_H100-2_GPUs-PyTorch-Others-* + DGX_H100-2_GPUs-PyTorch-Ray-1): SUCCESS. Others-1 ran -m gpu2 12/12 passed incl. test_llm_get_stats_pp2[False-False-True]; Ray-1 passed on auto-rerun." |
|
PR_Github #79818 [ skip ] triggered by Bot. Commit: |
|
PR_Github #79818 [ skip ] completed with state |
Description
Remove the two release/1.3 waive entries for closed NVBugs 6428092 and 6432826. These are the only closed-bug entries left whose tests run in pre-merge CI on the waived platform. #20585, #20590, #20592 and #20632 handle the other closed-bug entries.
unittest/llmapi/test_llm_multi_gpu_pytorch.py -m "gpu2"(all)IndexError, fixed by #16163, which is on release/1.3DGX_H100-2_GPUs-PyTorch-Others-*,DGX_H100-2_GPUs-PyTorch-Ray-1unittest/llmapi/test_llm_multi_gpu_pytorch.py::test_llm_get_stats_pp2[False-False-True](all)DGX_H100-2_GPUs-PyTorch-Others-*On main, #19642 removed both entries; since 10-01,
test_llm_get_stats_pp2has passed in all 245 DGX_H100 runs where it ran there.Note on the
-m "gpu2"entryThe 6428092 entry waives the whole
-m "gpu2"selection oftest_llm_multi_gpu_pytorch.py, which also holdstest_llm_rpc_tp2,test_llm_rpc_streaming_tp2andtest_llm_return_logprobs_streaming_tp2[None-1-False-False]. main waives those three under NVBug 6859934 because it does not have the fix #19847 yet. release/1.3 has #19847, so they are not re-waived here; the DGX_H100 2-GPU stages in this PR are their first run on release/1.3.Entries left waived
An earlier version of this PR also removed the four entries below. They were dropped because no pre-merge stage runs them on the waived platform, so CI cannot verify them. They stay waived until QA verifies them on that platform.
test_disaggregated_ctxtp2pp2_gentp2pp2[TinyLlama-1.1B-Chat-v1.0](B200)TestDeepSeekV3Lite::test_bfloat16_4gpus[pp4-mtp_nextn=2-attention_dp=True-cuda_graph=True-overlap_scheduler=True-torch_compile=True](GB300)test_eagle3_output_repetition_4gpus[Qwen3/Qwen3-30B-A3B-Qwen3/Qwen3-30B-eagle3](B200)test_multi_nodes_eval[DeepSeek-R1/DeepSeek-R1-0528-FP4-tp16-mmlu](GB200, multi-node)Test Coverage
Test-list change only. Both entries are covered by:
--disable-fail-fastis used because three earlier runs on the previous version of this PR (pipelines 428, 440 and 454) each hit an unrelated single-GPU failure, and fail-fast then blocked the multi-GPU phase, so these stages never ran.PR Checklist
🤖 Generated with Claude Code
Dev Engineer Review
The change updates only
tests/integration/test_lists/waives.txt. It removes two skipped entries forunittest/llm_multi_gpu_pytorch.py. No source code, API, or runtime behavior changes are included.QA Engineer Review
The change removes the
gpu2selector andtest_llm_get_stats_pp2[False-False-True]waiver. CI reruns reported failures, so coverage verdict: needs follow-up.Per-File QA Perspective
tests/integration/test_lists/waives.txt: Removes two waiver entries from the integration test list. Verification must confirm that the unwaived multi-GPU tests pass.