Skip to content

[TRTLLM-17005][refactor] Component-local C++ Tests (Part 3a) - #21022

Merged
Funatiq merged 13 commits into
NVIDIA:mainfrom
Funatiq:dev/refactor/component-local-tests-3-a
Oct 29, 2026
Merged

Funatiq merged 13 commits into
NVIDIA:mainfrom
Funatiq:dev/refactor/component-local-tests-3-a

Conversation

@Funatiq

@Funatiq Funatiq commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Dev Engineer Review

The change extracts selected kernels into static component libraries and links them into tensorrt_llm with whole-archive semantics. It also updates CUTLASS and FMHA dependencies, source discovery, and kernel-test target wiring. The FMHA custom-mask archive remains a private implementation dependency. The supplied information does not establish test results for this source state.

Review findings and severity counts are unavailable. CI reports failures for builds 66367, 66380, 66394, and 66423, and an unstable result for build 66548. The comments do not identify specific failing tests.

QA Engineer Review

cpp/tests/unit_tests/kernels/CMakeLists.txt migrates kernel test targets to add_tllm_gtest with explicit owners, dependencies, source files, and labels. The registrations cover logits, NVFP4 cold-page, attention masks, RoPE and shift-K-cache, MLA, chunked prefill, sparse attention, MoE communication, load balancing, conditional MoE LoRA, and sparse KV cache. The NVFP4 cold-page test now links reusable component libraries instead of listing support sources and link settings directly.

Changed test functions and IDs are not identified in the supplied information. No changes to tests/integration/test_lists/ are reported, so CI or manual-QA listing status cannot be established. Coverage verdict: needs follow-up. The supplied CI summary reports failures and an unstable pipeline, but does not identify specific failing tests.

Per-File QA Perspective

  • cpp/tensorrt_llm/CMakeLists.txt: Whole-archive links retain component kernels in the shared library. Verify that each registered component is linked correctly.
  • cpp/tensorrt_llm/kernels/CMakeLists.txt: Propagates the component target list to the parent scope. Verify that consumers receive the complete list.
  • cpp/tensorrt_llm/kernels/cutlass_kernels/CMakeLists.txt: Changes LoRA source discovery and links the LoRA component when OSS MoE GEMM is enabled. Verify both conditional paths.
  • cpp/tensorrt_llm/kernels/kernelComponents.cmake: Adds static component-library registration, source removal, and conditional MoE LoRA registration. Verify source ownership and dependency configuration.
  • cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/CMakeLists.txt: Excludes prepareCustomMask.cu from recursive discovery and privately links the attention-mask component. Verify custom-mask behavior and source ownership.
  • cpp/tests/unit_tests/kernels/CMakeLists.txt: Migrates kernel test targets and replaces manually listed NVFP4 support sources with component libraries. Verify dependencies, labels, and conditional MoE LoRA registration.

Description

Implement component-local tests as real CMake target dependencies. This PR migrates some of the kernel tests.

  • Extracts 10 kernel families into dedicated static libraries, covering logits, NVFP4 cold pages, RoPE/QKV preprocessing, sparse attention, MLA, MoE communication/load balancing/LoRA, and custom attention masks.
  • Migrates 14 test executables to add_tllm_gtest, with explicit component dependencies, ownership, and scheduling labels. This reduces their dependency on the full TensorRT-LLM library.
  • Keeps the extracted kernels included in the main shared library through whole-archive linking and updates CUTLASS/FМHA dependencies to avoid duplicate compilation.
  • Replaces the NVFP4 test’s manually assembled support sources with reusable component libraries.
  • Removes GPU resource locks introduced during the migration.

All committed changes are CMake build/test wiring; kernel implementations and test bodies are unchanged.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

@Funatiq

Funatiq commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80609 [ run ] triggered by Bot. Commit: 66f2c7f Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80609 [ run ] completed with state FAILURE. Commit: 66f2c7f
/LLM/main/L0_MergeRequest_PR pipeline #66367 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Funatiq
Funatiq force-pushed the dev/refactor/component-local-tests-3-a branch from 66f2c7f to 6067e22 Compare October 26, 2026 14:57
@Funatiq

Funatiq commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80625 [ run ] triggered by Bot. Commit: 6067e22 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80625 [ run ] completed with state FAILURE. Commit: 6067e22
/LLM/main/L0_MergeRequest_PR pipeline #66380 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Funatiq
Funatiq force-pushed the dev/refactor/component-local-tests-3-a branch from 6067e22 to e144d1b Compare October 26, 2026 16:02
@Funatiq

Funatiq commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80642 [ run ] triggered by Bot. Commit: e144d1b Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80642 [ run ] completed with state SUCCESS. Commit: e144d1b
/LLM/main/L0_MergeRequest_PR pipeline #66394 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Funatiq
Funatiq force-pushed the dev/refactor/component-local-tests-3-a branch from e144d1b to 9a8e8fd Compare October 26, 2026 18:36
@Funatiq

Funatiq commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80671 [ run ] triggered by Bot. Commit: 9a8e8fd Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80671 [ run ] completed with state SUCCESS. Commit: 9a8e8fd
/LLM/main/L0_MergeRequest_PR pipeline #66423 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Funatiq

Funatiq commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80816 [ run ] triggered by Bot. Commit: 9a8e8fd Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #80816 [ run ] completed with state SUCCESS. Commit: 9a8e8fd
/LLM/main/L0_MergeRequest_PR pipeline #66548 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

  • Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
  • If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
    Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@Funatiq
Funatiq marked this pull request as ready for review October 27, 2026 08:50
@Funatiq
Funatiq requested review from a team as code owners October 27, 2026 08:50
@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 73d65ca and 14479fb.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 9a8e8fd and 73d65ca.

Files selected for processing: 3
  • cpp/tensorrt_llm/kernels/kernelComponents.cmake
  • cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/CMakeLists.txt
  • cpp/tests/unit_tests/kernels/CMakeLists.txt

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The build now defines kernel component libraries, connects them to the shared target, and updates FMHA and MoE source wiring. Kernel unit-test declarations now use add_tllm_gtest with explicit owners, sources, dependencies, and labels.

Changes

Kernel component build

Layer / File(s) Summary
Define kernel component libraries
cpp/tensorrt_llm/kernels/kernelComponents.cmake
A new CMake helper validates arguments, creates static kernel libraries and namespaced aliases, and registers component targets with their sources and dependencies.
Wire component libraries into kernel sources
cpp/tensorrt_llm/kernels/cutlass_kernels/CMakeLists.txt, cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/CMakeLists.txt
The MoE and FMHA source lists exclude files handled by component libraries and link the corresponding targets. MOE_CUTLASS_DIR is converted to an absolute path.
Propagate and link kernel components
cpp/tensorrt_llm/kernels/CMakeLists.txt, cpp/tensorrt_llm/CMakeLists.txt
The kernel directory propagates component targets to its parent. The shared target links each component with WHOLE_ARCHIVE and sets its link-library override accordingly.
Migrate kernel test declarations
cpp/tests/unit_tests/kernels/CMakeLists.txt
Kernel tests use add_tllm_gtest declarations with explicit owners, sources, dependencies, and labels. The NVFP4 cold-page test links the kernel and KV-cache host-memory libraries.

Priority: Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Suggested reviewers: qijune

Merge Risk: Minimal · previous review 73d65

No concrete build or test-wiring defect is established by the reviewed changes; reported CI failures still need their individual causes identified through normal validation.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly identifies the ticket, change type, and main change: migrating component-local C++ tests.
Description check Passed The description clearly explains the CMake and test-wiring changes and includes the required sections and checklist. The Test Coverage section is empty and does not list specific tests or commands, bu…
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within survi…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description clearly explains the CMake and test-wiring changes and includes the required sections and checklist. The Test Coverage section is empty and does not list specific tests or commands, but the overall description is sufficiently complete.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 0 files. (3 skipped: 3 unsupported.)

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@yuxianq

yuxianq commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

@farazkh80, could you help to review this PR as trt-llm-torch-attention-devs? Thanks~

Assignment explanation
Componentsintegration/build-config (100%)
Tagsfeatures:custom-mask (100%)
PR effort3 points — The small CMake refactor requires checking that the component dependency preserves custom-mask symbols and avoids duplicate compilation. Missing dependency definitions make this more demanding than a mechanical edit.
Affinity (0–1)0.500 component/tag match
Outstanding points at selection3 → 6 (+3)
Points already reserved on this PR0
Effective weight1
Completed review points (preceding 168 hours)65
Combined points after assignment71
Demand-derived target61.00 points (soft share)
Additional fair share before assignment0.00 points
Share explanationWhole PR / qualification or coverage constraint exceeds a soft share; assignment is allowed.
Selection rulebalance-first maintenance
Assignment scope (farazkh80)all

Affinity and workload are snapshots at selection, not live counters. Existing reservations on this PR are counted once.

Keep the FMHA interface public while making the custom-mask kernel archive an implementation dependency. The shared library already retains the archive through its private whole-archive component linkage, so downstream consumers do not need it in the public link interface.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
@Funatiq
Funatiq force-pushed the dev/refactor/component-local-tests-3-a branch from 73d65ca to 14479fb Compare October 28, 2026 10:46
@Funatiq

Funatiq commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81169 [ run ] triggered by Bot. Commit: 14479fb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81169 [ run ] completed with state SUCCESS. Commit: 14479fb
/LLM/main/L0_MergeRequest_PR pipeline #66864 completed with status: 'FAILURE'

CI Report

⚠️ Resource Maintenance — Test Coverage Reduced:

Resource maintenance in LLM/main/L0_Test-x86_64-Single-GPU #9590: L40S-PyTorch-2 is temporarily paused due to Kubernetes agent provisioning and node issues.
Affected stages:
L40S-PyTorch-2

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Funatiq

Funatiq commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81185 [ run ] triggered by Bot. Commit: 14479fb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81185 [ run ] completed with state SUCCESS. Commit: 14479fb
/LLM/main/L0_MergeRequest_PR pipeline #66879 completed with status: 'FAILURE'

CI Report

⚠️ Resource Maintenance — Test Coverage Reduced:

Resource maintenance in LLM/main/L0_Test-x86_64-Single-GPU #9103: L40S-PyTorch-2 is temporarily paused due to Kubernetes agent provisioning and node issues.
Affected stages:
L40S-PyTorch-2

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@brnguyen2 brnguyen2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — the comments below are optional touch-ups, not blockers.

Moving each test onto add_tllm_gtest with a narrow LINK_LIBRARIES list is a clear improvement. The nvfp4 cold-page test in particular no longer carries its hand-maintained list of common/*.cpp and kv_cache_manager_v2/utils/*.cpp sources.

Two things before merge:

  • CI status: the PR history shows several failed or unstable pipelines. Please post a green /bot run on the current head. This change moves sources between link units, so only a full build plus the migrated kernels ctest labels can show that nothing went missing or got duplicated in libtensorrt_llm.so.
  • Description: the PR description is cut off partway through the list of extracted families ("MoE communication/loa…"). Please complete it, and say what "Part 3a" covers compared with the later parts, so a future reader can tell which kernel tests are still on add_gtest and why.

@github-actions

Copy link
Copy Markdown

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

@Funatiq

Funatiq commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81219 [ run ] triggered by Bot. Commit: 14479fb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81219 [ run ] completed with state FAILURE. Commit: 14479fb
/LLM/main/L0_MergeRequest_PR pipeline #66910 completed with status: 'FAILURE'

CI Report

⚠️ Resource Maintenance — Test Coverage Reduced:

Resource maintenance in LLM/main/L0_Test-x86_64-Single-GPU #10088: L40S-PyTorch-2 is temporarily paused due to Kubernetes agent provisioning and node issues.
Affected stages:
L40S-PyTorch-2

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Funatiq

Funatiq commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81255 [ run ] triggered by Bot. Commit: 14479fb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81255 [ run ] completed with state SUCCESS. Commit: 14479fb
/LLM/main/L0_MergeRequest_PR pipeline #66944 completed with status: 'FAILURE'

CI Report

⚠️ Resource Maintenance — Test Coverage Reduced:

Resource maintenance in LLM/main/L0_Test-x86_64-Single-GPU #9613: L40S-PyTorch-2 is temporarily paused due to Kubernetes agent provisioning and node issues.
Affected stages:
L40S-PyTorch-2

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Funatiq

Funatiq commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81337 [ run ] triggered by Bot. Commit: 14479fb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81337 [ run ] completed with state FAILURE. Commit: 14479fb
/LLM/main/L0_MergeRequest_PR pipeline #67020 completed with status: 'FAILURE'

CI Report

⚠️ Resource Maintenance — Test Coverage Reduced:

Resource maintenance in LLM/main/L0_Test-x86_64-Single-GPU #9637: L40S-PyTorch-2 is temporarily paused due to Kubernetes agent provisioning and node issues.
Affected stages:
L40S-PyTorch-2

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@Funatiq

Funatiq commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81389 [ run ] triggered by Bot. Commit: 14479fb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #81389 [ run ] completed with state SUCCESS. Commit: 14479fb
/LLM/main/L0_MergeRequest_PR pipeline #67064 completed with status: 'SUCCESS'

CI Report

Link to invocation

@Funatiq
Funatiq merged commit e655dd8 into NVIDIA:main Oct 29, 2026
14 checks passed
@Funatiq
Funatiq deleted the dev/refactor/component-local-tests-3-a branch October 29, 2026 10:16
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 2, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 3, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 3, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 3, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants