Skip to content

[AutoDeploy]: analyze and fix perf of meta-llama/Llama-4-Scout-17B-16E-Instruct #8733

Closed
Closed
[AutoDeploy]: analyze and fix perf of meta-llama/Llama-4-Scout-17B-16E-Instruct#8733
Assignees
tcherckez-nvidia
Labels
AutoDeploy<NV> AutoDeploy Backend

Description

@nzmora-nvidia
nzmora-nvidia
opened on Oct 28, 2025

🚀 The feature, motivation and pitch

TRT-LLM torch manual is about 3.5 faster than AD.
Analyze the differences and fix.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Activity

  1. nzmora-nvidia
    assigned
    tcherckez-nvidia
    on Oct 28, 2025
  2. nzmora-nvidia
    added
    AutoDeploy<NV> AutoDeploy Backend
    on Oct 28, 2025
  3. github-project-automation
    added this to AutoDeploy Boardon Oct 28, 2025
  4. github-project-automation
    moved this to Backlog in AutoDeploy Boardon Oct 28, 2025
  5. nzmora-nvidia
    moved this from Backlog to In progress in AutoDeploy Boardon Oct 28, 2025
  6. tcherckez-nvidia
    added a commit that references this issue on Nov 30, 2025

    [NVIDIA#8733][Perf]: Add Llama4 MoE handling to AutoDeploy

    f8526ae
  7. tcherckez-nvidia
    moved this from In progress to In review in AutoDeploy Boardon Nov 30, 2025
  8. tcherckez-nvidia
    added a commit that references this issue on Dec 4, 2025

    [#8733][feat] Add Llama4 MoE handling to AutoDeploy (#9556)

    f9aa86d
  9. tcherckez-nvidia
    moved this from In review to Done in AutoDeploy Boardon Dec 4, 2025
  10. tcherckez-nvidia
    closed this as completedby moving to Done in AutoDeploy Boardon Dec 4, 2025
  11. MinaHuai
    added a commit that references this issue on Dec 10, 2025

    [TRTLLM-7963][feat] Cold L2 cache when doing autotune benchmarking. (N…

    3ff5534
  12. usberkeley
    added a commit that references this issue on Dec 11, 2025

    [NVIDIA#8733][feat] Add Llama4 MoE handling to AutoDeploy (NVIDIA#9556)

    3d1f0b6
  13. codego7250
    added a commit that references this issue on Dec 11, 2025

    [NVIDIA#8733][feat] Add Llama4 MoE handling to AutoDeploy (NVIDIA#9556)

    85d9b6a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

AutoDeploy<NV> AutoDeploy Backend

Type

No type

Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions