Skip to content

Osimons/gdn mma prefill perf impro - #59

Open
ORippler wants to merge 7 commits into
am17an:aman/gdn-mma-prefillfrom
ORippler:osimons/gdn-mma-prefill-perf-impro
Open

ORippler wants to merge 7 commits into
am17an:aman/gdn-mma-prefillfrom
ORippler:osimons/gdn-mma-prefill-perf-impro

Conversation

@ORippler

@ORippler ORippler commented Oct 9, 2026

Copy link
Copy Markdown

Overview

10% operator (~0.5-1% E2E speedup) for chunked GDN.

image

Additional information

  1. No longer use lmem (should help older GPUs more than B4500)
  2. Wider + templated loads to get kv prefetching to work/be unrolled
  3. Hide latency of vv prefetching by value
  4. Mock m16n16k16 in mma.sync (register movement is faster than doing more iterations)

Requirements

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant