Hi, I'm @simondong1 — MLE @ Roblox (prev Meta), CS @ CMU.
Blog: simondong1.github.io — notes on LLM internals.
- Understanding MLA Decoding — materialize vs absorb at inference
- Understanding MLA — Multi-head Latent Attention
- Understanding GQA — Grouped-Query Attention
- Understanding RoPE — Rotary Position Embeddings
X: @dongsixiang · email: simondong0919@gmail.com