FlashMLA: Efficient Multi-head Latent Attention Kernels (13k stars)
What it does
All text below is quoted directly from the project README.
>FlashMLA is DeepSeek's library of optimized attention kernels, powering inference of the DeepSeek-V4.1 model on NVIDIA GPUs and Huawei Ascend NPUs.
This repo contains token-level sparse attention kernels for prefill and for decoding with an FP8 / FP4 KV cache, a fused kernel that combines Q-norm, Q-RoPE, attention, O-RoPE (conjugate) and the cast to FP8, and dense attention kernels for prefill and backward. The sparse kernels power DeepSeek Sparse Attention (DSA), as introduced in this paper.
News:
- 2026.09.30 Release of Ascend Attention Kernels: We've released sparse attention prefill and decoding kernels for the Huawei Ascend 950 NPU, which achieve up to 410 TFlops (95% hardware peak) and 360 TFlops (83% of hardware peak) during prefill and decoding, respectively.
- 2026.09.10 Release of DeepSeek v4.1's Attention Kernels: We've released attention kernels for DeepSeek-V4.1, including both prefill and decoding (with FP8 or FP4 KV cache).
- 2025.09.29 Release of Sparse Attention Kernels: With the launch of DeepSeek-V3.2, we are releasing the corresponding token-level sparse attention kernels. These kernels power the model's DeepSeek Sparse Attention (DSA) and achieve up to 640 TFlops during prefilling and 410 TFlops during decoding.
- 2025.08.01 Kernels for MHA on SM100: Thanks to NVIDIA's PR for MHA forward / backward kernels on SM100!
>Breaking change notice (2026.09.30): In the 2026.09.30 release, we removed support for the Hopper architecture and for earlier models (including DeepSeek V3 / V3.2 / V4.0), and we changed the FP8 / FP4 KV cache format. This version is therefore not compatible with previous ones.
Repository
deepseek-ai/FlashMLA
Repository stats
|Metric|Value|
|:-|:-|
|Stars|13,049|
|Forks|1,183|
|Last release|None|
|Languages|C++ 55.6%, Cuda 28.6%, Python 15.5%|
|Contributors|19|
|AI contributor accounts (Claude / Codex / Cursor / Copilot)|None|
|AI co-authored commits|None|
Main contributor
|Field|Value|
|:-|:-|
|Name|Shengyu Liu|
|GitHub|@interestingLSY|
|Commits to this repo|14 (top contributor)|
|Company / Location|@deepseek-ai, 中国，浙江省，杭州市|
|Website|interestinglsy.github.io|
|Followers|1,082|
|Public repos|31|
|On GitHub since|2017-08-05|
2 points
Mentions
FlashMLA: Efficient Multi-head Latent Attention Kernels (13k stars) What it does All text below is quoted directly from the project README. >FlashMLA is DeepSeek's library of optimized attention kernels, powering inference of the DeepSeek-V4.1 model on NVIDIA GPUs and Huawei Ascend NPUs. This repo contains token-level sparse attention kernels for prefill and for decoding with an FP8 / FP4 KV cache, a fused kernel that combines Q-norm, Q-RoPE, attention, O-RoPE (conjugate) and the cast to FP8, and dense attention kernels for prefill and backward. The sparse kernels power DeepSeek Sparse Attention (DSA), as introduced in this paper. News: - 2026.09.30 Release of Ascend Attention Kernels: We've released sparse attention prefill and decoding kernels for the Huawei Ascend 950 NPU, which achieve up to 410 TFlops (95% hardware peak) and 360 TFlops (83% of hardware peak) during prefill and decoding, respectively. - 2026.09.10 Release of DeepSeek v4.1's Attention Kernels: We've released attention kernels for DeepSeek-V4.1, including both prefill and decoding (with FP8 or FP4 KV cache). - 2025.09.29 Release of Sparse Attention Kernels: With the launch of DeepSeek-V3.2, we are releasing the corresponding token-level sparse attention kernels. These kernels power the model's DeepSeek Sparse Attention (DSA) and achieve up to 640 TFlops during prefilling and 410 TFlops during decoding. - 2025.08.01 Kernels for MHA on SM100: Thanks to NVIDIA's PR for MHA forward / backward kernels on SM100! >Breaking change notice (2026.09.30): In the 2026.09.30 release, we removed support for the Hopper architecture and for earlier models (including DeepSeek V3 / V3.2 / V4.0), and we changed the FP8 / FP4 KV cache format. This version is therefore not compatible with previous ones. Repository deepseek-ai/FlashMLA Repository stats |Metric|Value| |:-|:-| |Stars|13,049| |Forks|1,183| |Last release|None| |Languages|C++ 55.6%, Cuda 28.6%, Python 15.5%| |Contributors|19| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|None| |AI co-authored commits|None| Main contributor |Field|Value| |:-|:-| |Name|Shengyu Liu| |GitHub|@interestingLSY| |Commits to this repo|14 (top contributor)| |Company / Location|@deepseek-ai, 中国，浙江省，杭州市| |Website|interestinglsy.github.io| |Followers|1,082| |Public repos|31| |On GitHub since|2017-08-05|