ml-engineering

FYI @nvidia has announced it downgraded the performance of NVLink 6 and HBM4 speed for its upcoming Rubin products - NVLInk 6 is down to 1.5TBps from 1.8TBps (unidirectional per GPU) - HBM4 is down to 19.2TBps from 22TBps github.com/stas00/ml-engineering/blob/master/netw… github.com/stas00/ml-engineering/blob/master/comp… There are various speculations online to why this has been done, but no official explanation I was able to find.

Lots of new gpu arch additions to the MAMF/MSMF table: github.com/stas00/ml-engineering/blob/master/comp… Thank you: - Jon @HotAisle - MI300X - @JordanNanos @SemiAnalysis_ - H100+MI325X - @a_e_roberts @huggingface - MI300X+350X - Billy Cao @SemiAnalysis_ H200 NVL t.co

My fellow NVFP4 nerds. If you somehow don’t know who Stas is, that was your first mistake. Did you know he gives away all his knowledge for literally free github.com/stas00/ml-engineering/tree/master

amazing alpha here stas00/ml-engineering

To get much faster NCCL comms at small-to-medium payloads, start using symmetric memory, which has been added recently to NCCL and PyTorch. If you struggle to overlap comms and compute this could be your salvation in a few lines of code! Details github.com/stas00/ml-engineering/blob/master/netw… t.co