Nemotron 3.5 Super VL (120B) runs day-0 on Strix Halo, natively on Windows and Linux, ~28 tok/s with DFlash2 w/ MLPerf accuracy
NVIDIA dropped Nemotron 3.5 Super VL just today, 120B total/12B active, a hybrid Mamba-2 + LatentMoE + attention model, shipped in NVFP4. NVIDIA positions it for DGX Spark. I made sure it runs day-0 on Strix Halo via Atlas Inference, natively on Windows and on Linux compiled natively.
Numbers (Framework Desktop, Ryzen AI Max+ 395, gfx1151):
Windows 11: 18.4 tok/s without speculation, 27.5 tok/s with NVIDIA's LiLiCorr drafter (γ8), Linux (ROCm 7.13): 17.8 tok/s without speculation, 28.1 tok/s with the LiLiCorr drafter (γ8), BFCL ST-995 88.14, MLPerf Edge Agentic 0.611 across 1,007 recorded turns for BOTH!
For reference, DGX Spark on the same engine scores the same for an entirely different 128 GB box, for a 120B model lands at Spark accuracy without that explicit dtype support. The decode speeds are identical too!
Speculative decoding is also lossless. With the drafter on, output is byte-identical to plain decoding, checked on all 995 BFCL prompts. You get the speed without giving up ANY accuracy. Windows and Linux give the same answers. Their golden test prompts match byte for byte, so it's one engine, not a "Windows port" that drifts. 32K context, prefix caching on. Repeat agentic turns hit the cache, and a 31K-token prompt is tested. Same benchmarks as my MLPerf post: BFCL accuracy gates and the real recorded agentic turns, not a single-stream llama-bench.
Try it (the script builds Atlas for gfx1151, then serves an OpenAI-compatible API on 127.0.0.1:8391) Atlas-Inf/atlas:
Windows: scripts\serve-nemotron-amd.ps1 -Build
Linux: scripts/serve-nemotron-strix-linux.sh
Weights: huggingface.co and huggingface.co
Setup notes:
For windows BIOS set the iGPU allocation to 96 GB. The model needs ~80 GB of GPU memory and a fixed ~95 GB pagefile. The script checks the commit limit and tells you if it's too low.
Linux with Secure Boot: enroll the amdgpu-dkms key (one-time MOK prompt), or the GPU driver won't load.
Text-only on AMD for now. Vision runs on Spark; Strix vision is next. Sorry no prebuilt AMD binary yet. The script builds it for you QUICK, a release build is coming.
Next step is taking NVIDIA's LiLiCorr drafter with its small correlation head that re-ranks candidate tokens. We don't run it yet, which is why we get ~1.5× rather than NVIDIA's "up to 2.3×". It's the next thing I'm implementing.
Like last time, I'd love numbers and reviews from people! Discord is active if you want to help or request models:
discord.com
23 points · 9 comments
Mentions
@protect_whales @vllm_project Not I! Atlas-Inf/atlas
It's amazing to see +1 Rust-backed engine takeover :) Early 2026 Atlas came about as a pure CUDA zero dependency framework. I'd say we're worth a (re)try too. Just 30s to first token 😉 GLM5.3, Qwen3*, Nemotron3* Atlas-Inf/atlas sparkrun run @atlas/qwen3.8-27b-nvfp4
Nemotron 3.5 Super VL (120B) runs day-0 on Strix Halo, natively on Windows and Linux, ~28 tok/s with DFlash2 w/ MLPerf accuracy NVIDIA dropped Nemotron 3.5 Super VL just today, 120B total/12B active, a hybrid Mamba-2 + LatentMoE + attention model, shipped in NVFP4. NVIDIA positions it for DGX Spark. I made sure it runs day-0 on Strix Halo via Atlas Inference, natively on Windows and on Linux compiled natively. Numbers (Framework Desktop, Ryzen AI Max+ 395, gfx1151): Windows 11: 18.4 tok/s without speculation, 27.5 tok/s with NVIDIA's LiLiCorr drafter (γ8), Linux (ROCm 7.13): 17.8 tok/s without speculation, 28.1 tok/s with the LiLiCorr drafter (γ8), BFCL ST-995 88.14, MLPerf Edge Agentic 0.611 across 1,007 recorded turns for BOTH! For reference, DGX Spark on the same engine scores the same for an entirely different 128 GB box, for a 120B model lands at Spark accuracy without that explicit dtype support. The decode speeds are identical too! Speculative decoding is also lossless. With the drafter on, output is byte-identical to plain decoding, checked on all 995 BFCL prompts. You get the speed without giving up ANY accuracy. Windows and Linux give the same answers. Their golden test prompts match byte for byte, so it's one engine, not a "Windows port" that drifts. 32K context, prefix caching on. Repeat agentic turns hit the cache, and a 31K-token prompt is tested. Same benchmarks as my MLPerf post: BFCL accuracy gates and the real recorded agentic turns, not a single-stream llama-bench. Try it (the script builds Atlas for gfx1151, then serves an OpenAI-compatible API on 127.0.0.1:8391) Atlas-Inf/atlas: Windows: scripts\serve-nemotron-amd.ps1 -Build Linux: scripts/serve-nemotron-strix-linux.sh Weights: huggingface.co and huggingface.co Setup notes: For windows BIOS set the iGPU allocation to 96 GB. The model needs ~80 GB of GPU memory and a fixed ~95 GB pagefile. The script checks the commit limit and tells you if it's too low. Linux with Secure Boot: enroll the amdgpu-dkms key (one-time MOK prompt), or the GPU driver won't load. Text-only on AMD for now. Vision runs on Spark; Strix vision is next. Sorry no prebuilt AMD binary yet. The script builds it for you QUICK, a release build is coming. Next step is taking NVIDIA's LiLiCorr drafter with its small correlation head that re-ranks candidate tokens. We don't run it yet, which is why we get ~1.5× rather than NVIDIA's "up to 2.3×". It's the next thing I'm implementing. Like last time, I'd love numbers and reviews from people! Discord is active if you want to help or request models: discord.com
Day-0 support for Nemotron-3.5-Super with DFlash2 is live for DGX Spark and AMD Strix Halo (Windows/Linux) 🎊 💚 sparkrun run @atlas/nemotron-3.5-super-vl-120b-a12b-nvfp4-dflash2 ❤️ git clone Atlas-Inf/atlas && ./scripts/serve-nemotron-strix-linux.sh @NVIDIAAI @AMD t.co