this is incredible - I kinda can’t believe it’s really working! :)
ekzhang/resin t.co
206 likes · 4 reposts · 8,202 views
my laptop fans have been through a lot
resin#6 t.co
2 likes · 91 views
Resin now has tiered storage for KV cache
You can expand the GPU memory for holding KV and recurrent state with host CPU memory, or any other other tiers, with async fetches
It's "just code" so you can also do SSD, Mooncake, etc., with lookup/store/load
github.com/ekzhang/resin/blob/81b9e43f69737f109d4… t.co
106 likes · 2 reposts · 3,246 views
@ekzhang1 Hey Eric, sent these over
#9 resin#9
Render HTTP prompts on worker threads. Moved image decode/resize off the engine thread, tackling our biggest serving bottleneck: GPU idle time during page prep. This one is transferrable to resin across
#10 resin#10
Vectorized Qwen3.5 graph page tables, replacing a per-row Python loop with NumPy. Bit-identical output, tested against the old loop. Might be relevant for other runners, but I dont have GPUs to verify them on.
#11 resin#11
In-place GDN decode + one-forward MTP verification. Eliminated gather/clone/scatter for GDN state updates, making decode 2.5× faster at B=64. MTP verifies [token, draft] in one forward with a small commit kernel.
#12 resin#12
enqueue() overlap for Qwen3.5, building on #11.
Merge order: #9 and #10 are independent. Please take #11 before #12; if you squash #11, I'll rebase #12.
Validation: A100 + bf16 (chandra-ocr-2). Upstream Qwen3.5 tests match main, with new kernel tests and greedy parity checks. Haven't qualified B200/NVFP4 yet, so would appreciate your usual qualification there. Happy to adjust anything!
12 views
Show HN: Resin, fast native scheduler for LLM inference engines
1 point
TokenSpeed’s control plane was rewritten in Rust and merged into main yesterday. 👉 github.com/lightseekorg/tokenspeed/tree/main/toke…
28 likes · 3 reposts · 1,774 views
Mentions
this is incredible - I kinda can’t believe it’s really working! :) ekzhang/resin t.co
my laptop fans have been through a lot resin#6 t.co
Resin now has tiered storage for KV cache You can expand the GPU memory for holding KV and recurrent state with host CPU memory, or any other other tiers, with async fetches It's "just code" so you can also do SSD, Mooncake, etc., with lookup/store/load github.com/ekzhang/resin/blob/81b9e43f69737f109d4… t.co
@ekzhang1 Hey Eric, sent these over #9 resin#9 Render HTTP prompts on worker threads. Moved image decode/resize off the engine thread, tackling our biggest serving bottleneck: GPU idle time during page prep. This one is transferrable to resin across #10 resin#10 Vectorized Qwen3.5 graph page tables, replacing a per-row Python loop with NumPy. Bit-identical output, tested against the old loop. Might be relevant for other runners, but I dont have GPUs to verify them on. #11 resin#11 In-place GDN decode + one-forward MTP verification. Eliminated gather/clone/scatter for GDN state updates, making decode 2.5× faster at B=64. MTP verifies [token, draft] in one forward with a small commit kernel. #12 resin#12 enqueue() overlap for Qwen3.5, building on #11. Merge order: #9 and #10 are independent. Please take #11 before #12; if you squash #11, I'll rebase #12. Validation: A100 + bf16 (chandra-ocr-2). Upstream Qwen3.5 tests match main, with new kernel tests and greedy parity checks. Haven't qualified B200/NVFP4 yet, so would appreciate your usual qualification there. Happy to adjust anything!
Show HN: Resin, fast native scheduler for LLM inference engines
TokenSpeed’s control plane was rewritten in Rust and merged into main yesterday. 👉 github.com/lightseekorg/tokenspeed/tree/main/toke…