language model inside the language model🤔
divagr18/QwenPU
1 like · 42 views
@superalesha "Setup a neural network inside of a 4-9B Qwen model in order to run a RISC-C interpreter inside the LLM, written in C... | The neural network needs to be coherent relative to this repo (for optimal results): "divagr18/QwenPU""
c;
1 like · 56 views
做 Qwenputer 的作者又往 Qwen3.5-4B 的权重里编了一颗专门算矩阵的 NPU，这次让 Karpathy 的小模型 stories260K 在里面跑起来写故事。
矩阵乘、softmax 和注意力都是按位精确地算，小模型的权重就存在 Qwen 自己的「内存」里，全程不调外部工具。
代码和权重都放出来了：
divagr18/QwenPU
1 like · 137 views
🤯模型里又跑起了另一个模型
上次是权重里塞 RISC-V。这次同一作者把 NPU 编译进 Qwen3.5-4B，再跑 Karpathy 的 stories260K。视频里，里面的小模型在写故事。
温馨提示：实验演示，不是日常速度。你会先看完视频吗？
来源：QwenPU👇
divagr18/QwenPU t.co
5 likes · 682 views
An LLM inside an LLM - Same guy who put a RISK-V CPU inside Qwen has now made Karpathy's stories260k run on an NPU compiled inside Qwen 3.5-4B's weights
Code: divagr18/QwenPU
Weights: huggingface.co
From the original tweet:
1/3
A CPU inside an LLM is one thing, but running another LLM inside an LLM would be taking things a bit too far, wouldn't it?
Anyways, here's an LLM running inside an LLM : QwenPU, an LLM (@karpathy's stories260k) running on an NPU compiled inside Qwen 3.5-4B's weights.
2/3
stories260K is just a C program. To run it inside Qwen, I compiled an NPU into Qwen's weights that does matmuls, softmax and attention bit-exact. The LLM weights are in Qwen's own memory, with no external tools.
A lot of work went into making it fast, more on that soon.
3/3
I'm super sick right now, so the full writeup is still a work in progress, and I'm working on optimisations and size reductions too. Stay tuned, I'll finish the blog & post it soon.
305 points · 67 comments
@karpathy 3/3
I'm super sick right now, so the full writeup is still a work in progress, and I'm working on optimisations and size reductions too. Stay tuned, I'll finish the blog & post it soon.
Code and weights:
divagr18/QwenPU
huggingface.co
modelscope.ai
53 likes · 3 reposts · 2,255 views
Mentions
language model inside the language model🤔 divagr18/QwenPU
@superalesha "Setup a neural network inside of a 4-9B Qwen model in order to run a RISC-C interpreter inside the LLM, written in C... | The neural network needs to be coherent relative to this repo (for optimal results): "divagr18/QwenPU"" c;
做 Qwenputer 的作者又往 Qwen3.5-4B 的权重里编了一颗专门算矩阵的 NPU，这次让 Karpathy 的小模型 stories260K 在里面跑起来写故事。 矩阵乘、softmax 和注意力都是按位精确地算，小模型的权重就存在 Qwen 自己的「内存」里，全程不调外部工具。 代码和权重都放出来了： divagr18/QwenPU
🤯模型里又跑起了另一个模型 上次是权重里塞 RISC-V。这次同一作者把 NPU 编译进 Qwen3.5-4B，再跑 Karpathy 的 stories260K。视频里，里面的小模型在写故事。 温馨提示：实验演示，不是日常速度。你会先看完视频吗？ 来源：QwenPU👇 divagr18/QwenPU t.co
An LLM inside an LLM - Same guy who put a RISK-V CPU inside Qwen has now made Karpathy's stories260k run on an NPU compiled inside Qwen 3.5-4B's weights Code: divagr18/QwenPU Weights: huggingface.co From the original tweet: 1/3 A CPU inside an LLM is one thing, but running another LLM inside an LLM would be taking things a bit too far, wouldn't it? Anyways, here's an LLM running inside an LLM : QwenPU, an LLM (@karpathy's stories260k) running on an NPU compiled inside Qwen 3.5-4B's weights. 2/3 stories260K is just a C program. To run it inside Qwen, I compiled an NPU into Qwen's weights that does matmuls, softmax and attention bit-exact. The LLM weights are in Qwen's own memory, with no external tools. A lot of work went into making it fast, more on that soon. 3/3 I'm super sick right now, so the full writeup is still a work in progress, and I'm working on optimisations and size reductions too. Stay tuned, I'll finish the blog & post it soon.
@karpathy 3/3 I'm super sick right now, so the full writeup is still a work in progress, and I'm working on optimisations and size reductions too. Stay tuned, I'll finish the blog & post it soon. Code and weights: divagr18/QwenPU huggingface.co modelscope.ai