Le Chonk runs on colibri.
@MistralAI Large 4, 1T parameters, from @UnslothAI's GGUFs: text, pictures, reasoning and tools, on one 24 GB card with the experts read from NVMe. 1.34-1.36 tok/s.
colibri v2.1.0: JustVugg/colibri t.co
14 likes · 2 reposts · 589 views
colibri v2.1.0: huge open models on your own machine. CPU, Vulkan, CUDA.
Every engine runs GGUFs natively, @UnslothAI's included. New: @MistralAI Large 4 (1T), @AIatMeta Muse Spark Open, @Recursive_SI Flint, @liquidai LFM3-VL. 256k contexts, new chat UI.
github.com/JustVugg/colibri/releases/tag/v2.1.0 t.co
16 likes · 3 reposts · 800 views
Locally Run Large MOE models
This is a LLM 'compaction' backend option to try to run certain LLMs on disk on local machines. It examines the experts being run and makes many models somewhat runnable for some configurations
Thought some local folks may want to try out some bigger LLMs this way.
21 points
No GPU? Your PC Can Still Run a 744B AI Model
JustVugg/colibri
No GPU? Your PC can still run a 744B AI model.
It's colibri, a free, open-source engine (Apache-2.0) that needs no GPU. The 744B model is a 429 GB download, so colibri leaves it on your SSD, split into 13,260 experts. For each word, a router wakes only the experts it needs, about 5.4% of the model. Your most-used experts stay in RAM, so it speeds up with use, and the next ones load early. You can watch every expert light up live in the dashboard.
The speed, honestly: the big model is slow, about 1.8 tokens a second on a 128 GB machine with no GPU, and much slower on a small laptop. Smaller models are much quicker, about 6 tokens a second on a plain CPU. For the big one you need at least 16 GB of RAM and 429 GB of free disk; a fast SSD helps most.
Send this to the friend still saving up for a GPU.
Over 43,000 GitHub stars as of 26 Oct 2026; numbers from the project's README on the same date.
The link's in the description. Subscribe, and stop waiting on a GPU.
#localai #opensource #ai
182 likes · 18 comments · 6,742 views
Musím vyskúšať. Veľký model na notebooku 😉
JustVugg/colibri
6 likes
Lumabri — Run large AI models across multiple computers using distributed inference
Hi I’m working on Lumabri, a peoject that lets you connect multiple computers and use them together to run LLMs like a cluster.
The idea came from a simple question: what if you have a few computers at home that you barely use? Instead of leaving them there collecting dust, why not combine their resources to run AI models that wouldn’t fit on a single machine?
For example, if you have an old PC, a home server and your main computer, you could put them to work together and use their combined memory to load larger models without having to buy a single machine with expensive hardware.
The interesting part is being able to run models that you simply couldn’t fit on one computer before.
Lumabri is still a work in progress, but that’s the direction I’m taking it.
Here’s the repo if you want to check it out: JustVugg/lumabri
11 points · 17 comments
Lumabri — Run large AI models across multiple computers using distributed inference
Hi I’m working on Lumabri, a peoject that lets you connect multiple computers and use them together to run LLMs like a cluster.
The idea came from a simple question: what if you have a few computers at home that you barely use? Instead of leaving them there collecting dust, why not combine their resources to run AI models that wouldn’t fit on a single machine?
For example, if you have an old PC, a home server and your main computer, you could put them to work together and use their combined memory to load larger models without having to buy a single machine with expensive hardware.
The interesting part is being able to run models that you simply couldn’t fit on one computer before.
Lumabri is still a work in progress, but that’s the direction I’m taking it.
Here’s the repo if you want to check it out: JustVugg/lumabri
2 points
Show HN: Lumabri – Run large AI models across your computers and servers.Pure C
5 points
Show HN: Flint – Compact context, memory and versioned edits for coding agents
1 point
Has anyone tried Colibri? It seems to claim to run MoE models on local hardware, even huge ones like GLM-5.2 via streaming experts from disk and using ram as cache and things like that. If that's actually true then maybe you can get rather powerful models locally which would be pretty fucking good.
1 like
Mentions
Le Chonk runs on colibri. @MistralAI Large 4, 1T parameters, from @UnslothAI's GGUFs: text, pictures, reasoning and tools, on one 24 GB card with the experts read from NVMe. 1.34-1.36 tok/s. colibri v2.1.0: JustVugg/colibri t.co
colibri v2.1.0: huge open models on your own machine. CPU, Vulkan, CUDA. Every engine runs GGUFs natively, @UnslothAI's included. New: @MistralAI Large 4 (1T), @AIatMeta Muse Spark Open, @Recursive_SI Flint, @liquidai LFM3-VL. 256k contexts, new chat UI. github.com/JustVugg/colibri/releases/tag/v2.1.0 t.co
Locally Run Large MOE models This is a LLM 'compaction' backend option to try to run certain LLMs on disk on local machines. It examines the experts being run and makes many models somewhat runnable for some configurations Thought some local folks may want to try out some bigger LLMs this way.
No GPU? Your PC Can Still Run a 744B AI Model JustVugg/colibri No GPU? Your PC can still run a 744B AI model. It's colibri, a free, open-source engine (Apache-2.0) that needs no GPU. The 744B model is a 429 GB download, so colibri leaves it on your SSD, split into 13,260 experts. For each word, a router wakes only the experts it needs, about 5.4% of the model. Your most-used experts stay in RAM, so it speeds up with use, and the next ones load early. You can watch every expert light up live in the dashboard. The speed, honestly: the big model is slow, about 1.8 tokens a second on a 128 GB machine with no GPU, and much slower on a small laptop. Smaller models are much quicker, about 6 tokens a second on a plain CPU. For the big one you need at least 16 GB of RAM and 429 GB of free disk; a fast SSD helps most. Send this to the friend still saving up for a GPU. Over 43,000 GitHub stars as of 26 Oct 2026; numbers from the project's README on the same date. The link's in the description. Subscribe, and stop waiting on a GPU. #localai #opensource #ai
Musím vyskúšať. Veľký model na notebooku 😉 JustVugg/colibri
Lumabri — Run large AI models across multiple computers using distributed inference Hi I’m working on Lumabri, a peoject that lets you connect multiple computers and use them together to run LLMs like a cluster. The idea came from a simple question: what if you have a few computers at home that you barely use? Instead of leaving them there collecting dust, why not combine their resources to run AI models that wouldn’t fit on a single machine? For example, if you have an old PC, a home server and your main computer, you could put them to work together and use their combined memory to load larger models without having to buy a single machine with expensive hardware. The interesting part is being able to run models that you simply couldn’t fit on one computer before. Lumabri is still a work in progress, but that’s the direction I’m taking it. Here’s the repo if you want to check it out: JustVugg/lumabri
Lumabri — Run large AI models across multiple computers using distributed inference Hi I’m working on Lumabri, a peoject that lets you connect multiple computers and use them together to run LLMs like a cluster. The idea came from a simple question: what if you have a few computers at home that you barely use? Instead of leaving them there collecting dust, why not combine their resources to run AI models that wouldn’t fit on a single machine? For example, if you have an old PC, a home server and your main computer, you could put them to work together and use their combined memory to load larger models without having to buy a single machine with expensive hardware. The interesting part is being able to run models that you simply couldn’t fit on one computer before. Lumabri is still a work in progress, but that’s the direction I’m taking it. Here’s the repo if you want to check it out: JustVugg/lumabri
Show HN: Lumabri – Run large AI models across your computers and servers.Pure C
Show HN: Flint – Compact context, memory and versioned edits for coding agents
Has anyone tried Colibri? It seems to claim to run MoE models on local hardware, even huge ones like GLM-5.2 via streaming experts from disk and using ram as cache and things like that. If that's actually true then maybe you can get rather powerful models locally which would be pretty fucking good.