@mfranz_on @justvugg for anyone without four Pis: the same engine has Lumabri, which lets the laptops and mini PCs you already have at home share one model over the LAN, with a web chat
JustVugg/lumabri
did you print that purple case yourself?
2 likes · 84 views
Lumabri implements distributed inference in C, segmenting model layers across local machines to mitigate per-node RAM capacity limits.
github.com/JustVugg/lumabri/blob/main/README.md t.co
13 views
Lumabri — Run large AI models across multiple computers using distributed inference
Hi I’m working on Lumabri, a peoject that lets you connect multiple computers and use them together to run LLMs like a cluster.
The idea came from a simple question: what if you have a few computers at home that you barely use? Instead of leaving them there collecting dust, why not combine their resources to run AI models that wouldn’t fit on a single machine?
For example, if you have an old PC, a home server and your main computer, you could put them to work together and use their combined memory to load larger models without having to buy a single machine with expensive hardware.
The interesting part is being able to run models that you simply couldn’t fit on one computer before.
Lumabri is still a work in progress, but that’s the direction I’m taking it.
Here’s the repo if you want to check it out: JustVugg/lumabri
11 points · 17 comments
Lumabri — Run large AI models across multiple computers using distributed inference
Hi I’m working on Lumabri, a peoject that lets you connect multiple computers and use them together to run LLMs like a cluster.
The idea came from a simple question: what if you have a few computers at home that you barely use? Instead of leaving them there collecting dust, why not combine their resources to run AI models that wouldn’t fit on a single machine?
For example, if you have an old PC, a home server and your main computer, you could put them to work together and use their combined memory to load larger models without having to buy a single machine with expensive hardware.
The interesting part is being able to run models that you simply couldn’t fit on one computer before.
Lumabri is still a work in progress, but that’s the direction I’m taking it.
Here’s the repo if you want to check it out: JustVugg/lumabri
2 points
Show HN: Lumabri – Run large AI models across your computers and servers.Pure C
5 points
@mfranz_on @Raspberry_Pi Maybe you can try
JustVugg/lumabri
1 like · 46 views
Mentions
@mfranz_on @justvugg for anyone without four Pis: the same engine has Lumabri, which lets the laptops and mini PCs you already have at home share one model over the LAN, with a web chat JustVugg/lumabri did you print that purple case yourself?
Lumabri implements distributed inference in C, segmenting model layers across local machines to mitigate per-node RAM capacity limits. github.com/JustVugg/lumabri/blob/main/README.md t.co
Lumabri — Run large AI models across multiple computers using distributed inference Hi I’m working on Lumabri, a peoject that lets you connect multiple computers and use them together to run LLMs like a cluster. The idea came from a simple question: what if you have a few computers at home that you barely use? Instead of leaving them there collecting dust, why not combine their resources to run AI models that wouldn’t fit on a single machine? For example, if you have an old PC, a home server and your main computer, you could put them to work together and use their combined memory to load larger models without having to buy a single machine with expensive hardware. The interesting part is being able to run models that you simply couldn’t fit on one computer before. Lumabri is still a work in progress, but that’s the direction I’m taking it. Here’s the repo if you want to check it out: JustVugg/lumabri
Lumabri — Run large AI models across multiple computers using distributed inference Hi I’m working on Lumabri, a peoject that lets you connect multiple computers and use them together to run LLMs like a cluster. The idea came from a simple question: what if you have a few computers at home that you barely use? Instead of leaving them there collecting dust, why not combine their resources to run AI models that wouldn’t fit on a single machine? For example, if you have an old PC, a home server and your main computer, you could put them to work together and use their combined memory to load larger models without having to buy a single machine with expensive hardware. The interesting part is being able to run models that you simply couldn’t fit on one computer before. Lumabri is still a work in progress, but that’s the direction I’m taking it. Here’s the repo if you want to check it out: JustVugg/lumabri
Show HN: Lumabri – Run large AI models across your computers and servers.Pure C
@mfranz_on @Raspberry_Pi Maybe you can try JustVugg/lumabri