prabanta-dev

Janas: a C inference engine that runs MoE models bigger than your RAM by streaming experts from NVMe, plus an almost complete OpenAI-compatible server, MCP tools, vision and speech (GPL-3.0) Qwen3-Next-80B (48 GB) on a laptop with 32 GB of RAM: 26 tok/s writing, where llama.cpp manages 3.0. That is the core idea of Janas-LLM, the first component of Janas, a family of AI tools I'm writing in plain C, no dependencies beyond the C library. How it works. The weights of a mixture-of-experts model stay on disk. The experts each token needs are read from the NVMe SSD with direct I/O, and the ones that keep coming back stay in an expert cache in RAM. The engine measures the machine and picks threads, cache size and devices by itself: CPU alone, or with the GPU (Vulkan) and Intel's NPU reading the prompts. Measured on one laptop (Intel Core Ultra 9 185H, 32 GB DDR5, integrated Arc GPU and NPU, NVMe Gen4, Debian 13). Same files, same day, each engine in its best configuration, same conditions wherever possible (throughput in tok/s): |Model|Prompt (1,024 tok): Janas-LLM|llama.cpp|Reply: Janas-LLM|llama.cpp| |:-|:-|:-|:-|:-| |Qwen3.8-Flash-Next, 177B, 66.5 GB (2x the RAM)|33.0|4.82|12.3-12.5|3.12| |Qwen3-Next-80B-A3B, 48.4 GB|68.4|17.7|26.4|3.0| |Qwen3-Coder-Next, 48.4 GB|71.4|14.0|26.5|2.9| |Qwen3.6-35B-A3B|136|81.8|28.0|12.1| |Qwen3-30B-A3B|133|122|33.3|15.0| |Gemma-4-26B-A4B|85.0|78.0|18.9|12.6| |Qwen3.8-27B (dense)|31.4|25.5|4.4|2.1| |Gemma-4-12B|65.7|47.6|10.0|7.4| |Qwen3.5-9B|122|80.3|14.5|8.7| |Qwen3-4B|333 (NPU)|172|27.9|17.9| Replies are written on the CPU; prompts are read with the iGPU or the NPU where faster. The llama.cpp prompt figures are from its Vulkan backend on the same GPU; for the two models bigger than the RAM, its replies are from its CPU backend, which was faster there. Qwen3.8-Flash-Next: llama.cpp's Vulkan backend can't load it on 32 GB at all; the reply figure is from a later engine (28 Oct). NVIDIA Nemotron 3.5 Bolt (74B, Mamba layers) also runs, 6.8 tok/s on the CPU alone. Details and every caveat are in the README. Same answers as the model, not a lossy shortcut. Checked token by token against llama.cpp on the same file: the most likely token agrees at 390 of 400 positions of C code on Qwen3.8-Flash-Next and at 395 of 400 on Nemotron. MTP drafts from the model's own head make chat replies up to 63% faster on Flash-Next, with the text identical word for word. Long contexts. Qwen3-4B, CPU alone, 8-bit KV cache on both sides: at 8,192 tokens it writes at 18.2 tok/s vs 4.9, and reads at 74 vs 20. janas-server: the OpenAI API, almost all of it. Chat Completions and Responses with tools and JSON schemas (including bounds, formats and regex patterns), Conversations, Files, Uploads, Batches, Vector Stores with search, Evals, Assistants, web search, a code interpreter in its own sandbox (namespaces + seccomp, no Docker), and HTTPS. Point the official OpenAI SDK at it and it works. Vision. Qwen3.5 and Gemma 4 (E2B, E4B, 12B, 26B) describe and read pictures. The vision encoders match llama.cpp's at a mean cosine similarity of 0.99999 or better. The PNG and JPEG decoders are its own. Speech. Whisper transcription and translation, with logits matching whisper.cpp at cosine 1.000000. It decodes every format OpenAI accepts with its own decoders: WAV, FLAC, MP3, Ogg Vorbis, Opus, M4A/AAC (HE-AAC too), ALAC and WebM; on the MPEG-4 conformance streams it lands within 127-138 dB of FFmpeg. Word timestamps, Silero VAD chunking, streaming, and diarization with known speakers: on a 50-second, three-voice test, 10 of 10 turns went to the right speaker. MCP services. Nine MCP servers ship with it, usable from janas-chat or any MCP client (Claude Code included): flights, weather and sea, maps and public transport, Wikipedia, prices and finance, earthquakes, GitHub, your local git repos (writes are asked first), and the computer itself. Open data first, no keys needed for most. janas-nsm, and where this is going. The Natural Semantic Metalanguage describes any meaning with about 65 semantic primes, words every human language has (someone, something, do, want, know, good, before...). Janas is heading towards a core that works on meaning independently of the language it comes in, and NSM is the ground I'm exploring for it. janas-nsm is the first step: it runs DeepNSM (arXiv 2505.11764) to write a word's explication in NSM, scores it as the paper does, and can restrict the model to those words alone. A research tool today, but it shows where the project means to go. Also: a library with a C API, a Node.js module, FreeBASIC classes, Llama 3, and a terminal chat. Try it: prebuilt packages for Linux x86-64 (Ubuntu 22.04+, Debian 12+) and Windows 10/11 x86-64. Unpack and run janas-chat: it lists the models, downloads the one you pick (already converted, from Hugging Face) and checks it. janas-bench measures your own machine. prabanta-dev/janas It's experimental: not even alpha, developed and measured on one laptop. Numbers from other machines, bigger GPUs included, are very welcome. Maurizio "camauri" Cammalleri

Q4 flash next for £300/600 👀 - snapdragon mini PC results update - 315/43 Tok/s A3b, 46/20 Tok/s Flash Next (read/gen) 💥 Follow up to my post about the £300 Snapdragon mini PC: reddit.com Quick recap of how I got here: I started on llama.cpp, then pretty quickly moved over to Janas (github.com/prabanta-dev/janas). I ported it to ARM and contributed the work back as PRs (NEON kernels, a Windows on ARM build, and a cache bug fix Maurizio has since shipped in his own release). Working inside Janas gave me a load of ideas, so I'm now writing my own engine, codename **ToRc**. It's not released, YET, but it's what I used to produce the numbers below. Hardware: - Lenovo IdeaCentre Mini: Snapdragon X X1-26-100 (8 Oryon cores, Hexagon NPU; the Adreno GPU isn't used), 32 GB LPDDR5, the stock 512 GB NVMe (the models stream from it), Windows 11. I paid just £299.99 each, prices vary but under 400 is possible on eBay etc fairly easily and they are still under 400 at Amazon. - "Two boxes" = two of them (£600) joined by one USB4 cable. - No GPU anywhere. ToRc uses the CPU(s) cores, plus the NPU(s) for some of the dense maths. Same two boxes, all figures from today, nothing else running. The models: - **Qwen3.6 35B A3B**: bartowski `Qwen_Qwen3.6-35B-A3B-Q4_0.gguf` (19.4 GB), with its MTP head (Q4_0). - **Qwen3.8 Flash Next IQ4_NL**: ggml-org `Qwen3.8-Flash-Next-IQ4_NL` (95 GB). - **Qwen3.8 Flash Next UD-Q4_K_XL**: unsloth `Qwen3.8-Flash-Next-UD-Q4_K_XL` (104 GB). - Flash Next MTP head: bartowski `mtp-Qwen3.8-Flash-Next-Q8_0.gguf` (3.9 GB). - llama.cpp (b11700, CPU, ARM64) runs the GGUFs; Janas and ToRc run the same files converted to Janas's .jns format... I have a new file format I'm working on but that's what I used to test today 👍 The two payloads (identical for every engine): Title numbers: two boxes, read/generate on the bench payload below (a ~580-token English prose prompt, 128 tokens generated). Every engine gets byte-identical requests through its own OpenAI-style chat server (llama-server, janas-server, torc-server), timed from the client. Greedy (temperature 0), thinking off. 1. **Bench (prose, ~580 tokens in, 128 out)**: one user message: the opening of *Alice's Adventures in Wonderland* chapter I (Project Gutenberg #11, first 6 paragraphs, 2400 characters, ~580 tokens with the chat template) + "Continue the story.", 128 tokens out. **Read** = prompt tokens / time to first token (so it includes the request overhead and first token). **Write** = tokens / generation time. Each run starts with a different first line so no server gets its prompt cached from the run before. A warm-up run, then the average of two runs. Read is each engine's better configuration of the two (plain or MTP; ToRc two boxes uses 128-token prompt blocks). 2. **Real chat (code, 62 tokens in, 550-970 out)**: "Write a Python function that parses an ISO 8601 duration string like P3DT4H12M5S into total seconds, with input validation and docstring, plus 5 pytest test cases. Code only, no explanation." (62 tokens with the template), up to 1500 tokens out. Reply tok/s, then I ran the 5 tests each model wrote, plus the same 6 reference checks on every model's function (P3DT4H12M5S, PT1H30M, P1W, P2D, and two bad inputs that must raise). Qwen3.6 35B A3B Read / Write / Write + MTP: the prose bench (~580 tokens in, 128 out). Coding chat: the code payload (62 in, 550-970 out). Qwen3.6 35B A3B (read / write / write with MTP, then coding chat plain / MTP, then tests): llama.cpp, 1 box: 133.8 / 25.8 / 20.6. Coding 27.7 / 39.3. Tests 4/5, checks 6/6 llama.cpp RPC, 2 boxes: 107.1 / 24.9 / n/a. Coding 26.9. Tests 4/5, checks 6/6 Janas, 1 box: 169.5 / n/a* / 36.2. Coding 42.9 (MTP). Tests 5/5, checks 6/6 ToRc, 1 box: 228.0 / 43.4 / 49.5. Coding 42.6 / 53.3. Tests 5/5, checks 5/6 ToRc, 2 boxes: 315 / 43.3 / 43.3. Coding 42.6 / 59.6. Tests 5/5, checks 6/6 - *Janas (my update 4 build) uses the 35B file's built-in MTP layer even with `--no-mtp`, so it has no plain number: its "MTP" figures are the average of all its settled runs (bench 33.7-37.8, coding 41.3-45.4). - Engine-internal (ToRc's own bench, no server in the way): two boxes read **343.6 tok/s on a 600-token prompt**, and 349.4 on 512 tokens. The 315 in the title and table is the same engine timed from the client like every other engine, so it includes the HTTP request and the first token. - On two boxes ToRc uses both for the prompt, then pulls the state back and writes the reply on box 1 alone: generating across the cable costs a round trip per token, more than the second box saves. So plain writing is the same on one box and two (42.6 on the coding chat, byte-identical replies). With MTP the two runs wrote different replies (626 vs 968 tokens), so 53.3 vs 59.6 comes from the text, not the box. - MTP depends on the text. On the Alice prose llama.cpp's MTP was slower than plain (25.8 -> 20.6, drafts mostly rejected), ToRc one box gained 14% (43.4 -> 49.5) and two boxes gained nothing. On code they all gain: llama 27.7 -> 39.3, ToRc 42.6 -> 53.3 (one box) and 42.6 -> 59.6 (two boxes, same reply text both ways). - The tests: every engine runs the same model, but tiny numeric differences change a greedy reply, so each engine's reply is different code. llama.cpp's one failed test is the model's own arithmetic (it says P3DT4H12M5S is 300725 seconds, it's 274325), its function is right. The ToRc 5/6 is a reply that doesn't accept weeks (P1W, valid ISO 8601). And now the headline! Flash Next! Unfortunately Flash Next wouldn't run on Janas, which is why I started on my (hopefully soon to be released) engine, ToRc. Flash Next IQ4_NL is 95 GB. One box has 32 GB of RAM, two boxes 64 GB, so it never fits: the experts stream off the NVMe as they're needed. That's where the engines split apart. Read / Write: the prose bench (~580 tokens in, 128 out; ToRc two boxes with pipelined MTP, everything else plain). Coding chat: the code payload (62 in, ~750 out). Flash Next (read / write, then coding chat, then tests): llama.cpp IQ4_NL, 1 box: 7.3 / 4.2. Coding 3.2. Tests 5/5, checks 5/6 llama.cpp RPC IQ4_NL, 2 boxes: 10.9 / 5.8. Coding 5.0. Tests 5/5, checks 5/6 ToRc IQ4_NL, 1 box: 10.7 / 6.6. Coding 4.6. Tests 5/5, checks 6/6 ToRc IQ4_NL, 2 boxes: 46.2 / 20.3. Coding 19.2. Tests none written, checks 6/6 ToRc UD-Q4_K_XL, 2 boxes: 34.3 / 18.3. Coding 16.4. Tests 5/5, checks 5/6 - llama.cpp needs `--no-repack` here: on ARM it repacks IQ4_NL weights into RAM and 95 GB won't fit, so without it the model doesn't load at all. Its RPC server keeps its share of the weights in RAM (it can't stream from disk), so box 2 takes 12 of the 48 layers and box 1 maps the rest from its NVMe. - ToRc on two boxes generates across both: box 1 drafts the next tokens with the MTP head while box 2 is still finishing the current one, so the cable wait is mostly hidden. One box runs without the MTP head: with the experts coming off the SSD, MTP adds nothing there. - The Q4_K_XL is the bigger, better quant (lower perplexity than the IQ4_NL on a 3800-token Delphi code test) and costs ~10% speed. - The IQ4_NL two-box coding reply wrote a correct function but no tests, so "none written". Its function passes all 6 reference checks. Power and cost: No GPU anywhere in this: two mini PCs, £600 all in. Getting ~100 GB of model into GPU VRAM costs many times that before you've paid for the electricity. These won't beat a GPU on speed, obviously, but cost and speed... This has been an interesting little project IMHO 🙏. A 100 GB-class model at chat speed, on two small, quiet boxes that consume around 35W each at load (proper measurements to come) Anyway, this was a fun week long project and the first time I've used a snapdragon let alone tried to run an LLM on one... ToRc isn't released yet, I want it tidier and properly tested first. What's next on it: - Pipelined speculation for the 35B on two boxes (today the second box only helps with reading; it already works for Flash Next). - Flash Next on one box is limited by the SSD, not the CPU: a second NVMe with a copy of the experts might help by splitting every read across two drives. - Testing more models through the same path, maybe, but for my use cases, these two are probably enough 👍 Happy to answer questions or run something you want to see if I get the chance... Thoughts and feedback welcome 🙃👍

Conheça o Janas: execução de IA em C para computadores pessoais O que é O Janas é um projeto experimental para inteligência artificial escrito em linguagem C por Maurizio Cammalleri. Ele não possui dependências além da biblioteca padrão da linguagem. Seu primeiro componente, chamado Janas-LLM, foi projetado para executar modelos de linguagem baseados em mistura de especialistas (MoE) em computadores comuns, utilizando CPU ou GPU. O programa armazena os pesos no disco, realiza o streaming dos especialistas necessários via SSD NVMe e mantém na memória RAM os que são usados com frequência. Isso permite rodar modelos grandes mesmo em máquinas com limite de memória RAM. Para quem serve A ferramenta é destinada a desenvolvedores e entusiastas de IA que querem executar modelos grandes localmente em computadores pessoais, como notebooks com 32 GB de RAM. É compatível com sistemas Linux e Windows de arquitetura x86-64. Por ser um projeto em fase inicial de desenvolvimento, serve principalmente para testes e experimentos de desempenho. Como começar No Linux (Ubuntu 22.04, Debian 12 ou superior), faça o download do arquivo janas-linux-x86_64.tar.gz, extraia o conteúdo e execute ./janas-chat pelo terminal. No Windows 10 ou 11 (x86-64), baixe o arquivo janas-windows-x86_64.zip, descompacte a pasta e rode o comando .\janas-chat. Ao abrir o programa, você poderá selecionar um dos modelos suportados para ser baixado e verificado. Instalação curl -LO https://github.com/prabanta-dev/janas/releases/latest/download/janas-linux-x86_64.tar.gz tar xf janas-linux-x86_64.tar.gz && cd janas ./janas-chat # the models it knows: pick one, it is downloaded and checked 🔗 Repositório oficial: prabanta-dev/janas Veja em 1 minuto {% embed youtube.com %} 📚 Mais conteúdos do canal: gist.github.com

Janas 的仓库，从 SSD 流式加载专家层来跑比内存还大的模型：prabanta-dev/janas

What a £300, 35W Snapdragon mini PC can actually do 👀: 35B coder at ~30 tok/s, and an 80B model that doesnt even fit in RAM running at 17 tok/s £299.99 Lenovo IdeaCentre Mini (Snapdragon X1-26-100, 32GB, CPU only): - Qwen3.6 35B A3B coder: reads ~118 tok/s, writes 24.5, ~30 with MTP - Qwen3-Next 80B A3B: 17-18 tok/s writes with MTP How I got here... So I bought one of these boxes to test if software at work would run on ARM Windows - and then it was sitting there, but I was sufficiently impressed for the price (including taxes), that i decided in my ADHD fuelled boredom to buy one for myself to see if cheap ARM is actually usable for local models. The GPU is an Adreno X1-45 and nothing useful runs on it for LLMs yet, and so far the NPU just sits there, so this is all CPU... llama.cpp, 512 token prompt, 8 threads (read = prompt, write = generation): - coder (Q4_0 35b-a3b): read ~118, write 24.5, with MTP 29-33 - Gemma 26B: read 134, write 29.5, (need to test MTP, I've only had the box. Then I wanted to see how far I could push it. RAM is the wall on 32GB, so I went looking at the recently released Janas (github.com/prabanta-dev/janas). It keeps a RAM cache of the MoE experts and streams the rest off the NVMe, but its x86 only. So I re-coded it for ARM: NEON kernels for every quant format, i8mm, etc, (all tested bit exact against the scalar reference code, Windows ARM64 and Linux. Will submit it as PRs on GitHub soon.) Janas, same settings: - coder: read ~60, write 17.2, with MTP 29-31 - Gemma 26B: read 115, write 23.5, with MTP 32-34 - 80B: read (TBC as I don't trust my results), plain write 9.8, with MTP 17-18 llama.cpp still wins on read and plain write, but with MTP Janas matches or beats it, and it does it in a fraction of the RAM. I then squeezed the coder's expert cache to 12GB and it still wrote at 24 tok/s - so even a 16gb snapdragon could rin 35b-a3b 👀.. And this box runs an 80B that llama.cpp cant even load! Then I thought Id try connecting the two I had access to (mine and my work's dev box) together over USB4. Work in progress, will report back - but initial results are promising and I've had a 122b qwen running already - just working on streamlining my parallel experts build. I dunno about you guys and gals, I mean I don't have extensive experience with local LLMs and I certainly haven't played with ARM before but I found this pretty surprising, and not half bad for £300 with no GPU. Happy to test stuff on it when I get chance if anyone wants more numbers... But I can't see a cheaper way rn of getting 32gb of LPDDR5 rn, and only ~35ishW of power draw... Thoughts and feedback welcome 🙃👍

Anthropic left git for Scion, built for 30,000 merges a day Anthropic moved its monorepo off git and onto Scion, a version control system it built for 30,000 merges a day, mostly written by Claude. Also from Tuesday's Hacker News front page: Gemini 4 Argon and its million-token output, an eleven-year-old soundness hole in Rust's borrow checker, and Janas, which runs an 80-billion-parameter model on a 32 GB laptop. Redesigning source control at Anthropic anthropic.com Discussion: news.ycombinator.com nickg-anthropic on why one monorepo: news.ycombinator.com Gemini 4 Argon deepmind.google API pricing: ai.google.dev Discussion: news.ycombinator.com vidarh on the Try in Gemini button: news.ycombinator.com 11 year old unsoundness bugs found in Rust borrow checker rust#164250 Discussion: news.ycombinator.com theemathas on how simple the reproducer is: news.ycombinator.com Janas: Run LLMs larger than RAM by streaming experts from SSD prabanta-dev/janas Discussion: news.ycombinator.com Espressosaurus on expert caching: news.ycombinator.com The front page that day: news.ycombinator.com Follow @hacker-news on Bisque: bisque.today 0:00 Anthropic replaced git with Scion 0:30 Scion assumes every client is online 0:59 Engineers tell Claude no 1:29 Scion stays inside Anthropic 2:06 Gemini 4 Argon 2:40 Where Argon leads 3:11 An 11-year-old hole in Rust's borrow checker 3:51 Janas [ar] Anthropic تترك Git من أجل Scion، المصمم لـ 30,000 عملية دمج يومياً نقلت شركة أنثروبيك مستودعها الموحد من نظام جيت إلى نظام سايون، وهو نظام تحكم في الإصدارات صممته لمعالجة 30 ألف عملية دمج يوميًا، وقد كتب معظمه كلود. ومن بين المواضيع الأخرى التي تصدرت الصفحة الأولى لموقع هاكر نيوز يوم الثلاثاء: مشروع جيميني 4 أرجون ومخرجاته التي تبلغ مليون رمز، وثغرة أمنية عمرها أحد عشر عامًا في مدقق الاستعارات في لغة رست، ومشروع جاناس الذي يُشغّل نموذجًا يحتوي على 80 مليار مُعامل على حاسوب محمول بسعة 32 جيجابايت. ... إعادة تصميم نظام التحكم في المصادر في أنثروبيك anthropic.com نقاش: news.ycombinator.com nickg-anthropic حول سبب استخدام مستودع واحد: news.ycombinator.com جيميني 4 أرجون deepmind.google أسعار واجهة برمجة التطبيقات: ai.google.dev نقاش: news.ycombinator.com vidarh حول زر "جرب في جيميني": news.ycombinator.com اكتشاف أخطاء برمجية عمرها 11 عامًا في استعارة Rust مدقق rust#164250 نقاش: news.ycombinator.com theemathas حول سهولة إعادة إنتاج المشكلة: news.ycombinator.com Janas: تشغيل أنظمة إدارة التعلم (LLMs) بحجم أكبر من ذاكرة الوصول العشوائي (RAM) عن طريق بث الخبراء من وحدة تخزين SSD prabanta-dev/janas نقاش: news.ycombinator.com Espressosaurus حول التخزين المؤقت للخبراء: news.ycombinator.com الصفحة الرئيسية في ذلك اليوم: news.ycombinator.com تابع @hacker-news على بيسك: bisque.today 0:00 استبدلت أنثروبيك نظام جيت بسايون 0:30 يفترض سايون أن جميع العملاء متصلون بالإنترنت 0:59 المهندسون يرفضون طلب كلود 1:29 يبقى سايون ضمن أنثروبيك 2:06 جيميني 4 أرجون 2:40 إلى أين يقود أرجون 3:11 ثغرة عمرها 11 عامًا في مدقق الاستعارة في رست 3:51 جاناس [bn] অ্যানথ্রোপিক গিট ছেড়ে সায়োন-এ যোগ দিয়েছেন, যা দিনে ৩০,০০০ মার্জ করার জন্য তৈরি। অ্যানথ্রোপিক তার মোনোরেপো গিট থেকে সরিয়ে সায়োন-এ নিয়ে গেছে। সায়োন হলো একটি ভার্সন কন্ট্রোল সিস্টেম যা তারা দিনে ৩০,০০০ মার্জ করার জন্য তৈরি করেছে, যার বেশিরভাগই লিখেছেন ক্লদ। এছাড়াও মঙ্গলবারের হ্যাকার নিউজের প্রথম পাতা থেকে: জেমিনি ৪ আর্গন এবং এর দশ লক্ষ টোকেনের আউটপুট, রাস্টের বরো চেকারে এগারো বছরের পুরনো একটি সাউন্ডনেস হোল, এবং জানাস, যা একটি ৩২ জিবি ল্যাপটপে ৮০ বিলিয়ন প্যারামিটারের একটি মডেল চালায়। অ্যানথ্রোপিকে সোর্স কন্ট্রোলের নতুন ডিজাইন anthropic.com আলোচনা: news.ycombinator.com নিকজি-অ্যানথ্রোপিকের মতে কেন একটি মনোরেপো: news.ycombinator.com জেমিনি ৪ আর্গন deepmind.google এপিআই মূল্য: ai.google.dev আলোচনা: news.ycombinator.com ভিডার্হের মতে 'ট্রাই ইন জেমিনি' বাটন: news.ycombinator.com রাস্ট বোরোতে ১১ বছরের পুরনো ত্রুটিপূর্ণ বাগ পাওয়া গেছে চেকার rust#164250 আলোচনা: news.ycombinator.com প্রডিউসারটি কতটা সহজ সে সম্পর্কে দিমাথাস: news.ycombinator.com জানাস: এসএসডি থেকে এক্সপার্ট স্ট্রিমিং করে র‍্যামের চেয়ে বড় এলএলএম চালান prabanta-dev/janas আলোচনা: news.ycombinator.com এক্সপার্ট ক্যাশিং সম্পর্কে এসপ্রেসোসরাস: news.ycombinator.com সেদিনের প্রথম পাতা: news.ycombinator.com বিস্ক-এ @hacker-news-কে অনুসরণ করুন: bisque.today ০:০০ অ্যানথ্রোপিক গিট-কে সায়োন দিয়ে প্রতিস্থাপন করেছে ০:৩০ সায়োন ধরে নেয় যে প্রতিটি ক্লায়েন্ট অনলাইন আছে ০:৫৯ ইঞ্জিনিয়াররা ক্লদকে না বলে ১:২৯ সায়োন অ্যানথ্রোপিকের ভেতরেই থেকে যায় ২:০৬ জেমিনি ৪ আর্গন ২:৪০ আর্গন কোথায় নিয়ে যায় ৩:১১ রাস্ট-এর বরো চেকারে ১১ বছরের পুরনো একটি ফাঁক ৩:৫১ জানাস [de-DE] Anthropic hat Git für Scion verlassen, entwickelt für 30.000 Merges am Tag Anthropic hat sein Monorepo von Git auf Scion umgestellt, ein Versionskontrollsystem, das für 30.000 Merges pro Tag ausgelegt ist und größtenteils von Claude geschrieben wurde. Ebenfalls auf der Hacker News-Startseite vom Dienstag: Gemini 4 Argon mit seiner Ausgabemenge von einer Million Token, eine elf Jahre alte Sicherheitslücke im Borrow-Checker von Rust und Janas, das ein Modell mit 80 Milliarden Parametern auf einem Laptop mit nur 32 GB Speicher ausführt. Neugestaltung der Quellcodeverwaltung bei Anthropic anthropic.com Diskussion: news.ycombinator.com nickg-anthropic erklärt, warum ein Monorepo sinnvoll ist: news.ycombinator.com Gemini 4 Argon deepmind.google API-Preise: ai.google.dev Diskussion: news.ycombinator.com vidarh über den „In Gemini testen“-Button: news.ycombinator.com 11 Jahre alte Sicherheitslücken in Rust Borrow entdeckt Checker rust#164250 Diskussion: news.ycombinator.com theemathas zur Einfachheit der Reproduktion: news.ycombinator.com Janas: Ausführen von LLMs, die größer als der RAM sind, durch Streaming von Experten von SSD prabanta-dev/janas Diskussion: news.ycombinator.com Espressosaurus zum Experten-Caching: news.ycombinator.com Die Titelseite an diesem Tag: news.ycombinator.com Folge @hacker-news auf Bisque: bisque.today 0:00 Anthropic ersetzt Git durch Scion 0:30 Scion geht davon aus, dass jeder Client online ist 0:59 Entwickler sagen Claude Nein 1:29 Scion bleibt in Anthropic integriert 2:06 Gemini 4 Argon 2:40 Wohin Argon führt 3:11 Eine 11 Jahre alte Sicherheitslücke in Rusts Borrow-Checker 3:51 Janas [es-US] Anthropic dejó Git para pasarse a Scion, diseñado para 30.000 fusiones al día. Anthropic migró su monorepo de Git a Scion, un sistema de control de versiones que desarrolló para gestionar 30 000 fusiones diarias, escrito principalmente por Claude. También en la portada de Hacker News del martes: Gemini 4 Argon y su salida de un millón de tokens, una vulnerabilidad de once años en el verificador de préstamos de Rust y Janas, que ejecuta un modelo de 80 000 millones de parámetros en un portátil de 32 GB. Rediseño del control de versiones en Anthropic anthropic.com Discusión: news.ycombinator.com nickg-anthropic explica por qué usar un monorepo: news.ycombinator.com Gemini 4 Argon deepmind.google Precios de la API: ai.google.dev Discusión: news.ycombinator.com vidarh comenta sobre el botón "Probar en Gemini": news.ycombinator.com Se descubren errores de seguridad de hace 11 años en Rust borrow Comprobador rust#164250 Discusión: news.ycombinator.com theemathas comenta sobre la simplicidad del reproducible: news.ycombinator.com Janas: Ejecutar LLMs más grandes que la RAM mediante la transmisión de expertos desde SSD prabanta-dev/janas Discusión: news.ycombinator.com Espressosaurus sobre el almacenamiento en caché de expertos: news.ycombinator.com Portada de ese día: news.ycombinator.com Sigue a @hacker-news en Bisque: bisque.today 0:00 Anthropic reemplazó Git con Scion 0:30 Scion asume que todos los clientes están en línea 0:59 Los ingenieros le dicen que no a Claude 1:29 Scion permanece dentro de Anthropic 2:06 Gemini 4 Argon 2:40 Hacia dónde conduce Argon 3:11 Un fallo de seguridad de 11 años en el verificador de préstamos de Rust 3:51 Janas [fr-FR] Anthropic a abandonné Git pour Scion, conçu pour 30 000 fusions par jour Anthropic a migré son monorepo de Git vers Scion, un système de contrôle de version conçu pour gérer 30 000 fusions par jour, principalement développé par Claude. Également à la une de Hacker News mardi : Gemini 4 Argon et sa production d'un million de jetons, une faille de sécurité vieille de onze ans dans le vérificateur d'emprunts de Rust, et Janas, qui exécute un modèle de 80 milliards de paramètres sur un ordinateur portable de 32 Go. Refonte du contrôle de version chez Anthropic anthropic.com Discussion : news.ycombinator.com nickg-anthropic explique l’intérêt d’un monorepo : news.ycombinator.com Gemini 4 Argon deepmind.google Tarification de l’API : ai.google.dev Discussion : news.ycombinator.com vidarh parle du bouton « Essayer dans Gemini » : news.ycombinator.com Découverte de bogues de sécurité vieux de 11 ans dans Rust Borrow Vérificateur rust#164250 Discussion : news.ycombinator.com theemathas explique la simplicité du reproductible : news.ycombinator.com Janas : Exécution de LLM plus volumineux que la RAM en diffusant les experts depuis un SSD prabanta-dev/janas Discussion : news.ycombinator.com Espressosaurus parle de la mise en cache des experts : news.ycombinator.com À la une ce jour-là : news.ycombinator.com Suivez @hacker-news sur Bisque : bisque.today 0:00 Anthropic remplace Git par Scion 0:30 Scion suppose que tous les clients sont en ligne 0:59 Les ingénieurs refusent l'idée à Claude 1:29 Scion reste intégré à Anthropic 2:06 Gemini 4 Argon 2:40 Où mène Argon ? 3:11 Une faille de 11 ans dans le vérificateur d'emprunts de Rust 3:51 Janas [hi] Anthropic ने Git छोड़कर Scion अपनाया, जिसे प्रति दिन 30,000 मर्ज के लिए बनाया गया है एन्थ्रोपिक ने अपने मोनोरेपो को गिट से हटाकर सियोन पर स्थानांतरित कर दिया है, जो एक संस्करण नियंत्रण प्रणाली है जिसे कंपनी ने प्रतिदिन 30,000 मर्ज के लिए बनाया है और जिसका अधिकांश भाग क्लाउड द्वारा लिखा गया है। मंगलवार के हैकर न्यूज़ के मुख्य पृष्ठ से अन्य समाचार: जेमिनी 4 आर्गन और इसका दस लाख टोकन आउटपुट, रस्ट के बरो चेकर में ग्यारह साल पुरानी खामी, और जनास, जो 32 जीबी लैपटॉप पर 80 अरब पैरामीटर वाला मॉडल चलाता है। एन्थ्रोपिक में सोर्स कंट्रोल का पुनर्रचना anthropic.com चर्चा: news.ycombinator.com nickg-anthropic द्वारा एक मोनोरेपो की आवश्यकता पर चर्चा: news.ycombinator.com जेमिनी 4 आर्गन deepmind.google एपीआई मूल्य निर्धारण: ai.google.dev चर्चा: news.ycombinator.com vidarh द्वारा जेमिनी में 'ट्राई इन जेमिनी' बटन पर चर्चा: news.ycombinator.com रस्ट बरो में 11 साल पुराने बग पाए गए चेकर rust#164250 चर्चा: news.ycombinator.com थीमाथास द्वारा पुनरुत्पादन की सरलता पर टिप्पणी: news.ycombinator.com जनास: एसएसडी से स्ट्रीमिंग एक्सपर्ट्स का उपयोग करके रैम से बड़े एलएलएम चलाएं prabanta-dev/janas चर्चा: news.ycombinator.com एस्प्रेसोसॉरस द्वारा एक्सपर्ट कैशिंग पर टिप्पणी: news.ycombinator.com उस दिन का मुख्य पृष्ठ: news.ycombinator.com बिस्क पर @hacker-news को फॉलो करें: bisque.today 0:00 एंथ्रोपिक ने गिट की जगह सियोन का इस्तेमाल किया 0:30 सियोन मानता है कि हर क्लाइंट ऑनलाइन है 0:59 इंजीनियरों ने क्लाउड को मना किया 1:29 सियोन एंथ्रोपिक के अंदर ही रहा 2:06 जेमिनी 4 आर्गन 2:40 आर्गन का परिणाम 3:11 रस्ट के बरो चेकर में 11 साल पुरानी खामी 3:51 जनास [id] Anthropic meninggalkan git demi Scion, dibangun untuk 30.000 penggabungan per hari Anthropic memindahkan monorepo-nya dari Git ke Scion, sebuah sistem kontrol versi yang dibangunnya untuk menangani 30.000 penggabungan per hari, yang sebagian besar ditulis oleh Claude. Juga dari halaman depan Hacker News hari Selasa: Gemini 4 Argon dan output jutaan tokennya, celah kebenaran sebelas tahun dalam pemeriksa peminjaman Rust, dan Janas, yang menjalankan model 80 miliar parameter pada laptop 32 GB. Mendesain Ulang Kontrol Sumber di Anthropic anthropic.com Diskusi: news.ycombinator.com nickg-anthropic tentang mengapa satu monorepo: news.ycombinator.com Gemini 4 Argon deepmind.google Harga API: ai.google.dev Diskusi: news.ycombinator.com vidarh tentang tombol Coba di Gemini: news.ycombinator.com Bug ketidakstabilan berusia 11 tahun ditemukan di Rust borrow pemeriksa rust#164250 Diskusi: news.ycombinator.com theemathas tentang betapa sederhananya reproduksi masalah ini: news.ycombinator.com Janas: Menjalankan LLM yang lebih besar dari RAM dengan streaming expert dari SSD prabanta-dev/janas Diskusi: news.ycombinator.com Espressosaurus tentang caching expert: news.ycombinator.com Halaman depan hari itu: news.ycombinator.com Ikuti @hacker-news di Bisque: bisque.today 0:00 Anthropic mengganti git dengan Scion 0:30 Scion menganggap setiap klien online 0:59 Para insinyur mengatakan tidak kepada Claude 1:29 Scion tetap berada di dalam Anthropic 2:06 Gemini 4 Argon 2:40 Ke mana Argon mengarah 3:11 Sebuah lubang berusia 11 tahun di pemeriksa peminjaman Rust 3:51 Janas [it] Anthropic ha abbandonato Git per Scion, creato per gestire 30.000 merge al giorno Anthropic ha spostato il suo monorepo da Git a Scion, un sistema di controllo versione progettato per gestire 30.000 merge al giorno, scritto principalmente da Claude. Tra le altre notizie dalla prima pagina di Hacker News di martedì: Gemini 4 Argon e la sua capacità di generare un milione di token, una falla di sicurezza di undici anni nel borrow checker di Rust e Janas, che esegue un modello con 80 miliardi di parametri su un laptop da 32 GB. Riprogettazione del controllo di versione in Anthropic anthropic.com Discussione: news.ycombinator.com nickg-anthropic sul perché di un monorepo: news.ycombinator.com Gemini 4 Argon deepmind.google Prezzi delle API: ai.google.dev Discussione: news.ycombinator.com vidarh sul pulsante "Prova in Gemini": news.ycombinator.com Bug di instabilità di 11 anni fa trovati in Rust borrow checker rust#164250 Discussione: news.ycombinator.com theemathas spiega quanto sia semplice riprodurre il problema: news.ycombinator.com Janas: Eseguire LLM più grandi della RAM tramite lo streaming degli expert da SSD prabanta-dev/janas Discussione: news.ycombinator.com Espressosaurus parla della cache degli expert: news.ycombinator.com La prima pagina di quel giorno: news.ycombinator.com Segui @hacker-news su Bisque: bisque.today 0:00 Anthropic ha sostituito git con Scion 0:30 Scion presuppone che ogni client sia online 0:59 Gli ingegneri dicono di no a Claude 1:29 Scion rimane all'interno di Anthropic 2:06 Gemini 4 Argon 2:40 Dove porta Argon 3:11 Una falla vecchia di 11 anni nel borrow checker di Rust 3:51 Janas [iw] Anthropic עזבה את Git עבור Scion, שנבנתה ל-30,000 מיזוגים ביום חברת Anthropic העבירה את המונו-ריפו שלה מ-git אל Scion, מערכת בקרת גרסאות שבנתה עבור 30,000 מיזוגים ביום, שרובם נכתבו על ידי קלוד. כמו כן, מעמוד השער של Hacker News ביום שלישי: Gemini 4 Argon ופלט מיליון האסימונים שלו, חור תקין בן אחת עשרה שנים בבודק ההלוואה של Rust, ו-Janas, שמפעיל מודל של 80 מיליארד פרמטרים על מחשב נייד בנפח 32 ג'יגה-בייט. עיצוב מחדש של בקרת מקור ב-Anthropic anthropic.com דיון: news.ycombinator.com nickg-anthropic על למה להשתמש ב-monorepo אחד: news.ycombinator.com Gemini 4 Argon deepmind.google תמחור API: ai.google.dev דיון: news.ycombinator.com vidarh על כפתור "נסה ב-Gemini": news.ycombinator.com באגים לא תקינים בני 11 שנים נמצאו ב-Rust borrow בודק rust#164250 דיון: news.ycombinator.com theemathas על כמה פשוט המשחזר: news.ycombinator.com Janas: הפעלת LLMs גדולים יותר מ-RAM על ידי הזרמת מומחים מ-SSD prabanta-dev/janas דיון: news.ycombinator.com Espressosaurus על אחסון מטמון מומחים: news.ycombinator.com העמוד הראשון באותו יום: news.ycombinator.com עקבו אחרי @hacker-news ב-Bisque: bisque.today 0:00 אנתרופיק החליפה את git ב-Scion 0:30 Scion מניחה שכל לקוח מחובר 0:59 מהנדסים אומרים לקלוד שלא 1:29 Scion נשארת בתוך אנתרופיק 2:06 ג'מיני 4 ארגון 2:40 לאן ארגון מוביל 3:11 חור בן 11 שנים בכרטיס ההלוואה של ראסט 3:51 יאנאס [ja] AnthropicがGitからScionへ移行―1日3万回のマージに対応する独自システム Anthropicは、モノレポをGitからScionに移行しました。Scionは、1日に3万件のマージを処理するためにAnthropicが構築したバージョン管理システムで、主にClaudeが開発しました。火曜日のHacker Newsのトップページには、Gemini 4 Argonとその100万トークンの出力、Rustの借用チェッカーにおける11年前から存在する健全性に関する脆弱性、そして32GBのノートパソコンで800億個のパラメータを持つモデルを実行するJanasに関する記事も掲載されていました。 Anthropicにおけるソースコード管理の再設計 anthropic.com ディスカッション：news.ycombinator.com nickg-anthropicによるモノレポのメリットに関する解説：news.ycombinator.com Gemini 4 Argon deepmind.google API料金：ai.google.dev ディスカッション：news.ycombinator.com vidarhによる「Geminiで試す」ボタンに関する解説：news.ycombinator.com Rustの借用コードに11年前の不具合が発見されるチェッカー rust#164250 議論: news.ycombinator.com 再現手順の簡便さに関するtheemathas氏の解説: news.ycombinator.com Janas: SSDからエキスパートをストリーミングすることで、RAM容量を超えるLLMを実行 prabanta-dev/janas 議論: news.ycombinator.com エキスパートキャッシュに関するEspressosaurus氏の解説: news.ycombinator.com その日のトップページ: news.ycombinator.com @hacker-news をフォローBisque: bisque.today 0:00 AnthropicがgitをScionに置き換え 0:30 Scionはすべてのクライアントがオンラインであると想定 0:59 エンジニアがClaudeに拒否 1:29 ScionはAnthropic内部に留まる 2:06 Gemini 4 Argon 2:40 Argonの進むべき道 3:11 Rustの借用チェッカーにおける11年前の脆弱性 3:51 Janas [ko] Anthropic은 하루 3만 번의 병합을 처리하기 위해 Git을 떠나 Scion을 구축했습니다 Anthropic은 모노레포를 Git에서 Scion으로 옮겼습니다. Scion은 하루 3만 건의 병합을 처리할 수 있도록 자체 개발한 버전 관리 시스템으로, 대부분 Claude가 작성했습니다. 화요일 Hacker News 첫 페이지에는 Gemini 4 Argon과 백만 개 토큰 출력, Rust의 borrow checker에서 11년 전에 발견된 건전성 취약점, 그리고 32GB 노트북에서 800억 개 매개변수 모델을 실행하는 Janas에 대한 소식도 실렸습니다. Anthropic의 소스 코드 관리 시스템 재설계 anthropic.com 토론: news.ycombinator.com nickg-anthropic의 모노레포 사용 이유: news.ycombinator.com Gemini 4 Argon deepmind.google API 가격: ai.google.dev 토론: news.ycombinator.com vidarh의 Gemini에서 테스트하기 버튼에 대한 의견: news.ycombinator.com Rust borrow에서 11년 된 불안정성 버그 발견 검사기 rust#164250 토론: news.ycombinator.com theemathas의 재현 방법 설명: news.ycombinator.com Janas: SSD에서 전문가를 스트리밍하여 RAM보다 큰 LLM 실행 prabanta-dev/janas 토론: news.ycombinator.com Espressosaurus의 전문가 캐싱 설명: news.ycombinator.com 당일 메인 페이지: news.ycombinator.com Bisque에서 @hacker-news 팔로우하기: bisque.today 0:00 Anthropic이 git을 Scion으로 대체했습니다 0:30 Scion은 모든 클라이언트가 온라인 상태라고 가정합니다 0:59 엔지니어들이 Claude에게 안 된다고 말합니다 1:29 Scion은 Anthropic 내부에 남아 있습니다 2:06 Gemini 4 Argon 2:40 Argon이 이끄는 방향 3:11 Rust의 borrow checker에 있는 11년 된 허점 3:51 Janas [ml] Anthropic ഗിറ്റ് ഉപേക്ഷിച്ച് Scion-ലേക്ക് മാറി, ദിവസേന 30,000 മെർജുകൾക്കായി നിർമ്മിച്ചത് ആന്ത്രോപിക് അതിന്റെ മോണോറെപ്പോയെ ഗിറ്റിൽ നിന്ന് മാറ്റി, ഒരു ദിവസം 30,000 ലയനങ്ങൾക്കായി നിർമ്മിച്ച ഒരു പതിപ്പ് നിയന്ത്രണ സംവിധാനമായ സിയോണിലേക്ക് മാറ്റി, കൂടുതലും ക്ലോഡ് എഴുതിയതാണ്. ചൊവ്വാഴ്ചത്തെ ഹാക്കർ ന്യൂസിന്റെ ഒന്നാം പേജിൽ നിന്നും: ജെമിനി 4 ആർഗണും അതിന്റെ ദശലക്ഷം ടോക്കൺ ഔട്ട്‌പുട്ടും, റസ്റ്റിന്റെ കടം വാങ്ങൽ ചെക്കറിലെ പതിനൊന്ന് വർഷം പഴക്കമുള്ള സൗണ്ട്‌നെസ് ഹോളും, 32 ജിബി ലാപ്‌ടോപ്പിൽ 80 ബില്യൺ പാരാമീറ്റർ മോഡൽ പ്രവർത്തിപ്പിക്കുന്ന ജാനസും. ആന്ത്രോപിക്കിൽ സോഴ്‌സ് നിയന്ത്രണം പുനർരൂപകൽപ്പന ചെയ്യുന്നു anthropic.com ചർച്ച: news.ycombinator.com ഒരു മോണോറെപ്പോ എന്തിനാണെന്ന് നിക്ക്-ആന്ത്രോപിക്: news.ycombinator.com ജെമിനി 4 ആർഗോൺ deepmind.google API വിലനിർണ്ണയം: ai.google.dev ചർച്ച: news.ycombinator.com ജെമിനിയിൽ പരീക്ഷിക്കുക ബട്ടണിലെ vidarh: news.ycombinator.com റസ്റ്റിൽ 11 വയസ്സുള്ള അൺസൗണ്ട്നെസ് ബഗുകൾ കണ്ടെത്തി കടം വാങ്ങുക ചെക്കർ rust#164250 ചർച്ച: news.ycombinator.com പുനരുൽപ്പാദകൻ എത്ര ലളിതമാണെന്ന് തീമത്ത്: news.ycombinator.com ജനാസ്: SSD-യിൽ നിന്നുള്ള സ്ട്രീമിംഗ് വിദഗ്ദ്ധർ വഴി റാമിനേക്കാൾ വലിയ LLM-കൾ പ്രവർത്തിപ്പിക്കുക prabanta-dev/janas ചർച്ച: news.ycombinator.com വിദഗ്ദ്ധ കാഷിംഗിൽ എസ്പ്രസ്സോറസ്: news.ycombinator.com ആ ദിവസത്തെ ഒന്നാം പേജ്: news.ycombinator.com ബിസ്കിൽ @hacker-news പിന്തുടരുക: bisque.today 0:00 ആന്ത്രോപിക് ജിറ്റിനെ സിയോൺ ഉപയോഗിച്ച് മാറ്റിസ്ഥാപിച്ചു 0:30 എല്ലാ ക്ലയന്റുകളും ഓൺലൈനിലാണെന്ന് സിയോൺ അനുമാനിക്കുന്നു 0:59 എഞ്ചിനീയർമാർ ക്ലോഡിനോട് ഇല്ല എന്ന് പറയുന്നു 1:29 സിയോൺ ആന്ത്രോപിക്കിനുള്ളിൽ തന്നെ തുടരുന്നു 2:06 ജെമിനി 4 ആർഗോൺ 2:40 ആർഗോൺ എവിടെയാണ് നയിക്കുന്നത് 3:11 റസ്റ്റിന്റെ കടം വാങ്ങൽ ചെക്കറിൽ 11 വയസ്സുള്ള ഒരു ദ്വാരം 3:51 ജനാസ് [nl-NL] Anthropic verruilt git voor Scion, gebouwd voor 30.000 merges per dag Anthropic heeft zijn monorepo van Git naar Scion verplaatst, een versiebeheersysteem dat ze zelf hebben ontwikkeld voor 30.000 merges per dag, grotendeels geschreven door Claude. Ook op de voorpagina van Hacker News van dinsdag: Gemini 4 Argon en de output van een miljoen tokens, een elf jaar oud beveiligingslek in de borrow checker van Rust, en Janas, dat een model met 80 miljard parameters draait op een laptop met 32 ​​GB geheugen. Het herontwerpen van versiebeheer bij Anthropic anthropic.com Discussie: news.ycombinator.com nickg-anthropic over waarom één monorepo: news.ycombinator.com Gemini 4 Argon deepmind.google API-prijzen: ai.google.dev Discussie: news.ycombinator.com vidarh over de 'Probeer in Gemini'-knop: news.ycombinator.com 11 jaar oude bugs gevonden in Rust borrow checker rust#164250 Discussie: news.ycombinator.com theemathas over hoe eenvoudig het reproduceerbaar is: news.ycombinator.com Janas: Voer LLM's groter dan RAM uit door experts vanaf SSD te streamen prabanta-dev/janas Discussie: news.ycombinator.com Espressosaurus over het cachen van experts: news.ycombinator.com De voorpagina van die dag: news.ycombinator.com Volg @hacker-news op Bisque: bisque.today 0:00 Anthropic verving git door Scion 0:30 Scion gaat ervan uit dat elke client online is 0:59 Engineers zeggen nee tegen Claude 1:29 Scion blijft binnen Anthropic 2:06 Gemini 4 Argon 2:40 Waar Argon toe leidt 3:11 Een 11 jaar oud gat in Rusts borrow checker 3:51 Janas [pa] Anthropic ਨੇ 30,000 ਮਰਜ ਪ੍ਰਤੀ ਦਿਨ ਲਈ ਬਣਾਏ ਗਏ Scion ਲਈ Git ਨੂੰ ਛੱਡ ਦਿੱਤਾ ਐਂਥ੍ਰੋਪਿਕ ਨੇ ਆਪਣੇ ਮੋਨੋਰੇਪੋ ਨੂੰ ਗਿੱਟ ਤੋਂ ਹਟਾ ਕੇ ਸਾਇਓਨ 'ਤੇ ਤਬਦੀਲ ਕਰ ਦਿੱਤਾ, ਇੱਕ ਵਰਜਨ ਕੰਟਰੋਲ ਸਿਸਟਮ ਜੋ ਇਸਨੇ ਇੱਕ ਦਿਨ ਵਿੱਚ 30,000 ਮਰਜ ਲਈ ਬਣਾਇਆ ਸੀ, ਜ਼ਿਆਦਾਤਰ ਕਲੌਡ ਦੁਆਰਾ ਲਿਖਿਆ ਗਿਆ ਸੀ। ਮੰਗਲਵਾਰ ਦੇ ਹੈਕਰ ਨਿਊਜ਼ ਦੇ ਪਹਿਲੇ ਪੰਨੇ ਤੋਂ ਵੀ: ਜੈਮਿਨੀ 4 ਆਰਗਨ ਅਤੇ ਇਸਦੇ ਮਿਲੀਅਨ-ਟੋਕਨ ਆਉਟਪੁੱਟ, ਰਸਟ ਦੇ ਉਧਾਰ ਚੈਕਰ ਵਿੱਚ ਇੱਕ ਗਿਆਰਾਂ ਸਾਲ ਪੁਰਾਣਾ ਸਾਊਂਡਨੇਸ ਹੋਲ, ਅਤੇ ਜਾਨਸ, ਜੋ ਕਿ ਇੱਕ 32 GB ਲੈਪਟਾਪ 'ਤੇ 80-ਬਿਲੀਅਨ-ਪੈਰਾਮੀਟਰ ਮਾਡਲ ਚਲਾਉਂਦਾ ਹੈ। ਐਂਥ੍ਰੋਪਿਕ 'ਤੇ ਸਰੋਤ ਨਿਯੰਤਰਣ ਨੂੰ ਮੁੜ ਡਿਜ਼ਾਈਨ ਕਰਨਾ anthropic.com ਚਰਚਾ: news.ycombinator.com ਇੱਕ ਮੋਨੋਰੇਪੋ ਕਿਉਂ ਹੈ ਇਸ ਬਾਰੇ nickg-anthropic: news.ycombinator.com Gemini 4 Argon deepmind.google API ਕੀਮਤ: ai.google.dev ਚਰਚਾ: news.ycombinator.com ਟਰਾਈ ਇਨ ਜੈਮਿਨੀ ਬਟਨ 'ਤੇ vidarh: news.ycombinator.com Rust borrow ਵਿੱਚ ਮਿਲੇ 11 ਸਾਲ ਪੁਰਾਣੇ ਬੇਆਰਾਮੀ ਵਾਲੇ ਬੱਗ ਚੈਕਰ rust#164250 ਚਰਚਾ: news.ycombinator.com ਪ੍ਰਜਨਨਕਰਤਾ ਕਿੰਨਾ ਸਰਲ ਹੈ ਇਸ ਬਾਰੇ ਵਿਚਾਰ: news.ycombinator.com ਜਾਨਸ: SSD ਤੋਂ ਸਟ੍ਰੀਮਿੰਗ ਮਾਹਿਰਾਂ ਦੁਆਰਾ RAM ਤੋਂ ਵੱਡੇ LLM ਚਲਾਓ prabanta-dev/janas ਚਰਚਾ: news.ycombinator.com ਮਾਹਰ ਕੈਸ਼ਿੰਗ 'ਤੇ ਐਸਪ੍ਰੈਸੋਸੋਰਸ: news.ycombinator.com ਉਸ ਦਿਨ ਦਾ ਮੁੱਖ ਪੰਨਾ: news.ycombinator.com ਬਿਸਕ 'ਤੇ @hacker-news ਨੂੰ ਫਾਲੋ ਕਰੋ: bisque.today 0:00 ਐਂਥ੍ਰੋਪਿਕ ਨੇ ਗਿੱਟ ਨੂੰ ਸਾਇਓਨ ਨਾਲ ਬਦਲ ਦਿੱਤਾ 0:30 ਸਾਇਓਨ ਮੰਨਦਾ ਹੈ ਕਿ ਹਰ ਕਲਾਇੰਟ ਔਨਲਾਈਨ ਹੈ 0:59 ਇੰਜੀਨੀਅਰ ਕਲੌਡ ਨੂੰ ਨਹੀਂ ਦੱਸਦੇ 1:29 ਸਾਇਓਨ ਐਂਥ੍ਰੋਪਿਕ ਦੇ ਅੰਦਰ ਰਹਿੰਦਾ ਹੈ 2:06 ਜੇਮਿਨੀ 4 ਆਰਗਨ 2:40 ਜਿੱਥੇ ਆਰਗਨ ਅਗਵਾਈ ਕਰਦਾ ਹੈ 3:11 ਰਸਟ ਦੇ ਉਧਾਰ ਚੈਕਰ ਵਿੱਚ ਇੱਕ 11 ਸਾਲ ਪੁਰਾਣਾ ਮੋਰੀ 3:51 ਜਾਨਸ [pl] Anthropic porzuciło Gita na rzecz Scion, stworzonego do 30 000 merge'y dziennie Anthropic przeniósł swoje monorepo z Gita do Sciona, systemu kontroli wersji, który zbudował z myślą o 30 000 scaleń dziennie, w większości napisanego przez Claude'a. Na wtorkowej stronie głównej Hacker News znalazły się również: Gemini 4 Argon i jego milion tokenów na wyjściu, jedenastoletnia luka w systemie sprawdzającym pożyczkę w Rust oraz Janas, który uruchamia model o 80 miliardach parametrów na laptopie z 32 GB pamięci. Przeprojektowanie systemu kontroli źródeł w Anthropic anthropic.com Dyskusja: news.ycombinator.com nickg-anthropic o tym, dlaczego jedno monorepozytorium: news.ycombinator.com Gemini 4 Argon deepmind.google Ceny API: ai.google.dev Dyskusja: news.ycombinator.com vidarh o przycisku „Wypróbuj w Gemini”: news.ycombinator.com 11-letnie błędy w pożyczaniu w Rust zostały wykryte checker rust#164250 Dyskusja: news.ycombinator.com theemathas o tym, jak prosty jest reproduktor: news.ycombinator.com Janas: Uruchamianie LLM-ów większych niż pamięć RAM przez ekspertów od strumieniowania z dysków SSD prabanta-dev/janas Dyskusja: news.ycombinator.com Espressosaurus o buforowaniu przez ekspertów: news.ycombinator.com Strona główna z tego dnia: news.ycombinator.com Obserwuj @hacker-news na Bisque: bisque.today 0:00 Anthropic zastąpił Gita Scionem 0:30 Scion zakłada, że ​​każdy klient jest online 0:59 Inżynierowie mówią Claude'owi „nie” 1:29 Scion zostaje w Anthropicu 2:06 Gemini 4 Argon 2:40 Gdzie prowadzi Argon 3:11 11-letnia luka w module sprawdzającym pożyczki w Rust 3:51 Janas [ru] Anthropic ушла с Git на Scion, созданную для 30 000 слияний в день Anthropic перевела свой монорепозиторий с Git на Scion, систему контроля версий, разработанную для обработки 30 000 слияний в день, большая часть кода которой написана Клодом. Также на главной странице Hacker News за вторник: Gemini 4 Argon и его миллионный выпуск токенов, одиннадцатилетняя уязвимость в механизме проверки заимствований Rust и Janas, которая запускает модель с 80 миллиардами параметров на ноутбуке с 32 ГБ оперативной памяти. Перепроектирование системы контроля версий в Anthropic anthropic.com Обсуждение: news.ycombinator.com nickg-anthropic о том, почему нужен один монорепозиторий: news.ycombinator.com Gemini 4 Argon deepmind.google Цены на API: ai.google.dev Обсуждение: news.ycombinator.com vidarh о кнопке «Попробовать в Gemini»: news.ycombinator.com Обнаружены ошибки некорректной работы Rust borrow, существовавшие 11 лет проверка rust#164250 Обсуждение: news.ycombinator.com theemathas о простоте воспроизведения: news.ycombinator.com Janas: Запуск LLM-классов размером больше оперативной памяти путем потоковой передачи экспертов с SSD prabanta-dev/janas Обсуждение: news.ycombinator.com Espressosaurus о кэшировании экспертов: news.ycombinator.com Главная страница того дня: news.ycombinator.com Подписаться @hacker-news о Bisque: bisque.today 0:00 Anthropic заменила git на Scion 0:30 Scion предполагает, что каждый клиент онлайн 0:59 Инженеры говорят Клоду «нет» 1:29 Scion остается внутри Anthropic 2:06 Gemini 4 Argon 2:40 Куда ведет Argon 3:11 11-летняя дыра в проверке заимствований Rust 3:51 Янас [ta] Anthropic கிட்-ஐ (Git) விடுத்து Scion-க்கு மாறியது; இது தினமும் 30,000 மெர்ஜ்களுக்காக உருவாக்கப்ப... ஆந்த்ரோபிக் தனது மோனோரெப்போவை கிட்-டிலிருந்து, ஒரு நாளைக்கு 30,000 இணைப்புகளுக்காகத் தானே உருவாக்கிய சியோன் என்ற பதிப்புக் கட்டுப்பாட்டு அமைப்புக்கு மாற்றியது; இதை பெரும்பாலும் கிளாட் எழுதியுள்ளார். மேலும் செவ்வாய்க்கிழமை ஹேக்கர் நியூஸ் முதல் பக்கத்திலிருந்து: ஜெமினி 4 ஆர்கான் மற்றும் அதன் மில்லியன்-டோக்கன் வெளியீடு, ரஸ்ட்டின் கடன் சரிபார்ப்பியில் உள்ள பதினொரு வருடப் பழமையான நம்பகத்தன்மைக் குறைபாடு, மற்றும் 32 ஜிபி மடிக்கணினியில் 80 பில்லியன் அளவுருக்கள் கொண்ட மாதிரியை இயக்கும் ஜானாஸ். ஆந்த்ரோபிக்கில் சோர்ஸ் கண்ட்ரோலை மறுவடிவமைத்தல் anthropic.com கலந்துரையாடல்: news.ycombinator.com ஒரே மோனோரெப்போ ஏன் என்பது பற்றி நிக்ஜி-ஆந்த்ரோபிக்: news.ycombinator.com ஜெமினி 4 ஆர்கான் deepmind.google API விலை நிர்ணயம்: ai.google.dev கலந்துரையாடல்: news.ycombinator.com ஜெமினியில் முயற்சி செய் பொத்தான் பற்றி விடார்ஹ்: news.ycombinator.com ரஸ்ட் பாரோவில் 11 வருடப் பழமையான நம்பகத்தன்மையற்ற பிழைகள் கண்டறியப்பட்டன செக்கர் rust#164250 கலந்துரையாடல்: news.ycombinator.com மீண்டும் உருவாக்கும் கருவி எவ்வளவு எளிமையானது என்பது பற்றி தீமதாஸ்: news.ycombinator.com ஜானாஸ்: SSD-யிலிருந்து நிபுணர்களை ஸ்ட்ரீம் செய்வதன் மூலம் RAM-ஐ விட பெரிய LLM-களை இயக்குதல் prabanta-dev/janas கலந்துரையாடல்: news.ycombinator.com நிபுணர் கேச்சிங் பற்றி எஸ்ப்ரஸோசாரஸ்: news.ycombinator.com அன்றைய முதல் பக்கம்: news.ycombinator.com பிஸ்க்-இல் @hacker-news-ஐப் பின்தொடரவும்: bisque.today 0:00 ஆந்த்ரோபிக், கிட்-க்கு பதிலாக சியோனைப் பயன்படுத்தியது 0:30 ஒவ்வொரு கிளையன்ட்டும் ஆன்லைனில் இருப்பதாக சியோன் கருதுகிறது 0:59 பொறியாளர்கள் கிளாடிடம் வேண்டாம் என்கிறார்கள் 1:29 சியோன் ஆந்த்ரோபிக்கிற்குள் நீடிக்கிறது 2:06 ஜெமினி 4 ஆர்கான் 2:40 ஆர்கான் எங்கே வழிநடத்துகிறது 3:11 ரஸ்ட்டின் கடன் சரிபார்ப்பியில் 11 வருடப் பழைய ஓட்டை 3:51 ஜனாஸ் [te] రోజుకు 30,000 మెర్జ్‌ల కోసం నిర్మించబడిన సియాన్ కోసం ఆంత్రోపిక్ గిట్‌ను విడిచిపెట్టింది. ఆంత్రోపిక్ తన మోనోరెపోను గిట్ నుండి సైయాన్‌కు మార్చింది. ఇది రోజుకు 30,000 మెర్జ్‌ల కోసం వారు నిర్మించిన ఒక వెర్షన్ కంట్రోల్ సిస్టమ్, దీనిని ఎక్కువగా క్లాడ్ రాశారు. మంగళవారం నాటి హ్యాకర్ న్యూస్ మొదటి పేజీలోని ఇతర విశేషాలు: జెమిని 4 ఆర్గన్ మరియు దాని మిలియన్-టోకెన్ అవుట్‌పుట్, రస్ట్ బారో చెకర్‌లో పదకొండేళ్ల నాటి ఒక సౌండ్‌నెస్ హోల్, మరియు 32 GB ల్యాప్‌టాప్‌లో 80-బిలియన్-పారామీటర్ మోడల్‌ను నడిపే జానాస్. ఆంత్రోపిక్‌లో సోర్స్ కంట్రోల్‌ను పునఃరూపకల్పన చేయడం anthropic.com చర్చ: news.ycombinator.com ఒకే మోనోరెపో ఎందుకో వివరిస్తూ నిక్‌జి-ఆంత్రోపిక్: news.ycombinator.com జెమిని 4 ఆర్గన్ deepmind.google API ధరల నిర్ణయం: ai.google.dev చర్చ: news.ycombinator.com 'ట్రై ఇన్ జెమిని' బటన్‌పై విడార్: news.ycombinator.com రస్ట్ బారోలో 11 ఏళ్ల నాటి అసంబద్ధ బగ్‌లు కనుగొనబడ్డాయి చెకర్ rust#164250 చర్చ: news.ycombinator.com రీప్రొడ్యూసర్ ఎంత సులభమో వివరిస్తూ థీమాథాస్: news.ycombinator.com జానాస్: SSD నుండి ఎక్స్‌పర్ట్స్‌ను స్ట్రీమింగ్ చేయడం ద్వారా RAM కంటే పెద్ద LLMలను రన్ చేయండి prabanta-dev/janas చర్చ: news.ycombinator.com ఎక్స్‌పర్ట్ క్యాషింగ్‌పై ఎస్ప్రెసోసారస్: news.ycombinator.com ఆ రోజు ఫ్రంట్ పేజీ: news.ycombinator.com బిస్క్‌లో @hacker-newsను అనుసరించండి: bisque.today 0:00 ఆంత్రోపిక్, గిట్ స్థానంలో సియాన్‌ను ప్రవేశపెట్టింది 0:30 ప్రతి క్లయింట్ ఆన్‌లైన్‌లో ఉందని సియాన్ భావిస్తుంది 0:59 ఇంజనీర్లు క్లాడ్‌కు వద్దని చెబుతారు 1:29 సియాన్ ఆంత్రోపిక్‌లోనే ఉంటుంది 2:06 జెమిని 4 ఆర్గన్ 2:40 ఆర్గన్ ఎక్కడికి దారితీస్తుంది 3:11 రస్ట్ బారో చెకర్‌లో 11 ఏళ్ల నాటి లోపం 3:51 జానాస్ [uk] Anthropic перейшла з git на Scion, створену для 30 000 злиття коду на день Anthropic перенесла свій монорепозиторій з git на Scion, систему контролю версій, яку вона створила для 30 000 злиттів на день, здебільшого написаних Клодом. Також з першої шпальти Hacker News за вівторок: Gemini 4 Argon та його мільйонний вихід токенів, одинадцятирічна діра в надійності перевірки запозичень Rust та Janas, який запускає модель з 80 мільярдами параметрів на ноутбуці з 32 ГБ. Редизайн системи контролю версій в Anthropic anthropic.com Обговорення: news.ycombinator.com nickg-anthropic про те, чому одне монорепозиторій: news.ycombinator.com Gemini 4 Argon deepmind.google Ціни API: ai.google.dev Обговорення: news.ycombinator.com vidarh про кнопку «Спробувати в Gemini»: news.ycombinator.com У Rust знайдено 11-річні помилки ненадійності перевірка rust#164250 Обговорення: news.ycombinator.com тема про те, наскільки простий репродуктор: news.ycombinator.com Janas: Запускайте LLM, більші за оперативну пам'ять, шляхом потокової передачі експертів з SSD prabanta-dev/janas Обговорення: news.ycombinator.com Еспресозавр про кешування експертів: news.ycombinator.com Головна сторінка того дня: news.ycombinator.com Слідкуйте за @hacker-news на Bisque: bisque.today 0:00 Anthropic замінив git на Scion 0:30 Scion припускає, що кожен клієнт онлайн 0:59 Інженери кажуть Клоду «ні» 1:29 Scion залишається всередині Anthropic 2:06 Gemini 4 Argon 2:40 Куди веде Argon 3:11 11-річна дірка в системі перевірки боргів Rust 3:51 Janas

Janas: prabanta-dev/janas HN thread: news.ycombinator.com Strata, for comparison: Niko1221/Strata

Janas: Run LLMs larger than RAM by streaming experts from SSD prabanta-dev/janas (news.ycombinator.com)

Run a 48 GB AI Model on a 32 GB Laptop: Janas Streams Model Experts From Your SSD at 37 tok/s The wall everyone hits with local models is always the same one: the model you actually want to run is bigger than your memory. You find a promising 80B mixture-of-experts model, check the file size, see 48 GB, look at your 32 GB of RAM, and close the tab. Or worse, you try it anyway and watch your machine drown in page faults while tokens crawl out at 3 per second. A project that hit the Hacker News front page this week claims to break that wall. It is called Janas, a language model engine written in C by Maurizio Cammalleri, and its headline number is striking: 37 tokens per second from Qwen3-Next-80B, a 48.4 GB model, on a laptop with 32 GB of RAM. The same model on llama.cpp on the same laptop: 2.8 tokens per second. Full disclosure before anything else: I have not run Janas myself. Every number in this article comes from the project's own published benchmarks, its README, and the Hacker News discussion, all linked inline. Where a claim is the author's own measurement rather than an independent one, I say so. Treat this as a guided tour with a checklist, not a personal success story. Why models bigger than RAM usually crawl The obvious question is: does not the operating system already handle this? If the model file sits on an SSD, the OS can memory-map it and page in pieces as needed. Tools like llama.cpp do exactly that with mmap. The problem is what happens when the working set exceeds memory. A mapped file does not know which parts of the model matter. Every page fault is a trip through the kernel's page cache machinery, and when memory runs out, the kernel starts evicting pages you are about to need. Researchers who studied this problem describe it as a fault storm: at around twice your RAM in model size, the mmap approach collapses, because the OS is juggling pages blindly while your inference stalls waiting on disk. Mixture-of-experts models make this worse in a specific way and also hint at the fix. A MoE model contains many separate "expert" feed-forward blocks, and each generated token only routes through a few of them, often 8 out of 256. Most of those billions of parameters sit idle for any given token. The bottleneck is not reading the model, it is reading the right part of the model, fast, every single token. What Janas does differently Janas attacks that problem directly, and the design is easier to trust because it is simple to state: - Weights stay on disk. The engine never loads the whole model. It keeps a converted .jns file on your NVMe SSD and reads expert weights with O_DIRECT, bypassing the page cache entirely so the OS never starts thrashing. - Hot experts get cached. As the model runs, the engine notices which experts keep coming back and holds those in a memory cache. It measures your machine at startup and sizes its own caches and thread choices from what it finds. - The GPU reads prompts, not the whole model. On machines with an integrated GPU, Janas uses it for prompt processing, where parallelism helps most, and streams expert weights for token generation. You can force it CPU-only with JANAS_GPU=0. - Draft tokens hide latency. The engine supports speculative drafting: a small block guesses the next few tokens and the big model checks them in one pass. On Qwen3-Next-80B that took writing speed from 24 to 37 tok/s, because the text is identical, just produced faster. The result on the author's test laptop, a Debian 13 machine with an Intel Core Ultra 9 185H, 32 GB of DDR5, and a 1 TB NVMe SSD reading at about 5.7 GB/s: Qwen3-Next-80B reads a 1,024-token prompt at 62 tok/s and writes at 24 tok/s without drafts. Qwen3-Coder-Next, also 48.4 GB, does 66 and 22. The benchmark numbers, model by model The author reports Janas beat llama.cpp on all 13 models compared, on the same laptop, in Q4_K_M quantization, measured over 9-11 October 2026. These are the author's own measurements, so weigh them accordingly. The headline rows: - Qwen3-Next-80B-A3B (48.4 GB, larger than the RAM): 62 tok/s reading vs 16, 24 tok/s writing vs 2.8. That is roughly 8.5x faster at writing. - Qwen3-Coder-Next (48.4 GB, larger than the RAM): 66 tok/s reading vs 14, 22 tok/s writing vs 2.2. - Qwen3.6-35B-A3B (22.3 GB): 104 vs 79 reading, 19 vs 12 writing. - Qwen3-30B-A3B (18.6 GB): 135 vs 122 reading, 32 vs 15 writing. - Gemma-4-26B-A4B-it (18.3 GB): 90 vs 76 reading, 18 vs 12 writing. - Small dense models: Qwen3-0.6B reads at 939 tok/s vs 670. Even here, Janas is faster. One nuance worth knowing: for the oversized model, llama.cpp did not simply fall behind, it broke. With its usual build it ran out of memory as soon as the GPU joined, because its CPU backend copies weights to repack them and its Vulkan backend copies what it leaves on the CPU into buffers the system cannot reclaim. The 2.8 tok/s figure required a special build without repacking plus flags that leave the experts in the file. Janas ran the same model with no special options. Skeptics in the Hacker News thread made the fair point that llama.cpp has had mmap-based disk streaming for years, and other projects like Strata and flash-moe work the same territory. The difference in the numbers, if they hold up on other machines, is that Janas measures your hardware and actively manages which experts live in memory, rather than leaving page-in decisions to the kernel. Whether that gap survives independent testing is exactly the kind of thing the janas-bench report is meant to answer, and the author explicitly says results on untested hardware are unknown. How to try it yourself If you have a Linux x86-64 machine (Ubuntu 22.04, Debian 12 or later) and want to reproduce or falsify these numbers, the quickest path from the README: curl -LO https://github.com/prabanta-dev/janas/releases/latest/download/janas-linux-x86_64.tar.gz tar xf janas-linux-x86_64.tar.gz && cd janas ./janas-chat Pick a model from the catalog and it downloads and verifies it. On Windows 10 (1903+) and 11 there is a zip release; the author reports writing speed on Windows 11 matched Linux, and dense models read prompts 1.8x faster there because whole blocks go to the GPU through Intel's own driver. Practical flags worth knowing before you start: - --cache <GiB> puts a hard ceiling on the memory Janas takes. The engine fills free memory with its expert cache by default, which is polite to nobody. - JANAS_GPU=0 keeps it off the GPU entirely. The README honestly notes the GPU path once caused a driver reset that took the desktop down with it. - janas-bench measures your machine and produces a report. If you run it anywhere, that report is the single most useful thing you can send the project, because it has so far been tested on exactly one laptop. - Close your browser first. With a browser and editor open, the same model kept about three quarters of its speed but waited five times longer on disk. Run your first test on an idle machine. The catches, stated plainly - It is an experiment, not a product. The command line has changed several times a week. Only the .jns model file format is promised stable. - One machine, one author. Everything was developed and measured on a single Intel laptop with an integrated Arc GPU. AMD CPUs, discrete GPUs, and slower disks are explicitly untested territory. - It is deliberately hard on your machine. Full-memory caches plus full-speed O_DIRECT reads can push a machine without headroom into swap. The author says it degrades gently, but budget your cache anyway. - Model availability is the real constraint. You need converted .jns files, which currently means the author's Hugging Face catalog of Qwen and Gemma models, or converting GGUF files yourself. Your favorite model may not exist in this format yet. - Draft speedup depends on the text. Drafts survived 74-89% of the time on English prose in the author's tests, but on Italian prose one model kept only about a third. The README is candid that an earlier version of that benchmark was flattering and wrong, and it was corrected. Should you use it? A decision checklist Here is how I would decide, given that these are published numbers rather than my own testing: - Your model fits in RAM: stay on llama.cpp. It is mature, universal, and the gap on small models, while real in the benchmarks, buys you little. - Your model is bigger than your RAM and you have a fast NVMe SSD: Janas is exactly built for this case, and 24-37 tok/s on an 80B model changes it from a curiosity into something usable. Verify with janas-bench before you believe it on your hardware. - You need reliability or a specific model: the ecosystem and tooling around llama.cpp are years deep. Janas lists no stability guarantee at all. - You just need good answers and have a connection: a paid API still beats every local setup on quality per token of effort. Local models are about privacy, cost at scale, and offline use, not beating frontier APIs. - You enjoy the edge: run janas-bench and send the report. Projects like this live or die on measurements from machines the author does not own. The bigger story here is that the definition of "too big to run locally" is now tied to your SSD bandwidth, not your RAM capacity. An NVMe drive moving 5-7 GB/s can stream more expert weights per second than a token generator consumes. If that gap holds, the practical ceiling on local models stops being your memory and starts being your disk, and disks are cheaper to upgrade. The author has written more about the engine on his Substack, and the Hacker News discussion is worth reading for the skeptics' counterpoints. I write about AI infrastructure, developer tools, and backend engineering every week. Subscribe, it is free. Have you pushed a local model past your RAM limit? What happened when you tried, and did anything actually work? I am collecting real-world results on this one, so tell me in the comments.

Janas: Run LLMs larger than RAM by streaming experts from SSD prabanta-dev/janas news.ycombinator.com