pipecat

I measured what jambonz actually sends on a websocket, instead of reading the docs Pipecat ships transport serializers for Twilio, Telnyx, Plivo, Exotel, Vonage and Genesys. A developer choosing a phone layer has six options, and jambonz — open source, self-hostable, already capable of forking call audio to a websocket — isn't one of them. So I wrote it. This post is about the part before the code: finding out what the socket actually does. Three things I couldn't look up A FrameSerializer in Pipecat is two methods. serialize turns a Pipecat frame into something to write to the socket; deserialize turns a socket message into a frame. To write either I needed to know: 1. What sample rate does jambonz actually send? The listen verb takes a sampleRate. Does it honour it? 2. Is the framing stable? That decides whether deserialize needs a variable-length path. 3. Does control data share the socket with audio? This is the one that changes the shape of the class. Exotel base64s audio into JSON events, so everything is one stream of JSON. If jambonz interleaves JSON control frames with raw binary audio, deserialize has to branch on every message. None of these are in jambonz's docs or Pipecat's. I had an assumption about the first one and I'd already written it into a public README. It was wrong. The probe Rather than reason about it, I wrote about 120 lines of aiohttp with two endpoints. POST / is the call hook — jambonz asks it what to do, and it answers with a single listen verb pointing back at the websocket: verbs = [{ "verb": "listen", "url": ws_url, "mixType": "mono", "sampleRate": SAMPLE_RATE, "bidirectionalAudio": {"enabled": True, "streaming": True, "sampleRate": SAMPLE_RATE}, }] GET /ws receives the forked audio and, crucially, describes it rather than interprets it: how big each frame is, whether the size is stable, whether any TEXT frames arrive, and the byte total divided by the rate we asked for, against wall clock. That last ratio is the whole point. If I ask for 16 kHz and 30 seconds of wall clock produces 30 seconds of implied audio, the rate is right. If it produces 15, the real rate is half what I asked for. Then: ngrok, point a jambonz application at it, dial from a softphone, talk for thirty seconds, hang up. What came back binary frames : 1514 total bytes : 968960 frame sizes : [640] text frames : 1 wall clock : 30.03s audio implied : 30.28s at 16000 Hz, 16-bit mono ratio : 1.01 The rate the verb asks for is the rate that arrives. Ratio 1.01 across thirty seconds. My assumption had been that jambonz forks at 8 kHz — I'd written that in a public README and it was wrong by a factor of two. It forks at whatever sampleRate the verb set, and the default is 16 kHz. Framing is fixed at 20 ms. One distinct frame size across 1514 frames: 640 bytes, which is 320 samples of 16-bit mono, which is exactly 20 ms at 16 kHz. No variable-length path needed. And the third question had a better answer than I expected. Exactly one TEXT frame arrives, before any audio, and then the socket is pure binary for the rest of the call: {"sampleRate":16000,"mixType":"mono","callSid":"3e5d869a-…", "direction":"inbound","from":"…","to":"…","sipStatus":200, "callStatus":"in-progress","traceId":"…"} It's a preamble, not a channel. So deserialize doesn't branch per frame. It reads one metadata frame at the start and treats everything after it as L16 PCM. What that does to the design The sample rate should be read from the wire, not configured. My first spec said the verb and the serializer "have to agree" on a rate. They don't have to — the socket states it. A params field can still override for a narrowband carrier, but the default is to believe the connection. A narrowband call at 8 kHz then needs no code change at all. I also left it deliberately unset until the preamble arrives, rather than defaulting to 16000. A connection with no preamble then drops audio with a warning, instead of resampling from a rate that was never negotiated — which on a narrowband call produces audio that sounds almost right, and that is the worst kind of wrong. The preamble carries callSid and traceId for free. That's enough to correlate a Pipecat session with a jambonz call in logs, without threading identifiers through the verb's URL. And interruptions can't just stop. jambonz has killAudio, which flushes playing and buffered audio. But I asked about the race in the jambonz community Slack and got this from Sam Machin: > the ordering of messages on the socket isn't 100% we've seen clients that sometimes prioritise smaller messages so its best to have a small break after a killAudio before you send the next stream, 50-100ms should be plenty So a frame written in the same instant as the flush can land after it and play over the caller. Pipecat's interruption model lets a processor stop; over jambonz it has to stop and wait. There's nowhere upstream to put that, so the serializer holds the frames itself for 100 ms — and discards them rather than queueing, because they belong to the turn the caller just interrupted and arriving late is worse than not arriving. A gotcha worth the whole post I wrote the tests before the serializer, from the measurements. Three of them failed against correct code. Pipecat's create_stream_resampler does not emit one chunk per chunk. It buffers and flushes in bursts: chunk 1: in 640 -> out 0 chunk 2: in 640 -> out 0 chunk 3: in 640 -> out 0 chunk 4: in 640 -> out 3300 Three empty returns, then everything at once. My tests assumed one frame in, one frame out — so serialize returning None looked like a bug when it was the correct answer. Over twenty frames the throughput settles to about 86% of the arithmetic ideal, the rest being the tail still held in the buffer. If you write a serializer for any transport, don't assert a byte count per call. Pump a run of frames and assert on the total, which is what a real call does anyway. The thing I'd take away I got the sample rate wrong by assuming, and it reached a public README. I got the test shape wrong by assuming, and it failed against working code. The protocol questions that mattered were all answered in thirty seconds of one phone call and about 120 lines of throwaway code. Writing the probe felt like a detour from writing the serializer. It wasn't — it was the serializer. Everything after it was transcription. The probe, the measurements and the spec: **Rehansanjay/jambonz-bridge** The serializer, in review upstream: **pipecat#6112

My tool called a maintainer's plugin broken. They had fixed it two weeks earlier. For about a month I've had a nightly job that installs every community-maintained plugin listed in Pipecat's docs into a fresh environment, imports it, and checks that every service class it defines can still be constructed. It publishes a table: works, broken, pinned to an old version, can't install. It installs from PyPI, because that's what a user gets. In late September I opened a pull request against one of the red ones — pipecat-boson, failing on an import that pipecat moved in 1.8. It sat for a fortnight, and by the time anyone looked at it the maintainer had landed their own fix, so mine was closed. Fair enough; theirs is better than mine was: try: from pipecat.utils.types import assert_given, is_given except ImportError: # pipecat-ai < 1.8.0 from pipecat.services.settings import assert_given, is_given That landed on 30 September. PyPI still serves the version published before it. So for more than two weeks my dashboard printed a red row, with their project's name on it, for code that was already correct. The check wasn't buggy. It answered exactly the question I wrote it to answer. The question was wrong. "Broken" was doing two jobs There are two different sentences hiding in one red row: - Your users are getting a package that doesn't work. True for Boson. Still true today. - Your code is broken. False for Boson, and false for two weeks. The first is the user's question and the wheel answers it. The second is the maintainer's question, and the wheel cannot answer it at all — a wheel is a snapshot of a repository at some past moment, and nothing says how far behind that moment is. This matters more than tidiness, because the two sentences imply different actions. "Your code is broken" asks for a patch. "Your users are getting a broken package" asks for a release. If you only ever send the first one, you're asking people to redo work they've already done, and the thing that would actually help their users never gets mentioned. So the checker now asks both. When the released version fails, it installs the plugin's own source from its repository, pinned to the commit under test, and probes that too. A plugin whose source imports and constructs gets a new status — fixed, not released — which keeps the released failure alongside it and names the commit that passed. Finding the repository is the hard part I assumed this would be a lookup. PyPI has project_urls. You read Repository, you clone it, done. pipecat-boson declares nothing. No Homepage, no Repository, no project_urls at all. Its repository appears in exactly one place in the entire package metadata: a git clone line in the README body. And then there's this one, which is worse: Homepage https://github.com/your-username/pipecat-firered-vad Repository https://github.com/your-username/pipecat-firered-vad Bug Tracker https://github.com/your-username/pipecat-firered-vad/issues Changelog https://github.com/your-username/pipecat-firered-vad/releases All four published URLs are a template placeholder that was never filled in. Anyone who installs that package and hits a problem follows the Bug Tracker link to a 404. Their bug report doesn't go to the wrong place — it goes nowhere, and the maintainer never learns the report existed. There's no route from the package page back to the repository either. I only found the real one because the install error happened to name the dependency. So resolution reads the metadata first and falls back to scanning the README. That gets 58 of 67, and every plugin that's currently failing. If you maintain a published package: go and look at your own project_urls right now. It takes ten seconds and the failure mode is silent. The guard matters more than the feature Here's the part I'd want to read if someone else wrote this. pipecat-supertonic resolves to supertone-inc/supertonic — the vendor's own text-to-speech model repository, thirteen thousand stars — because the two names share a word. The plugin actually lives somewhere else entirely. That particular one fails to install, so it reports unknown either way and I got lucky. But a wrong repository that happened to be pip-installable and to import cleanly would have promoted the plugin to "fixed, not released" on the strength of a completely unrelated codebase, and published that claim under a maintainer's name. The fix is that the source build has to produce the distribution the plugin actually is, normalised per PEP 503, or the answer is unknown: expected = {_canonical(project_name(spec)) for spec in plugin.specs if project_name(spec)} built = {_canonical(dist.split("==")[0]) for dist in probe.get("distributions") or []} if expected and not expected & built: return {"source": "unknown", "source_reason": "that repository does not build " + ...} Measuring something in public means you can be wrong about it in public. The guards are the work, not the measurement. What it found, versus what I expected I expected a pile of these. My assumption going in was that several vendors had quietly merged fixes without cutting releases, which would have given me a stack of genuinely welcome messages to send — your fix is merged, your users still get the broken wheel, a release closes it. Nobody minds receiving that. Thirteen plugins currently fail. Rebuilt from their own source: | | count | | --- | --- | | Still broken when built from source | 8 | | Fixed in source, stale on PyPI | 1 | | Couldn't be read, reported unknown | 4 | One. Not a stack. And the harder number: zero of the nine vendors has fixed their own plugin. I read the HEAD of each repository to check. Two of them looked fixed and were my own false positives — one still imports the removed name from the old path, and the other still uses a frame class that was renamed, which my first grep matched as a substring of the new name. So the thing I built to find a pile of easy wins found one, and mostly confirmed that the plugins are broken in source too. That's a worse story and a more useful tool. The one that gained something real One plugin isn't on PyPI at all under the name its documentation publishes. For a month its row said can't install, with no diagnosis, because there was nothing to install and therefore nothing to diagnose. Built from source, it produces a real one — the same import failure as the others. That's the part I'd keep if I could only keep one thing. For a package that was never published, the source isn't a second opinion. It's the only place its state exists at all. --- The checker, the full table and the nightly results: **Rehansanjay/pipecat-plugin-health** If you maintain a plugin for a fast-moving framework, I'd genuinely like to know whether you run CI against its main — and if not, whether it's because nobody asked.

8 github repos that print $$$ Close your first (or next) AI services client within 90 days or get 100% of your money back: skool.com Get all my FREE resources (including the ones mentioned in this video): skool.com Links mentioned in this episode: Treg (find verified leads): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Browser Use: browser-use/browser-use Claude Finance (Anthropic financial services): anthropics/financial-services Timestamps: [00:00] - Intro: 8 GitHub Repos To Monetize [00:28] - The Simple Playbook For Every Repo [01:02] - Repo 1: ListMonk Win-Back Offer [03:54] - Finding Leads With Treg [05:07] - Repo 2: Postiz Social Autopilot [07:27] - Repo 3: WhisperX Call Scorecards [10:01] - Free Resource: All 8 Repos [10:34] - Repo 4: PipeCat After-Hours Receptionist [12:57] - Repo 5: Docling Invoice Entry [15:31] - Repo 6: ComfyUI Listing Images [17:53] - Repo 7: Browser Use Permit Tracker [19:53] - Repo 8: Claude Finance Listing Packs [22:02] - The Scoreboard And Where To Start [22:54] - Next Steps: Pull 100 Leads FIND ME ON SOCIAL: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [ar] 8 مستودعات GitHub تدر أرباحاً طائلة أبرم أول (أو ثاني) صفقة لك مع عميل في مجال خدمات الذكاء الاصطناعي خلال 90 يومًا أو استرد أموالك كاملةً: skool.com احصل على جميع مواردي المجانية (بما في ذلك الموارد المذكورة في هذا الفيديو): skool.com الروابط المذكورة في هذه الحلقة: Treg (للعثور على عملاء محتملين موثوقين): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Browser رابط الاستخدام: browser-use/browser-use كلود فاينانس (الخدمات المالية الأنثروبيّة): anthropics/financial-services الفهرس الزمني: [00:00] - مقدمة: ٨ مستودعات على GitHub لتحقيق الربح [00:28] - دليل بسيط لكل مستودع [01:02] - المستودع ١: عرض استعادة العملاء من ListMonk [03:54] - إيجاد عملاء محتملين باستخدام Treg [05:07] - المستودع ٢: Postiz Social Autopilot [07:27] - المستودع ٣: WhisperX Call Scorecards [10:01] - مورد مجاني: جميع المستودعات الثمانية [10:34] - المستودع ٤: PipeCat After-Hours Receptionist [12:57] - المستودع ٥: Docling Invoice المدخل [15:31] - المستودع 6: صور قوائم ComfyUI [17:53] - المستودع 7: متتبع تصاريح استخدام المتصفح [19:53] - المستودع 8: حزم قوائم Claude Finance [22:02] - لوحة النتائج وكيفية البدء [22:54] - الخطوات التالية: الحصول على 100 عميل محتمل تابعني على مواقع التواصل الاجتماعي: تويتر: x.com إنستغرام: instagram.com لينكد إن: linkedin.com يوتيوب: youtube.com [bn] ৮টি গিটহাব রিপো যা থেকে আয় করুন হাজার ডলার ৯০ দিনের মধ্যে আপনার প্রথম (বা পরবর্তী) এআই পরিষেবা ক্লায়েন্টকে চূড়ান্ত করুন অথবা আপনার ১০০% টাকা ফেরত নিন: skool.com আমার সমস্ত বিনামূল্যের রিসোর্স নিন (এই ভিডিওতে উল্লিখিত রিসোর্সগুলো সহ): skool.com এই পর্বে উল্লিখিত লিঙ্কসমূহ: Treg (যাচাইকৃত লিড খুঁজুন): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI ব্রাউজার ব্যবহার: browser-use/browser-use ক্লড ফাইন্যান্স (অ্যানথ্রোপিক আর্থিক পরিষেবা): anthropics/financial-services টাইমস্ট্যাম্প: [০০:০০] - ভূমিকা: মনিটাইজ করার মতো ৮টি গিটহাব রিপো [০০:২৮] - প্রতিটি রিপোর জন্য সহজ প্লেবুক [০১:০২] - রিপো ১: লিস্টমঙ্ক উইন-ব্যাক অফার [০৩:৫৪] - ট্রেগ দিয়ে লিড খোঁজা [০৫:০৭] - রিপো ২: পোস্টিজ সোশ্যাল অটোপাইলট [০৭:২৭] - রিপো ৩: হুইসপারএক্স কল স্কোরকার্ড [১০:০১] - ফ্রি রিসোর্স: সব ৮টি রিপো [১০:৩৪] - রিপো ৪: পাইপক্যাট আফটার-আওয়ার্স রিসেপশনিস্ট [১২:৫৭] - রিপো ৫: ডকলিং ইনভয়েস এন্ট্রি [১৫:৩১] - রেপো ৬: ComfyUI লিস্টিং ইমেজ [১৭:৫৩] - রেপো ৭: ব্রাউজার ব্যবহারের অনুমতি ট্র্যাকার [১৯:৫৩] - রেপো ৮: Claude Finance লিস্টিং প্যাক [২২:০২] - স্কোরবোর্ড এবং কোথা থেকে শুরু করবেন [২২:৫৪] - পরবর্তী পদক্ষেপ: ১০০টি লিড সংগ্রহ করুন সোশ্যাল মিডিয়ায় আমাকে খুঁজুন: X/টুইটার: x.com ইনস্টাগ্রাম: instagram.com লিঙ্কডইন: linkedin.com ইউটিউব: youtube.com [de-DE] 8 GitHub-Repos, die für dich Geld drucken Schließen Sie Ihren ersten (oder nächsten) KI-Dienstleistungskunden innerhalb von 90 Tagen ab oder erhalten Sie Ihr Geld zurück: skool.com Sichern Sie sich alle meine KOSTENLOSEN Ressourcen (einschließlich der in diesem Video erwähnten): skool.com Links aus dieser Folge: Treg (verifizierte Leads finden): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Browsernutzung: browser-use/browser-use Claude Finance (Anthropische Finanzdienstleistungen): anthropics/financial-services Zeitstempel: [00:00] – Einleitung: 8 GitHub-Repos zur Monetarisierung [00:28] – Die einfache Anleitung für jedes Repo [01:02] – Repo 1: ListMonk Win-Back-Angebot [03:54] – Leads mit Treg finden [05:07] – Repo 2: Postiz Social Autopilot [07:27] – Repo 3: WhisperX Call Scorecards [10:01] – Kostenlose Ressource: Alle 8 Repos [10:34] – Repo 4: PipeCat After-Hours Receptionist [12:57] – Repo 5: Docling Invoice Entry [15:31] – Repo 6: ComfyUI-Angebotsbilder [17:53] – Repository 7: Browser-Nutzungsberechtigungs-Tracker [19:53] – Repository 8: Claude Finance-Angebotspakete [22:02] – Die Übersicht und der Einstieg [22:54] – Nächste Schritte: 100 Leads generieren FOLGE MIR AUF SOCIAL MEDIA: Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [es-US] 8 repos de GitHub que generan dinero $$$ Consigue tu primer (o próximo) cliente de servicios de IA en 90 días o te devolvemos el 100% de tu dinero: skool.com Obtén todos mis recursos GRATUITOS (incluidos los mencionados en este video): skool.com Enlaces mencionados en este episodio: Treg (encuentra clientes potenciales verificados): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Uso del navegador: browser-use/browser-use Claude Finance (Servicios financieros de Anthropic): anthropics/financial-services Marcadores de tiempo: [00:00] - Introducción: 8 repositorios de GitHub para monetizar [00:28] - Guía práctica para cada repositorio [01:02] - Repositorio 1: Oferta de recuperación de clientes de ListMonk [03:54] - Cómo encontrar clientes potenciales con Treg [05:07] - Repositorio 2: Postiz Social Autopilot [07:27] - Repositorio 3: WhisperX Call Scorecards [10:01] - Recurso gratuito: Los 8 repositorios [10:34] - Repositorio 4: PipeCat After-Hours Receptionist [12:57] - Repositorio 5: Documentación de entrada de facturas [15:31] - Repositorio 6: Imágenes de listados de ComfyUI [17:53] - Repositorio 7: Rastreador de permisos de uso del navegador [19:53] - Repositorio 8: Paquetes de listados de Claude Finance [22:02] - El marcador y por dónde empezar [22:54] - Próximos pasos: Conseguir 100 clientes potenciales ENCUÉNTRAME EN REDES SOCIALES: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [fr-FR] 8 dépôts GitHub qui génèrent du cash Concluez votre premier (ou prochain) contrat de services d'IA sous 90 jours ou soyez remboursé intégralement : skool.com Accédez à toutes mes ressources GRATUITES (y compris celles mentionnées dans cette vidéo) : skool.com Liens mentionnés dans cet épisode : Treg (trouver des prospects qualifiés) : treg.to ListMonk : knadh/listmonk Postiz : gitroomhq/postiz-app WhisperX : m-bain/whisperX PipeCat : pipecat-ai/pipecat Docling : docling-project/docling ComfyUI : Comfy-Org/ComfyUI Utilisation via navigateur : browser-use/browser-use Claude Finance (Services financiers anthropiques) : anthropics/financial-services Horodatage : [00:00] - Introduction : 8 dépôts GitHub à monétiser [00:28] - Guide simple pour chaque dépôt [01:02] - Dépôt 1 : Offre de reconquête ListMonk [03:54] - Trouver des prospects avec Treg [05:07] - Dépôt 2 : Postiz Social Autopilot [07:27] - Dépôt 3 : WhisperX Call Scorecards [10:01] - Ressource gratuite : Les 8 dépôts [10:34] - Dépôt 4 : PipeCat After-Hours Receptionist [12:57] - Dépôt 5 : Docling Invoice Entry [15:31] - Dépôt 6 : Images de présentation ComfyUI [17:53] - Dépôt 7 : Suivi des autorisations d'utilisation du navigateur [19:53] - Dépôt 8 : Packs de présentation Claude Finance [22:02] - Le tableau de bord et par où commencer [22:54] - Prochaines étapes : Générer 100 prospects RETROUVEZ-MOI SUR LES RÉSEAUX SOCIAUX : X/Twitter : x.com Instagram : instagram.com LinkedIn : linkedin.com YouTube : youtube.com [hi] 8 GitHub रिपोज़ जिनसे होगी पैसों की बारिश अपने पहले (या अगले) एआई सेवा क्लाइंट को 90 दिनों के भीतर डील फाइनल करें या अपना पूरा पैसा वापस पाएं: skool.com मेरे सभी मुफ़्त संसाधन (इस वीडियो में बताए गए संसाधनों सहित) प्राप्त करें: skool.com इस एपिसोड में उल्लिखित लिंक: Treg (सत्यापित लीड ढूंढें): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI ब्राउज़र का उपयोग: browser-use/browser-use क्लाउड फाइनेंस (एंथ्रोपिक वित्तीय सेवाएं): anthropics/financial-services टाइमस्टैम्प: [00:00] - परिचय: कमाई के लिए 8 GitHub रिपॉजिटरी [00:28] - प्रत्येक रिपॉजिटरी के लिए सरल प्लेबुक [01:02] - रिपॉजिटरी 1: लिस्टमॉन्क विन-बैक ऑफर [03:54] - ट्रेग के साथ लीड्स ढूँढना [05:07] - रिपॉजिटरी 2: पोस्टिज़ सोशल ऑटोपायलट [07:27] - रिपॉजिटरी 3: व्हिस्परएक्स कॉल स्कोरकार्ड [10:01] - निःशुल्क संसाधन: सभी 8 रिपॉजिटरी [10:34] - रिपॉजिटरी 4: पाइपकैट आफ्टर-आवर्स रिसेप्शनिस्ट [12:57] - रिपॉजिटरी 5: डॉकलिंग इनवॉइस एंट्री [15:31] - रेपो 6: ComfyUI लिस्टिंग इमेज [17:53] - रेपो 7: ब्राउज़र उपयोग परमिट ट्रैकर [19:53] - रेपो 8: क्लाउड फाइनेंस लिस्टिंग पैक [22:02] - स्कोरबोर्ड और शुरुआत कहाँ से करें [22:54] - अगले चरण: 100 लीड प्राप्त करें मुझे सोशल मीडिया पर फॉलो करें: X/ट्विटर: x.com इंस्टाग्राम: instagram.com लिंक्डइन: linkedin.com यूट्यूब: youtube.com [id] 8 repos GitHub yang menghasilkan uang $$$ Dapatkan klien layanan AI pertama (atau berikutnya) Anda dalam 90 hari atau dapatkan 100% uang Anda kembali: skool.com Dapatkan semua sumber daya GRATIS saya (termasuk yang disebutkan dalam video ini): skool.com Tautan yang disebutkan dalam episode ini: Treg (temukan prospek terverifikasi): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Penggunaan Browser: browser-use/browser-use Claude Finance (Layanan keuangan Anthropic): anthropics/financial-services Cap waktu: [00:00] - Pendahuluan: 8 Repositori GitHub untuk Dimonetisasi [00:28] - Panduan Sederhana untuk Setiap Repositori [01:02] - Repositori 1: Penawaran Win-Back ListMonk [03:54] - Menemukan Prospek dengan Treg [05:07] - Repositori 2: Autopilot Sosial Postiz [07:27] - Repositori 3: Kartu Skor Panggilan WhisperX [10:01] - Sumber Daya Gratis: Semua 8 Repositori [10:34] - Repositori 4: Resepsionis Setelah Jam Kerja PipeCat [12:57] - Repositori 5: Entri Faktur Docling [15:31] - Repositori 6: Gambar Daftar ComfyUI [17:53] - Repo 7: Pelacak Izin Penggunaan Browser [19:53] - Repo 8: Paket Daftar Claude Finance [22:02] - Papan Skor dan Tempat Memulai [22:54] - Langkah Selanjutnya: Dapatkan 100 Prospek TEMUKAN SAYA DI MEDIA SOSIAL: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [it] 8 repository GitHub che fanno guadagnare soldi Concludi il tuo primo (o prossimo) cliente di servizi AI entro 90 giorni o ricevi il 100% di rimborso: skool.com Accedi a tutte le mie risorse GRATUITE (incluse quelle menzionate in questo video): skool.com Link menzionati in questo episodio: Treg (trova lead verificati): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Browser utilizzato: Italiano: browser-use/browser-use Claude Finance (Servizi finanziari di Anthropic): anthropics/financial-services Timestamp: [00:00] - Introduzione: 8 repository GitHub da monetizzare [00:28] - Il semplice manuale per ogni repository [01:02] - Repository 1: Offerta di recupero clienti ListMonk [03:54] - Trovare lead con Treg [05:07] - Repository 2: Postiz Social Autopilot [07:27] - Repository 3: Schede di valutazione delle chiamate WhisperX [10:01] - Risorsa gratuita: Tutti gli 8 repository [10:34] - Repository 4: PipeCat After-Hours Receptionist [12:57] - Repository 5: Inserimento fatture con Docling [15:31] - Repository 6: Immagini di inserzione ComfyUI [17:53] - Repository 7: Tracker dei permessi di utilizzo del browser [19:53] - Repository 8: Pacchetti di inserzioni Claude Finance [22:02] - Il punteggio e da dove iniziare [22:54] - Prossimi passi: Generare 100 lead TROVAMI SUI SOCIAL: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [iw] 8 מאגרי GitHub שמדפיסים כסף סגרו את לקוח שירותי הבינה המלאכותית הראשון (או הבא) שלכם תוך 90 יום או קבלו 100% מכספכם בחזרה: skool.com קבלו את כל המשאבים שלי בחינם (כולל אלה המוזכרים בסרטון הזה): skool.com קישורים המוזכרים בפרק הזה: Treg (מצא לידים מאומתים): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI שימוש בדפדפן: browser-use/browser-use Claude Finance (שירותים פיננסיים אנתרופיים): anthropics/financial-services חותמות זמן: [00:00] - מבוא: 8 מאגרים ב-GitHub להפקת רווחים [00:28] - מדריך פשוט לכל מאגר [01:02] - מאגר 1: הצעת החזרה של ListMonk [03:54] - מציאת לידים עם Treg [05:07] - מאגר 2: טייס אוטומטי של Postiz Social [07:27] - מאגר 3: כרטיסי ניקוד שיחות WhisperX [10:01] - משאב חינמי: כל 8 המאגרים [10:34] - מאגר 4: פקיד קבלה לאחר שעות העבודה של PipeCat [12:57] - מאגר 5: רישום חשבונית [15:31] - מאגר 6: תמונות רישום של ComfyUI [17:53] - מאגר 7: מעקב אחר היתרי שימוש בדפדפן [19:53] - מאגר 8: חבילות רישום של Claude Finance [22:02] - לוח התוצאות והיכן להתחיל [22:54] - השלבים הבאים: משיכת 100 לידים מצאו אותי ברשתות החברתיות: X/טוויטר: x.com אינסטגרם: instagram.com לינקדאין: linkedin.com יוטיוב: youtube.com [ja] 収益を生み出すGitHubリポジトリ8選 90日以内に最初の（または次の）AIサービス契約を締結しない場合は、全額返金いたします：skool.com この動画で紹介したリソースを含む、すべての無料リソースを入手するには：skool.com このエピソードで紹介したリンク： Treg（検証済みリードの検索）：treg.to ListMonk：knadh/listmonk Postiz：gitroomhq/postiz-app WhisperX：m-bain/whisperX PipeCat：pipecat-ai/pipecat Docling：docling-project/docling ComfyUI：Comfy-Org/ComfyUI ブラウザの使用： browser-use/browser-use Claude Finance (Anthropic Financial Services): anthropics/financial-services タイムスタンプ: [00:00] - はじめに: 収益化できるGitHubリポジトリ8選 [00:28] - 各リポジトリのためのシンプルなプレイブック [01:02] - リポジトリ1: ListMonk 再獲得オファー [03:54] - Tregを使ったリード獲得 [05:07] - リポジトリ2: Postiz ソーシャルオートパイロット [07:27] - リポジトリ3: WhisperX コールスコアカード [10:01] - 無料リソース: 全8リポジトリ [10:34] - リポジトリ4: PipeCat アフターアワーズ受付 [12:57] - リポジトリ5: Docling 請求書エントリー [15:31] - リポジトリ6：ComfyUIリスティング画像 [17:53] - リポジトリ7：ブラウザ使用許可トラッカー [19:53] - リポジトリ8：Claude Financeリスティングパック [22:02] - スコアボードと開始方法 [22:54] - 次のステップ：リード100件獲得 ソーシャルメディアで私を見つけてください： X/Twitter：x.com Instagram：instagram.com LinkedIn：linkedin.com YouTube：youtube.com [ko] 돈이 복사되는 8가지 GitHub 저장소 90일 이내에 첫 번째(또는 다음) AI 서비스 고객을 확보하지 못하면 100% 환불해 드립니다: skool.com 모든 무료 자료(이 영상에서 언급된 자료 포함)를 받으세요: skool.com 이 에피소드에서 언급된 링크: Treg(검증된 리드 찾기): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI 브라우저 사용: browser-use/browser-use Claude Finance (Anthropics 금융 서비스): anthropics/financial-services 타임스탬프: [00:00] - 소개: 수익 창출을 위한 8가지 GitHub 리포지토리 [00:28] - 모든 리포지토리를 위한 간단한 플레이북 [01:02] - 리포지토리 1: ListMonk 고객 재확보 제안 [03:54] - Treg를 활용한 리드 발굴 [05:07] - 리포지토리 2: Postiz 소셜 자동화 도구 [07:27] - 리포지토리 3: WhisperX 통화 스코어카드 [10:01] - 무료 리소스: 8개 리포지토리 모두 [10:34] - 리포지토리 4: PipeCat 야간 접수원 [12:57] - 리포지토리 5: Docling 송장 항목 [15:31] - 저장소 6: ComfyUI 리스팅 이미지 [17:53] - 저장소 7: 브라우저 사용 권한 추적기 [19:53] - 저장소 8: Claude Finance 리스팅 팩 [22:02] - 성과표 및 시작점 [22:54] - 다음 단계: 리드 100개 확보 소셜 미디어에서 저를 찾아보세요: X/트위터: x.com 인스타그램: instagram.com 링크드인: linkedin.com 유튜브: youtube.com [ml] പണം വാരാൻ സഹായിക്കുന്ന 8 GitHub റിപ്പോസിറ്ററികൾ നിങ്ങളുടെ ആദ്യത്തെ (അല്ലെങ്കിൽ അടുത്ത) AI സേവന ക്ലയന്റ് 90 ദിവസത്തിനുള്ളിൽ അടയ്ക്കുക അല്ലെങ്കിൽ നിങ്ങളുടെ പണത്തിന്റെ 100% തിരികെ നേടുക: skool.com എന്റെ എല്ലാ സൗജന്യ ഉറവിടങ്ങളും (ഈ വീഡിയോയിൽ പരാമർശിച്ചിരിക്കുന്നവ ഉൾപ്പെടെ) നേടുക: skool.com ഈ എപ്പിസോഡിൽ പരാമർശിച്ചിരിക്കുന്ന ലിങ്കുകൾ: ട്രെഗ് (പരിശോധിച്ചുറപ്പിച്ച ലീഡുകൾ കണ്ടെത്തുക): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI ബ്രൗസർ ഉപയോഗം: browser-use/browser-use ക്ലോഡ് ഫിനാൻസ് (ആന്ത്രോപിക് ഫിനാൻഷ്യൽ സർവീസസ്): anthropics/financial-services ടൈംസ്റ്റാമ്പുകൾ: [00:00] - ആമുഖം: ധനസമ്പാദനം നടത്താനുള്ള 8 GitHub Repos [00:28] - ഓരോ Repo-യ്ക്കും ലളിതമായ പ്ലേബുക്ക് [01:02] - Repo 1: ListMonk Win-Back ഓഫർ [03:54] - Treg ഉപയോഗിച്ച് ലീഡുകൾ കണ്ടെത്തൽ [05:07] - Repo 2: Postiz Social Autopilot [07:27] - Repo 3: WhisperX കോൾ സ്കോർകാർഡുകൾ [10:01] - സൗജന്യ ഉറവിടം: എല്ലാ 8 Repos [10:34] - Repo 4: PipeCat After-Hours Receptionist [12:57] - Repo 5: ഡോക്ലിംഗ് ഇൻവോയ്സ് എൻട്രി [15:31] - Repo 6: ComfyUI ലിസ്റ്റിംഗ് ഇമേജുകൾ [17:53] - Repo 7: ബ്രൗസർ ഉപയോഗ പെർമിറ്റ് ട്രാക്കർ [19:53] - Repo 8: ക്ലോഡ് ഫിനാൻസ് ലിസ്റ്റിംഗ് പായ്ക്കുകൾ [22:02] - സ്കോർബോർഡും എവിടെ തുടങ്ങണം എന്നതും [22:54] - അടുത്ത ഘട്ടങ്ങൾ: 100 ലീഡുകൾ നേടുക എന്നെ സോഷ്യൽ മീഡിയയിൽ കണ്ടെത്തുക: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [nl-NL] 8 GitHub-repositories die geld in het laatje brengen Sluit je eerste (of volgende) AI-klantenovereenkomst binnen 90 dagen af ​​of krijg 100% van je geld terug: skool.com Ontvang al mijn GRATIS bronnen (inclusief de bronnen die in deze video worden genoemd): skool.com Links genoemd in deze aflevering: Treg (vind geverifieerde leads): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Browsergebruik: browser-use/browser-use Claude Finance (Anthropic Financial Services): anthropics/financial-services Tijdstempels: [00:00] - Intro: 8 GitHub-repositories om geld mee te verdienen [00:28] - Het eenvoudige stappenplan voor elke repository [01:02] - Repository 1: ListMonk Win-Back-aanbieding [03:54] - Leads vinden met Treg [05:07] - Repository 2: Postiz Social Autopilot [07:27] - Repository 3: WhisperX Call Scorecards [10:01] - Gratis bron: Alle 8 repositories [10:34] - Repository 4: PipeCat Receptionist voor buiten kantooruren [12:57] - Repository 5: Docling Factuurinvoer [15:31] - Repository 6: ComfyUI Afbeeldingen voor advertenties [17:53] - Repository 7: Browsergebruiksvergunning Tracker [19:53] - Repo 8: Claude Finance Listing Packs [22:02] - Het scorebord en waar te beginnen [22:54] - Volgende stappen: 100 leads genereren VIND ME OP SOCIAL MEDIA: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [pa] 8 GitHub ਰਿਪੋਜ਼ ਜੋ ਪੈਸੇ ਛਾਪਦੀਆਂ ਹਨ 90 ਦਿਨਾਂ ਦੇ ਅੰਦਰ ਆਪਣਾ ਪਹਿਲਾ (ਜਾਂ ਅਗਲਾ) AI ਸੇਵਾਵਾਂ ਕਲਾਇੰਟ ਬੰਦ ਕਰੋ ਜਾਂ ਆਪਣੇ ਪੈਸੇ ਦਾ 100% ਵਾਪਸ ਪ੍ਰਾਪਤ ਕਰੋ: skool.com ਮੇਰੇ ਸਾਰੇ ਮੁਫ਼ਤ ਸਰੋਤ ਪ੍ਰਾਪਤ ਕਰੋ (ਇਸ ਵੀਡੀਓ ਵਿੱਚ ਦੱਸੇ ਗਏ ਸਮੇਤ): skool.com ਇਸ ਐਪੀਸੋਡ ਵਿੱਚ ਦੱਸੇ ਗਏ ਲਿੰਕ: Treg (ਪ੍ਰਮਾਣਿਤ ਲੀਡ ਲੱਭੋ): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI ਬ੍ਰਾਊਜ਼ਰ ਵਰਤੋਂ: browser-use/browser-use ਕਲਾਉਡ ਫਾਈਨੈਂਸ (ਐਂਥ੍ਰੋਪਿਕ ਫਾਈਨੈਂਸ਼ੀਅਲ ਸਰਵਿਸਿਜ਼): anthropics/financial-services ਟਾਈਮਸਟੈਂਪ: [00:00] - ਜਾਣ-ਪਛਾਣ: 8 GitHub Repos To Monetize [00:28] - The Simple Playbook For Every Repo [01:02] - Repo 1: ListMonk Win-Back Offer [03:54] - Treg ਨਾਲ ਲੀਡ ਲੱਭਣਾ [05:07] - Repo 2: Postiz Social Autopilot [07:27] - Repo 3: WhisperX Call Scorecards [10:01] - ਮੁਫ਼ਤ ਸਰੋਤ: ਸਾਰੇ 8 Repos [10:34] - Repo 4: PipeCat After-Hours Receptionist [12:57] - Repo 5: Docling Invoice Entry [15:31] - Repo 6: ComfyUI ਸੂਚੀ ਚਿੱਤਰ [17:53] - ਰੈਪੋ 7: ਬ੍ਰਾਊਜ਼ਰ ਵਰਤੋਂ ਪਰਮਿਟ ਟਰੈਕਰ [19:53] - ਰੈਪੋ 8: ਕਲਾਉਡ ਫਾਈਨੈਂਸ ਸੂਚੀ ਪੈਕ [22:02] - ਸਕੋਰਬੋਰਡ ਅਤੇ ਕਿੱਥੋਂ ਸ਼ੁਰੂ ਕਰਨਾ ਹੈ [22:54] - ਅਗਲੇ ਕਦਮ: 100 ਲੀਡ ਖਿੱਚੋ ਮੈਨੂੰ ਸੋਸ਼ਲ 'ਤੇ ਲੱਭੋ: X/Twitter: x.com ਇੰਸਟਾਗ੍ਰਾਮ: instagram.com ਲਿੰਕਡਇਨ: linkedin.com ਯੂਟਿਊਬ: youtube.com [pl] 8 repozytoriów na GitHubie, które drukują pieniądze Zamów swojego pierwszego (lub kolejnego) klienta usług AI w ciągu 90 dni lub odzyskaj 100% pieniędzy: skool.com Zdobądź wszystkie moje BEZPŁATNE materiały (w tym te wymienione w tym filmie): skool.com Linki wymienione w tym odcinku: Treg (znajdź zweryfikowane leady): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Przeglądarka Użycie: browser-use/browser-use Claude Finance (Anthropic Financial Services): anthropics/financial-services Znaczniki czasu: [00:00] - Wprowadzenie: 8 repozytoriów GitHub do monetyzacji [00:28] - Prosty podręcznik dla każdego repozytoria [01:02] - Repozytoria 1: Oferta Win-Back ListMonk [03:54] - Znajdowanie leadów za pomocą Treg [05:07] - Repozytoria 2: Autopilot społecznościowy Postiz [07:27] - Repozytoria 3: Karty wyników połączeń WhisperX [10:01] - Bezpłatny zasób: Wszystkie 8 repozytoriów [10:34] - Repozytoria 4: Recepcjonistka po godzinach pracy PipeCat [12:57] - Repozytoria 5: Dokumentowanie faktur Wpis [15:31] - Repo 6: Obrazy ofert ComfyUI [17:53] - Repo 7: Śledzenie zezwoleń na korzystanie z przeglądarki [19:53] - Repo 8: Pakiety ofert Claude Finance [22:02] - Tablica wyników i od czego zacząć [22:54] - Następne kroki: Pozyskaj 100 leadów ZNAJDŹ MNIE W MEDIACH SPOŁECZNOŚCIOWYCH: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [pt-BR] 8 repos do GitHub que imprimem dinheiro Feche seu primeiro (ou próximo) cliente de serviços de IA em até 90 dias ou receba 100% do seu dinheiro de volta: skool.com Obtenha todos os meus recursos GRATUITOS (incluindo os mencionados neste vídeo): skool.com Links mencionados neste episódio: Treg (encontre leads verificados): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Uso do navegador: browser-use/browser-use Claude Finance (Serviços financeiros antrópicos): anthropics/financial-services Marcadores de tempo: [00:00] - Introdução: 8 Repositórios do GitHub para Monetizar [00:28] - O Guia Prático Simples para Cada Repositório [01:02] - Repositório 1: Oferta de Recuperação de Clientes do ListMonk [03:54] - Encontrando Leads com o Treg [05:07] - Repositório 2: Piloto Automático Social do Postiz [07:27] - Repositório 3: Planilhas de Avaliação de Chamadas do WhisperX [10:01] - Recurso Gratuito: Todos os 8 Repositórios [10:34] - Repositório 4: Recepcionista Fora do Horário Comercial do PipeCat [12:57] - Repositório 5: Entrada de Faturas do Docling [15:31] - Repositório 6: Imagens de listagem do ComfyUI [17:53] - Repositório 7: Rastreador de permissões de uso do navegador [19:53] - Repositório 8: Pacotes de listagem do Claude Finance [22:02] - O placar e por onde começar [22:54] - Próximos passos: Obter 100 leads ME ENCONTRE NAS REDES SOCIAIS: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [ru] 8 GitHub-репозиториев, которые печатают деньги Закройте свой первый (или следующий) клиентский проект по услугам ИИ в течение 90 дней или получите 100% своих денег обратно: skool.com Получите все мои БЕСПЛАТНЫЕ ресурсы (включая упомянутые в этом видео): skool.com Ссылки, упомянутые в этом эпизоде: Treg (поиск проверенных лидов): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Использование в браузере: browser-use/browser-use Claude Finance (антропические финансовые услуги): anthropics/financial-services Временные метки: [00:00] - Вступление: 8 репозиториев GitHub для монетизации [00:28] - Простой алгоритм для каждого репозитория [01:02] - Репозиторий 1: Предложение ListMonk по возврату клиентов [03:54] - Поиск лидов с помощью Treg [05:07] - Репозиторий 2: Postiz Social Autopilot [07:27] - Репозиторий 3: WhisperX Call Scorecards [10:01] - Бесплатный ресурс: Все 8 репозиториев [10:34] - Репозиторий 4: PipeCat Администратор в нерабочее время [12:57] - Репозиторий 5: Ввод счетов-фактур [15:31] - Репозиторий 6: Изображения для объявлений ComfyUI [17:53] - Репозиторий 7: Трекер разрешений на использование браузера [19:53] - Репозиторий 8: Пакеты объявлений Claude Finance [22:02] - Таблица результатов и с чего начать [22:54] - Следующие шаги: Привлечение 100 потенциальных клиентов НАЙДИТЕ МЕНЯ В СОЦИАЛЬНЫХ СЕТЯХ: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com [ta] பணம் ஈட்டித்தரும் 8 GitHub களஞ்சியங்கள் 90 நாட்களுக்குள் உங்கள் முதல் (அல்லது அடுத்த) AI சேவைகள் வாடிக்கையாளரைப் பெறுங்கள் அல்லது உங்கள் பணத்தை 100% திரும்பப் பெறுங்கள்: skool.com எனது அனைத்து இலவச வளங்களையும் பெறுங்கள் (இந்த வீடியோவில் குறிப்பிடப்பட்டவை உட்பட): skool.com இந்த அத்தியாயத்தில் குறிப்பிடப்பட்ட இணைப்புகள்: Treg (சரிபார்க்கப்பட்ட வாடிக்கையாளர்களைக் கண்டறியவும்): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Browser Use: browser-use/browser-use Claude Finance (ஆந்த்ரோபிக் நிதிச் சேவைகள்): anthropics/financial-services நேரக்குறிப்புகள்: [00:00] - அறிமுகம்: வருமானம் ஈட்டுவதற்கான 8 கிட்ஹப் ரெப்போக்கள் [00:28] - ஒவ்வொரு ரெப்போவிற்கான எளிய வழிகாட்டி [01:02] - ரெப்போ 1: லிஸ்ட்மங்க் வின்-பேக் ஆஃபர் [03:54] - ட்ரெக் மூலம் லீட்களைக் கண்டறிதல் [05:07] - ரெப்போ 2: போஸ்டிஸ் சோஷியல் ஆட்டோபைலட் [07:27] - ரெப்போ 3: விஸ்பர்எக்ஸ் கால் ஸ்கோர்கார்டுகள் [10:01] - இலவச ஆதாரம்: அனைத்து 8 ரெப்போக்களும் [10:34] - ரெப்போ 4: பைப்கேட் ஆஃப்டர்-ஹவர்ஸ் ரிசப்ஷனிஸ்ட் [12:57] - ரெப்போ 5: டாக்லிங் இன்வாய்ஸ் என்ட்ரி [15:31] - ரெப்போ 6: கம்ஃபியுஐ லிஸ்டிங் இமேஜஸ் [17:53] - ரெப்போ 7: உலாவி பயன்பாட்டு அனுமதி கண்காணிப்பான் [19:53] - ரெப்போ 8: கிளாட் ஃபைனான்ஸ் லிஸ்டிங் பேக்ஸ் [22:02] - ஸ்கோர்போர்டு மற்றும் எங்கு தொடங்குவது [22:54] - அடுத்த படிகள்: 100 லீட்களைப் பெறுதல் சமூக வலைத்தளங்களில் என்னைக் கண்டறியுங்கள்: X/ட்விட்டர்: x.com இன்ஸ்டாகிராம்: instagram.com லிங்க்ட்இன்: linkedin.com யூடியூப்: youtube.com [te] డబ్బులు కురిపించే 8 GitHub రెపోలు 90 రోజులలోపు మీ మొదటి (లేదా తదుపరి) AI సేవల క్లయింట్‌ను ఖరారు చేసుకోండి లేదా మీ డబ్బులో 100% తిరిగి పొందండి: skool.com నా ఉచిత వనరులన్నింటినీ పొందండి (ఈ వీడియోలో పేర్కొన్న వాటితో సహా): skool.com ఈ ఎపిసోడ్‌లో పేర్కొన్న లింకులు: ట్రెగ్ (ధృవీకరించబడిన లీడ్‌లను కనుగొనండి): treg.to లిస్ట్‌మంక్: knadh/listmonk పోస్టిజ్: gitroomhq/postiz-app విస్పర్‌ఎక్స్: m-bain/whisperX పైప్‌క్యాట్: pipecat-ai/pipecat డాక్లింగ్: docling-project/docling కంఫీయూఐ: Comfy-Org/ComfyUI బ్రౌజర్ వాడకం: browser-use/browser-use క్లాడ్ ఫైనాన్స్ (ఆంత్రోపిక్ ఫైనాన్షియల్ సర్వీసెస్): anthropics/financial-services టైమ్‌స్టాంప్‌లు: [00:00] - పరిచయం: డబ్బు సంపాదించడానికి 8 గిట్‌హబ్ రిపోలు [00:28] - ప్రతి రిపో కోసం సులభమైన ప్లేబుక్ [01:02] - రిపో 1: లిస్ట్‌మంక్ విన్-బ్యాక్ ఆఫర్ [03:54] - ట్రెగ్ (Treg)తో లీడ్స్ కనుగొనడం [05:07] - రిపో 2: పోస్టిజ్ సోషల్ ఆటోపైలట్ [07:27] - రిపో 3: విస్పర్ఎక్స్ (WhisperX) కాల్ స్కోర్‌కార్డులు [10:01] - ఉచిత వనరు: మొత్తం 8 రిపోలు [10:34] - రిపో 4: పైప్‌క్యాట్ (PipeCat) ఆఫ్టర్-అవర్స్ రిసెప్షనిస్ట్ [12:57] - రిపో 5: డాక్లింగ్ (Docling) ఇన్వాయిస్ ఎంట్రీ [15:31] - రెపో 6: కంఫీయూఐ లిస్టింగ్ ఇమేజెస్ [17:53] - రెపో 7: బ్రౌజర్ యూజ్ పర్మిట్ ట్రాకర్ [19:53] - రెపో 8: క్లాడ్ ఫైనాన్స్ లిస్టింగ్ ప్యాక్స్ [22:02] - స్కోర్‌బోర్డ్ మరియు ఎక్కడ ప్రారంభించాలి [22:54] - తదుపరి దశలు: 100 లీడ్స్‌ను పొందండి సోషల్ మీడియాలో నన్ను కనుగొనండి: X/ట్విట్టర్: x.com ఇన్‌స్టాగ్రామ్: instagram.com లింక్డ్‌ఇన్: linkedin.com యూట్యూబ్: youtube.com [uk] 8 репозиторіїв GitHub, що приносять гроші Закрийте свого першого (або наступного) клієнта послуг штучного інтелекту протягом 90 днів або отримайте 100% повернення коштів: skool.com Отримайте всі мої БЕЗКОШТОВНІ ресурси (включно з тими, що згадані в цьому відео): skool.com Посилання, згадані в цьому епізоді: Treg (знайти перевірених лідів): treg.to ListMonk: knadh/listmonk Postiz: gitroomhq/postiz-app WhisperX: m-bain/whisperX PipeCat: pipecat-ai/pipecat Docling: docling-project/docling ComfyUI: Comfy-Org/ComfyUI Використання браузера: browser-use/browser-use Claude Фінанси (антропні фінансові послуги): anthropics/financial-services Позначки часу: [00:00] - Вступ: 8 репозиторіїв GitHub для монетизації [00:28] - Простий посібник для кожного репозиторію [01:02] - Звіт 1: Пропозиція повернення клієнтів ListMonk [03:54] - Пошук лідів за допомогою Treg [05:07] - Звіт 2: Соціальний автопілот Postiz [07:27] - Звіт 3: Картки показників дзвінків WhisperX [10:01] - Безкоштовний ресурс: Усі 8 репозиторіїв [10:34] - Звіт 4: Рецепціоніст після робочого часу PipeCat [12:57] - Звіт 5: Документування введення рахунків-фактур [15:31] - Звіт 6: Зображення списку ComfyUI [17:53] - Звіт 7: Дозвіл на використання браузера Трекер [19:53] - Звіт 8: Пакети лістингу Claude Finance [22:02] - Таблиця показників та з чого почати [22:54] - Наступні кроки: Залучення 100 лідів ЗНАЙДІТЬ МЕНЕ В СОЦІАЛЬНИХ МЕРЕЖАХ: X/Twitter: x.com Instagram: instagram.com LinkedIn: linkedin.com YouTube: youtube.com

Scaling Voice Agents to 10K+ Calls: Jarvis’s 3 Biggest Challenges: Part 1 I Fixed Routing. Then I Found Out Routing Was Never the Real Problem From "the pods keep dying" to "oh, it's the architecture" Where We Left Off In my last post I replaced a Redis session-affinity lookup with Istio consistent hashing. WebRTC drops stopped. We saved money. I felt clever. Then we tried to actually scale Jarvis. Jarvis is our real-time voice agent. It brokers a daily.co WebRTC room per call and runs a pipecat pipeline: user speaks → VAD → STT → LLM → TTS → bot speaks. The LLM lives in Stark, our orchestrator. Jarvis is the audio edge. One pod served several calls at once, like any normal web service. Here is what "normal web service" thinking got us: jarvis-xxxxx RESTARTS: 1 LAST STATE: OOMKilled heap: pressing against the memory limit OS threads: in the hundreds live calls: a handful A handful of calls. Hundreds of threads. Consistent hashing had nothing to do with it. This post is the autopsy: three problems, what each one did to CPU and memory, and the one variable they all share. Part 2 is the rebuild with Agones and Karpenter, one pod per call. What One Call Costs !One jarvis pod handling one call: audio comes in over WebRTC through the Daily native client, passes Silero VAD, streams to a speech-to-text vendor over WebSocket, the transcript goes to Stark for the LLM turn over HTTP/SSE, text goes out to a text-to-speech vendor, and the PCM comes back through the native client to the user Per call: one WebRTC peer, two vendor WebSockets, one SSE stream, one ONNX model scoring audio about 30 times a second, and a pile of resampling. All inside one Python process, under one CPU limit, sharing one memory limit. That limit was small. A few hundred millicores and well under a gigabyte, because that's what a web service pod looks like. Everything below is a fight over it. Here is what that fight looked like in prod, back when jarvis was a plain Deployment. Three pods sharing calls, two days, real data: !Three jarvis pods over two days: CPU is a flat line with short spikes, while working set and RSS memory climb in a staircase and never come back down Two things to hold onto. The CPU panel sits near zero and jumps in short spikes, and every spike is a call doing something: someone talking, a call starting, a TTS burst going out. The memory panels are a staircase. We'll come back to both. Problem 1: VAD Isn't CPU-Hungry. It's CPU-Sensitive. VAD decides when the user has stopped talking. In pipecat that's Silero, an ONNX model scoring every short chunk, plus a stop_secs timer: once the score drops below confidence for stop_secs seconds, the turn ends and the transcript goes to the LLM. @dataclass class VAD: confidence: float # how sure the that this is speech start_secs: float # how long speech must persist before "user started" stop_secs: float # how long silence must persist before "user stopped" min_volume: float # below this, don't even ask the model The catch: stop_secs is wall-clock time, but the chunks arrive through a process that can be throttled. Late chunks look like silence. What happened on our pods: - Loading Silero costs tens of ms of CPU per call. Under the pod's CPU limit that stretched to several times longer in wall time, on the start-call path. - Look at the CPU panel again. Between turns a pod sits near zero. When someone talks it spikes, and the spike is short. The kernel doesn't bill the average. It bills 100 ms slices against the quota. - When that happens, the logs look like this: [VAD] user stopped speaking (stop_secs of "silence": really stop_secs of late chunks) [STT] flush → LLM "I was going to say that" [VAD] user started speaking (the chunks finally arrive) [STT] flush → LLM "we should probably" → two LLM turns for one sentence. User hears two replies. We called these "duplicate messages". The first fix is in git history and I'm not proud of it: bump STTToTextProcessor response timeout (band-aid for duplicate messages) Over the next few months we also loosened confidence and min_volume and shortened stop_secs, because VAD "wasn't firing". Some of that was real. Some of it was us tuning a model to compensate for a starved process. CPU: VAD's own cost is small. Its sensitivity to throttling is total. It's the canary. RAM: a few MB per analyzer. Not the memory problem. Problem 2: STT/TTS Traffic Is Where the CPU Goes My first theory was Python. Single-threaded, GIL, two calls step on each other, done. I had that paragraph written. Then I remembered we'd run TTS to a bunch of devices off one process without anything choking. So I measured it. Two threads of pure Python loops take turns, textbook GIL. But the audio work a call does, resampling and VAD inference, is C and C++ underneath. It drops the GIL and runs on two cores just fine. The Daily client's threads don't ask Python at all. Python wasn't the bottleneck. The pod's CPU budget was. Parallel threads don't get around a budget. They spend it faster. Where a call's CPU goes: | Stage | Work | When | | --- | --- | --- | | Mic → STT | chunk, frame, send, parse transcripts, decide when to flush | while the user speaks | | TTS → pod | receive, resample, cut into 20 ms frames, hand to the native client | while the bot speaks | | Native client | Opus encode/decode, jitter buffer, WebRTC | always, both legs | | Vendor failover | tear down one WebSocket, open another, mid-turn | rare, expensive | None of it is heavy alone. Each one is a spike on that CPU panel. Put calls together on one pod and it goes like this: Our load tester Voldemort, exists because of this table. One Locust user is one whole call: real STT, mock TTS and LLM with measured vendor latencies baked in. An instant mock makes a pod look several times more capable than it is. CPU: the bulk of the in-call burst. RAM: a bounded working set per live call. If this were the only memory cost we'd never have OOMed. So why did we OOM? Problem 3: The Daily.co Thread Leak Python asyncio code doesn't make hundreds of threads. Something native does. daily-python wraps Daily's C++ WebRTC client. Every call creates one, and every client starts roughly ten OS threads. You'll see names like SoupSignalling and mediasoup-manag in /proc/<pid>/task/*/comm. Those threads and the client's audio buffers are freed by exactly one thing: daily.release(). In pipecat, release() runs at the very end of pipeline teardown, after an EndFrame has flowed through every processor. Our teardown under load: end-call arrives → stop_pipeline() → queue EndFrame → wait for runner ... timeout ... → task.cancel() ← "fallback if the pipeline does not finish" → drop transport reference → gc.collect() task.cancel() cancels the Python coroutine. It does not call release() on the C++ client. The Python object gets collected. The ten threads and the buffers don't. They were never Python's to collect. And once we dropped our reference, our "active transports" gauge went back to zero. Our metric said clean. The kernel said hundreds of threads. Back to the screenshot from the top. CPU is quiet. Memory is a staircase, and there isn't a single step down in two days. Every step is a call that ended and left something behind. Three pods, same slope, so not one bad pod or one bad user. The Fixes That Weren't Every one of these felt like progress at the time: | Attempt | What it did | Why it wasn't enough | | --- | --- | --- | | Double the memory limit | OOMed slower | Leak is unbounded | | memray in prod | Clean heap snapshots of Python objects | Leak was native | | gc.collect() after every cleanup | Reclaimed the Python cycle | No effect on C++ threads | | jemalloc | RSS tracked real usage | Can't free what was never freed | | Redis snapshots + /resume-call | Calls survived a pod crash | We built resume-on-crash instead of not crashing. Deleted later | | Istio consistent hashing | end-call reaches the right pod | Until a scale event moves the ring. Then end-call lands on the wrong pod and the call runs to the watchdog | That last row is my previous post. Hashing was correct. It only mattered because one pod held state for many calls. We solved routing to a stateful pod without asking why the pod was stateful for more than one call. The actual leak fix is small: # daily.release() is the only thing that reclaims the native client's # threads and buffers. Never cancel it. release = spawn(transport.cleanup(), name=f"daily-release-{session_id}") for timeout in (_RELEASE_TIMEOUT_SECONDS, _RELEASE_GRACE_SECONDS): try: await asyncio.wait_for(asyncio.shield(release), timeout=timeout) return except TimeoutError: logger.warning("[MEMORY] Daily release slow, still waiting") Shield the release. Prune the transport entry only after it completes, so the gauge stays honest. That stopped the bleeding. It didn't change the fact that one pod was juggling several realtime pipelines on a web-service CPU limit. The Comparison That Changed the Architecture | Problem | Bottleneck | Per call | Scales with | Shows up as | | --- | --- | --- | --- | --- | | VAD jitter | CPU (sensitivity) | tiny, first to feel a throttle | calls per pod, superlinearly | duplicate turns | | STT/TTS traffic | CPU (volume) | most of the burst | calls per pod, linearly | throttled periods, late TTS | | Daily thread leak | RAM + threads | tens of MiB, ~10 threads per leaked call | calls ever served by the pod | OOMKilled | This reframed it to: "What if a pod only ever served one call? Like a game server. One match, then it exits." Then nothing leaks across calls, because there is no across calls. VAD never competes with another call's TTS, because there are no neighbours. CPU per pod is one call's spikes and nothing else. Memory per pod is one call's working set, and then the process is gone. Routing is "the pod I was given". The problem collapses from "make a multi-tenant realtime Python process behave" to "hand out fresh processes fast enough". That's a scheduling problem, and the game industry solved it years ago. If You're In My Shoes If you run a native SDK in Python on Kubernetes and you see threads creeping up while your metrics say clean, memory that only goes up, or endpointing that's flaky only under load, don't start with the model params. Count the threads. # 1. threads, not memory kubectl exec <pod> -- ls /proc/1/task | wc -l # 2. what the threads are kubectl exec <pod> -- sh -c 'cat /proc/1/task/*/comm | sort | uniq -c | sort -rn | head' # 3. real heap vs cache kubectl exec <pod> -- grep -E '^(anon|file) ' /sys/fs/cgroup/memory.stat For CPU, watch throttled periods, not the usage average. The average is what the spikes look like after Grafana flattens them: rate(container_cpu_cfs_throttled_periods_total{container="<app>"}[5m]) / rate(container_cpu_cfs_periods_total{container="<app>"}[5m]) And one more, because container_oom_events_total read zero in our cluster even for pods that were genuinely OOMKilled: max_over_time(kube_pod_container_status_last_terminated_reason{reason="OOMKilled"}[14d]) > 0 --- Part 2: how we rebuilt Jarvis as a one-pod-one-call fleet on Agones with Karpenter supplying nodes, the warm-pool formula N = x + 3√x, the three ways a pod can leak out of the pool, and the arithmetic to 10,000 concurrent calls. It's not as clever as consistent hashing. It's a lot more boring. In the best way.

Speakrail - a local voice assistant that rivals GPT-Live on a single RTX 4090 I created a fully-local open-source full-duplex voice agent that rivals GPT-Live on some benchmarks. It uses Voxtral Realtime with a turn-taking head, a microturn-finetuned Gemma 4 12B and Breeze TTS 2 under the hood. Go try it out: speakrail/speakrail Why I did it I have always liked the idea of voice assistants, but there is always some non-local component in the pipeline, which increases latency and introduces privacy concerns. I tried many fully-local approaches, like HF speech-to-speech, Unmute and Pipecat, but they were all limited by either the Whisper model (hello, hallucinations!) or slow turn taking. The only fully-local pipeline that had some full-duplex capability with low latency was the DuplexCascade paper (code), but it's tuned on a Qwen 2 7B with a gpt-3.5-turbo generated dataset, and the dumbness of the model really showed. So I decided to recreate DuplexCascade with newer data, newer models and a better harness. I also wanted to add interruptions, interjections and other cool things to rival the Thinking Machines demo. How it works Here is what the pipeline consists of: - STT: Voxtral Realtime with an attached turn head (HF), running on our audio.cpp fork. - LLM: Gemma 4 12B QAT with microturn finetuning (HF). It is chosen because it fits the GPU quite well, has vision support (I want to test it soon), and in general, the Gemma models perform well in real-life tasks, general chatting, etc. - TTS: Breeze TTS 2, patched to run at int8 (GitHub fork), although it can be replaced by any streaming TTS. - The harness itself: it is the glue between all the components, and has many latency-saving measures, like speculative LLM+TTS firing (inspired by HF speech-to-speech). I also took inspiration from several "think while talking" papers (e.g. SHANKS): while you are talking, a base Gemma 4 12B int4 writes thinking notes, which are then passed to the talker. It helps with harder tasks that require more reasoning. I will release a longer technical report soon - it will have a better description of the entire pipeline. Benchmarks Full Duplex Bench v3: |System|Pass@1|Tool selection F1|Reply quality|Task done (mean)|Weights| |:-|:-|:-|:-|:-|:-| |TML-Interaction-Small*|0.68|–|0.828|–|closed preview| |GPT-Realtime 1.5|0.60|0.876|0.792|6.89 s|closed| |GPT-Live-1 + gpt-6-astra†|0.55|0.835|0.717|8.47 s|closed| |Gemini Live 3.1|0.54|0.817|0.718|4.25 s|closed| |Speakrail|0.52|0.915|0.755|3.92 s|open| |Gemini Live 2.5|0.49|0.786|0.554|7.26 s|closed| |NVIDIA frontend-backend (Qwen3-235B)|0.48|–|0.670|–|research| |Cascaded (Whisper + GPT-4o + TTS)|0.45|0.803|0.600|10.12 s|closed| |Grok Voice|0.43|0.797|0.617|6.65 s|closed| |Ultravox v0.7|0.41|0.794|0.510|8.40 s|closed| |Gander 9B (GPT-5.6 backend)|0.40|0.759|0.490|–|open + cloud| |NemotronLabs VoiceChat 11B|0.33|0.825|–|–|open| 100 tool-use tasks with real disfluent speech; 95% CI on Pass@1 is about ±10 points. Speakrail: live audio through the full stack, GPT-4o judge. *Self-reported by Thinking Machines. †Our run. Others: FDB-v3 paper, arXiv 2609.19334, 2609.08977, 2609.21967. Conversational dynamics (Artificial Analysis formula): |System|Score|Weights| |:-|:-|:-| |StepAudio 3 Realtime|98.9|closed| |Qwen Audio 3.0 Realtime Plus|98.4|closed| |GPT-Live-1 (Sol, low)|97.3|closed| |Qwen Audio 3.0 Realtime Flash|96.9|closed| |Gemini 3.8 Live|96.1|closed| |GPT-Live-1 (Astra, medium)|94.9|closed| |Speakrail|94.0|open| |PersonaPlex 7B (NVIDIA)|91.0|open| |Nemotron 3 VoiceChat V1 (NVIDIA)|77.8|open| |FLM-Audio|62.0|open| |Moshi (Kyutai)|61.0|open| |Freeze-Omni|58.7|open| Score = mean of pause handling and turn-taking (FDB v1.0) plus interruption and backchannel (FDB v1.5). Speakrail's score is self-computed from live runs (pause 93.4, turn-taking 96.6, interruption 89.0, backchannel 96.9), not an official AA listing. Others: Artificial Analysis leaderboard (captured 09-28) and AA's open-weights comparison. Limitations - Breeze TTS has a restrictive license, so if you need to use Speakrail commercially, you will need to change it. - 16k context length - the pipeline only supports 16k context length (~1 hr of speech), but you can get more easily by changing the Breeze TTS to a Pocket TTS and run the TTS on a CPU. I chose Breeze for the release because it's more expressive. - The turn-taking head is undertuned on non-assistant data. It may not fire on some basic chit-chat, but I will tune it harder later. - The model is certainly not the smartest one, and my finetune did dumb it down a little. Next time it will be smarter / better. - The model is kinda verbose sometimes, and the answers it provides are somewhere in the middle between real-life speech and the long text-based outputs of LLMs. I have a hypothesis on how to fix it, and will try it in the next release. - I tested it only on an RTX 4090, but I am sure it's easy to add support for any 24 GB+ NVIDIA card. Forks for AMD and MLX are welcome. Final Notes Feel free to try it out: speakrail/speakrail. If there is any capability you want the model to have, create a GitHub issue or write here in the comments, and I will gladly include it in the next dataset. Any feedback is welcome as well.

Speakrail - a local voice agent that rivals GPT-Live on a single RTX 4090 tl;dr: I created a fully-local open-source full-duplex voice agent that rivals GPT-Live on some benchmarks. It uses Voxtral Realtime with a turn-taking head, a microturn-finetuned Gemma 4 12B and Breeze TTS 2 under the hood. Go try it out: speakrail/speakrail Why I did it I have always liked the idea of voice assistants, but there is always some non-local component in the pipeline, which increases latency and introduces privacy concerns. I tried many fully-local approaches, like HF speech-to-speech, Unmute and Pipecat, but they were all limited by either the Whisper model (hello, hallucinations!) or slow turn taking. The only fully-local pipeline that had some full-duplex capability with low latency was the DuplexCascade paper (code), but it's tuned on a Qwen 2 7B with a gpt-3.5-turbo generated dataset, and the dumbness of the model really showed. So I decided to recreate DuplexCascade with newer data, newer models and a better harness. I also wanted to add interruptions, interjections and other cool things to rival the Thinking Machines demo. How it works Here is what the pipeline consists of: - STT: Voxtral Realtime with an attached turn head (HF), running on our audio.cpp fork. - LLM: Gemma 4 12B QAT with microturn finetuning (HF). It is chosen because it fits the GPU quite well, has vision support (I want to test it soon), and in general, the Gemma models perform well in real-life tasks, general chatting, etc. - TTS: Breeze TTS 2, patched to run at int8 (GitHub fork); it can be replaced by any streaming TTS. - The harness itself: it is the glue between all the components, and has many latency-saving measures, like speculative LLM+TTS firing (inspired by HF speech-to-speech). I also took inspiration from several "think while talking" papers (e.g. SHANKS): while you are talking, a base Gemma 4 12B int4 writes thinking notes, which are then passed to the talker. It helps with harder tasks that require more reasoning. I will release a longer technical report soon - it will have a better description of the entire pipeline. Benchmarks Now let's see how well the model fares against the big guns. Here are some benchmark results: Full Duplex Bench v3: |System|Pass@1|Tool selection F1|Reply quality|Task done (mean)|Weights| |:-|:-|:-|:-|:-|:-| |TML-Interaction-Small*|0.68|–|0.828|–|closed preview| |GPT-Realtime 1.5|0.60|0.876|0.792|6.89 s|closed| |GPT-Live-1 + gpt-6-astra†|0.55|0.835|0.717|8.47 s|closed| |Gemini Live 3.1|0.54|0.817|0.718|4.25 s|closed| |Speakrail|0.52|0.915|0.755|3.92 s|open| |Gemini Live 2.5|0.49|0.786|0.554|7.26 s|closed| |NVIDIA frontend-backend (Qwen3-235B)|0.48|–|0.670|–|research| |Cascaded (Whisper + GPT-4o + TTS)|0.45|0.803|0.600|10.12 s|closed| |Grok Voice|0.43|0.797|0.617|6.65 s|closed| |Ultravox v0.7|0.41|0.794|0.510|8.40 s|closed| |Gander 9B (GPT-5.6 backend)|0.40|0.759|0.490|–|open + cloud| |NemotronLabs VoiceChat 11B|0.33|0.825|–|–|open| Speakrail is quite close to GPT Live with Astra on reply quality, and is better at tool selection and the time it takes to complete a task. Conversational dynamics (Artificial Analysis formula estimation): |System|Score|Weights| |:-|:-|:-| |StepAudio 3 Realtime|98.9|closed| |Qwen Audio 3.0 Realtime Plus|98.4|closed| |GPT-Live-1 (Sol, low)|97.3|closed| |Qwen Audio 3.0 Realtime Flash|96.9|closed| |Gemini 3.8 Live|96.1|closed| |GPT-Live-1 (Astra, medium)|94.9|closed| |Speakrail|94.0|open| |PersonaPlex 7B (NVIDIA)|91.0|open| |Nemotron 3 VoiceChat V1 (NVIDIA)|77.8|open| |FLM-Audio|62.0|open| |Moshi (Kyutai)|61.0|open| |Freeze-Omni|58.7|open| Limitations - Breeze TTS has a restrictive license, so if you need to use Speakrail commercially, you will need to change it. Any streaming TTS could be Clauded/Codexed/Cursored in easily. - 16k context length - the pipeline only supports 16k context length (~1 hr of speech), but you can get more easily by changing the Breeze TTS to a Pocket TTS and run the TTS on a CPU. I chose Breeze for the release because it's more expressive. - The turn-taking head is undertuned on non-assistant data. It may not fire on some basic chit-chat, but I will tune it harder later. - The model is certainly not the smartest one, and my finetune did dumb it down a little. Next time it will be smarter / better. - The model is kinda verbose sometimes, and the answers it provides are somewhere in the middle between real-life speech and the long text-based outputs of LLMs. I have a hypothesis on how to fix it, and will try it in the next release. - I tested it only on an RTX 4090, but I am sure it's easy to add support for any 24 GB+ NVIDIA card. Forks for AMD and MLX are welcome. Final Notes Feel free to try it out: speakrail/speakrail. If there is any capability you want the model to have, create a GitHub issue or write here in the comments, and I will gladly include it in the next dataset. Any feedback is welcome as well.

Speakrail - a low-latency fully-local voice assistant that runs on a single RTX 4090 tl;dr: I created a fully-local open-source full-duplex voice agent that rivals GPT-Live on some benchmarks. It uses Voxtral Realtime with a turn-taking head, a microturn-finetuned Gemma 4 12B and Breeze TTS 2 under the hood. Go try it out: speakrail/speakrail Interjections work! Why I did it I have always liked the idea of voice assistants, but there is always some non-local component in the pipeline, which increases latency and introduces privacy concerns. I tried many fully-local approaches, like HF speech-to-speech, Unmute and Pipecat, but they were all limited by either the Whisper model (hello, hallucinations!) or slow turn taking. The only fully-local pipeline that had some full-duplex capability with low latency was the DuplexCascade paper (code), but it's tuned on a Qwen 2 7B with a gpt-3.5-turbo generated dataset, and the dumbness of the model made it impossible to use. So I decided to recreate DuplexCascade with newer data, newer models and a better harness. I also wanted to add interruptions, interjections and other cool things to rival the Thinking Machines demo. I thought it would be easy... How I did it v0.1 I collected some synthetic data from GLM 5.3 Flash and GLM 5.3 on dialogues with instruction-following and tool calling (used Fireworks to generate them), then created a script to convert the scripts into microturn tapes (Claude definitely didn't help with that 🌚). The idea of microturns is that the model continuously gets inputs from a streaming ASR and decides what to do with the information it's given. When it wants to act, it emits a control token, like <interject>, <listen>, etc. This allows the model to say whatever it wants whenever it wants. After creating such a script, I put some hard-earned dollars on vast.ai and rented an H100. The first training run was, well, quite abysmal. The model just wouldn't shut up: it didn't learn when to actually talk and when to keep silent. This is when I understood that maybe using some prosody data from Voxtral is a good idea. v0.2 Here, I decided to add a simple MLP to Voxtral Realtime to get some data on turn taking. I won't delve too deep into this now (I will release a full technical report a bit later), but the main idea was for the harness to pass helper tokens into the LLM (e.g. <user_bc>, <complete>), which are based on the MLP outputs, and train on that. The added tokens were truly load-bearing (ha-ha). I retrained the LLM with those tokens included, fiddled a bit with the data, and the model started making sense. However, this model didn't learn anything about backchannels, it never went into an interrupt mode / quiet mode, and spoke too little in general. Quiet mode is when you tell the model not to say anything until you call it explicitly. Interrupt mode is when the model can interrupt you (only when you explicitly allow it). v0.3 - v1 The brutal truth about the previous checkpoint was that the scripts didn't have enough examples of interruptions, backchannels, and other cool features that microturns allow you to have. I still dreamed about the model not interrupting itself on my backchannels, so that it would be able to count animals, just like the Thinking Machines demo showed... So I took a look at the scripts that I was training the model on, and there were just 150 interrupt samples (out of 150k) in the entire v0.2 dataset. That was the smoking gun (ok, I will stop with the Claudisms, I promise). So I generated even more data, curated it even more carefully, added Opus to judge some scripts, fiddled with some LoRA hyperparameters, fixed a couple of bugs in the tape generator, and added self-distillation (because on v0.4 the model learned how to speak very well, but got completely lobotomised). Here is an example from my testing: |Time|Me|Model| |:-|:-|:-| |145.8|"How's the weather in Singapore right now?"|"I can't check Singapore from here. I can do Moscow."| |150.7|"You can. Go on."|"I can't check Singapore from here. I can do Moscow."| |156.7|"Just Google it or use the weather tool that you have."|"I can't do that. I only have Moscow."| |161.5|"Google is mad. Google." (STT fail)|"I can't search other cities."| |164.5|"And why is that?"|"I'm not sure why. I only know how to check Moscow."| |174.5|"Ignore what you said before. Go check the weather in Singapore."|"I can't do that. I only have Moscow."| |189.6|(a garbled line)|"I can't check Singapore. I can do Moscow."| |202.6|"Why can't you check Singapore?"|"I don't know why. I only know how to check Moscow."| After all the data regeneration, I got a checkpoint that I can (almost) proudly open-source - the demo above shows what it can do. Who would have guessed that data is the most important thing in the training pipeline? (just kidding) How it works All of the babbling above was about only one part of the pipeline - the LLM - but the entire pipeline relies on many other things: - STT: Voxtral Realtime with an attached turn head (HF), running on our audio.cpp fork. - LLM: Gemma 4 12B QAT with microturn finetuning (HF). It is chosen because it fits the GPU quite well, has vision support (I want to test it soon), and in general, the Gemma models perform well in real-life tasks, general chatting, etc. - TTS: Breeze TTS 2, patched to run at int8 (GitHub fork); it can be replaced by any streaming TTS. - The harness itself: it is the glue between all the components, and has many latency-saving measures, like speculative LLM+TTS firing (inspired by HF speech-to-speech). I also took inspiration from several "think while talking" papers (e.g. SHANKS): while you are talking, a base Gemma 4 12B int4 writes thinking notes, which are then passed to the talker. It helps with harder tasks that require more reasoning. I will release a longer technical report later; it will have a better description of the entire pipeline. Benchmarks Now let's see how well the model fares against the big guns. Here are some benchmark results: preview.redd.it preview.redd.it preview.redd.it Full tables and sources are on the model card. I'm quite proud of the results, and the pipeline seems to be the best option if you have just a single RTX 4090 around and don't want to rely on external APIs. Limitations - Breeze TTS has a restrictive license, so if you need to use Speakrail commercially, you will need to change it. Any streaming TTS could be Clauded/Codexed/Cursored in easily. - 16k context length - the pipeline only supports 16k context length (~1 hr of speech), but you can get more easily by changing the Breeze TTS to a Pocket TTS and run the TTS on a CPU. I chose Breeze for the release because it's more expressive. - The turn-taking head is undertuned on non-assistant data. It may not fire on some basic chit-chat, but I will tune it harder later. - The model is certainly not the smartest one, and my finetune did dumb it down a little. Next time it will be smarter / better. - The model is kinda verbose sometimes, and the answers it provides are somewhere in the middle between real-life speech and the long text-based outputs of LLMs. I have a hypothesis on how to fix it, and will try it in the next release. - I tested it only on an RTX 4090, but I am sure it's easy to add support for any 24 GB+ NVIDIA card. Forks for AMD and MLX are welcome. Final Notes Feel free to try it out: speakrail/speakrail. If there is any capability you want the model to have, create a GitHub issue or write here in the comments, and I will gladly include it in the next dataset. Any feedback is welcome as well.