FluidInference

The new 18M / 7M / 2M Pocket TTS models now run on coreml in FluidAudio, open source, with the flow model on ANE. M5 Pro: 18M at 24 rtfx, 2M at 51 rtfx, TTFT ~25 ms after, under 110 MB of memory. macOS 15 / iOS 18. Model: huggingface.co Code: FluidInference/FluidAudio Great work @kyutai_labs 🙏

FluidAudio: Frontier CoreML audio models in your apps: text-to-speech, speech-to-text, voice activity detection, and speaker diarization. In Swift, powered by SOTA open source. (3.1k stars) What it does All text below is quoted directly from the project README. >FluidAudio is a Swift SDK for fully local, low-latency audio AI on Apple devices, with inference offloaded to the Apple Neural Engine (ANE), resulting in less memory and generally faster inference. >The SDK includes state-of-the-art speaker diarization, transcription, and voice activity detection via open-source models (MIT/Apache 2.0) that can be integrated with just a few lines of code. Models are optimized for background processing, ambient computing and always on workloads by running inference on the ANE, minimizing CPU usage and avoiding GPU/MPS entirely. Highlights: - Automatic Speech Recognition (ASR): Parakeet TDT v3 (0.6b) and other TDT/CTC models for batch transcription supporting 25 European languages and Japanese, plus SenseVoice and Paraformer for Mandarin Chinese; Parakeet EOU (120m) for streaming ASR with end-of-utterance detection (English only). - Text-to-Speech (TTS): Kokoro (82m) for parallel synthesis with SSML and pronunciation control across 9 languages (EN, ES, FR, HI, IT, JA, PT, ZH); PocketTTS for streaming TTS with voice cloning support (EN, DE, ES, FR, IT, PT, 6L and 24L variants); Chatterbox Multilingual (520M, 18 languages) and Chatterbox Nano (110M, English with [laugh]/[chuckle] paralinguistic tags) in beta; Paradee (8M Kokoro distill, English, 12 MB) in beta - Speaker Diarization (Online + Offline): Speaker separation and identification across audio streams. Streaming pipeline for real-time processing and offline batch pipeline with advanced clustering. - Speaker Embedding Extraction: Generate speaker embeddings for voice comparison and clustering, you can use this for speaker identification - Voice Activity Detection (VAD): Voice activity detection with Silero models - Speech Enhancement (AEC + Noise Suppression): LocalVQE (4.8M) removes loudspeaker echo, noise and reverb from 16 kHz mic audio given a far-end reference; streaming with 16 ms latency (beta) - Apple Neural Engine: Models run efficiently on Apple's ANE for maximum performance with minimal power consumption - Open-Source Models: All models are publicly available on HuggingFace, converted and optimized by our team; permissive licenses. >Add FluidAudio to your project using Swift Package Manager. Repository FluidInference/FluidAudio Repository stats |Metric|Value| |:-|:-| |Stars|3,079| |Forks|472| |Last release|v0.17.7 (2026-10-08)| |Languages|Swift 97.6%, Python 1.1%| |Contributors|122| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|claude (22 commits), Copilot (3)| |AI co-authored commits|Claude: 23, Copilot: 3| Main contributor |Field|Value| |:-|:-| |Name|A W| |GitHub|@Alex-Wengg| |Commits to this repo|403 (top contributor)| |Bio|AI Researcher| |X / Twitter|@Alex_tra_memory| |Followers|86|

Inspired by Shahir's demo, FluidAudio Phonon 2 coreml transcribing an hour of audio in 5.6 s and a full 24 hours in 2 min 19 s on a M5 Pro. Phonon-2 Core ML 1 hour rtfx went from 480× → 644×. test-clean wer the same while 150 -> 195 rtfx FluidInference/FluidAudio huggingface.co

How I benchmarked on-device speech-to-text on Apple silicon (Whisper vs Parakeet) --- title: How I benchmarked on-device speech-to-text on Apple silicon (Whisper vs Parakeet) published: true tags: swift, machinelearning, macos, opensource --- I build Plainsay, an open-source dictation app for the Mac. You hold a key, speak, release, and the text is pasted at your cursor. It can run two speech engines entirely on the Mac: OpenAI's Whisper large-v3-turbo through WhisperKit, and NVIDIA's Parakeet TDT 0.6B v3 through FluidAudio. Both run on Core ML. When I had to choose a default, I found plenty of claims about which model is "faster" or "more accurate", and very few measurements I could rerun. So I wrote a small harness, committed the audio and the results, and wrote down what it does and doesn't show. Here's how it works and what I learned from reading the errors one by one. What I wanted to measure Two things matter for dictation: 1. Accuracy, as word error rate (WER): substitutions + deletions + insertions, divided by the number of words in the reference. 2. Latency: how long after you stop talking until the text is ready. In push-to-talk dictation you transcribe the whole clip when the key is released, so that wait is what you feel. I deliberately did not benchmark other dictation apps. Driving someone else's proprietary app in an unattended script isn't something I can do fairly, and I didn't want to republish other vendors' numbers I couldn't reproduce. This measures my own two engines, with enough detail that anyone can check it. The dataset: a small, disclosed sample of LibriSpeech I used LibriSpeech test-clean (CC BY 4.0), downloaded from OpenSLR and checked against its published MD5. The full set has 2,620 utterances from 40 speakers. I used a 50-utterance stratified sample: 10 speakers spread evenly through the sorted speaker list (every 4th), and 5 utterances spread across each speaker's first chapter. That's 1,002 reference words and about 356 seconds of audio. It's small enough that I can read every hypothesis by hand, and it spans 10 different voices. It's also small enough that one or two words move the WER by 0.1 to 0.2 points, which matters later. The 50 FLAC files and their reference transcripts are committed to the repo, so the benchmark runs without downloading the 346 MB archive. The harness The harness is a ~240-line Swift command-line target, Sources/BenchmarkCLI/main.swift. The design choices that matter: - Same engine code as the app. It calls WhisperKitEngine and ParakeetEngine from the app's core module directly, not a reimplementation. Dependencies were pinned (WhisperKit 1.1.0, FluidAudio 0.15.6). - Same audio format as the microphone path. Each FLAC is decoded with AVAudioFile and resampled with AVAudioConverter to 16 kHz mono Float32, which is what the app's recorder produces. All clips are decoded up front, so file I/O isn't timed. - Only transcription is timed. The clock wraps transcribe(samples:prompt:) after the engine has been prepared. Model download and Core ML preparation are outside the timed region. - No hidden warm-up. The first timed clip is the first inference after prepare(), and the same model instance is reused for all 50 clips. I'd rather include the slow first call than quietly drop it. - No prompt. Both engines run with an empty vocabulary prompt. - Simple normalization. Before scoring, both the reference and the hypothesis are lowercased, punctuation except apostrophes is stripped, and whitespace is collapsed. LibriSpeech references are bare uppercase words, and Whisper outputs capitalized, punctuated text, so without this you'd count formatting as errors. - WER is a plain Levenshtein alignment over words. It writes a JSON file with every reference, hypothesis and per-clip latency. Results Run on 2026-08-22, MacBook Air (M4, 10-core CPU, 16 GB), macOS 26.5.1: | Engine | WER | Avg latency | Real-time factor | |---|---:|---:|---:| | Parakeet TDT 0.6B v3 | 2.50% (25 / 1,002) | 0.10 s | 0.014 | | Whisper large-v3-turbo | 2.89% (29 / 1,002) | 0.68 s | 0.095 | Neither engine failed on any clip. The latency gap is the clear result. From the per-clip data, Parakeet's median clip took 0.088 s and its slowest 0.19 s. Whisper's median was 0.64 s and its slowest 1.04 s. The first inference after preparation was 0.13 s for Parakeet and 0.93 s for Whisper, so the cold first call matters a lot more for Whisper. For push-to-talk, Parakeet's result is effectively instant. Whisper's is a short but noticeable pause. The accuracy gap is not a clear result, and reading the errors shows why. Reading the errors With per-clip hypotheses committed, you can see what each counted "error" actually was. A good share of them aren't recognition mistakes: - British vs American spelling, in both directions. Some references use British spelling (HONOUR, ENQUIRE) and both engines wrote "honor" and "inquire". Other references use American spelling (COLOR, CHECK) and Parakeet wrote "colour" and "cheque". Each counts as a full word error. - Abbreviations. Whisper wrote "St. Francis" for SAINT FRANCIS, and "Mr Watson" for MISTER WATSON. - Quotation marks. On two clips Whisper produced nested quotes ("'We wish to talk with him,' answered Kenneth. "'Talk!'). My normalizer keeps apostrophes so "don't" stays one word, which means 'we and him' no longer match the reference. The clip above accounts for 4 of Whisper's 29 errors, and quote artifacts cost it 6 in total. - Rare proper nouns. "Margolotte" became "Margolot", "Donnithorne" became "donnythorne" (Whisper) or "Donny Thorne" (Parakeet, two errors), and "Silvia" became "Sylvia" in both engines. There are real mistakes too. Both engines heard BOOKKEEPER as "bit keeper", and Whisper turned the French word BADAUDERIE into "Bad-Dot-Grey". The takeaway: on clean read speech, the 0.39-point WER difference between these engines is about the same size as the normalization noise. An earlier committed run (Aug 17) had Parakeet at 23 errors instead of 25 on the same clips. Before you conclude one model is more accurate, look at what the errors are. A more complete normalizer (spelling variants, abbreviations, edge apostrophes, like Whisper's English text normalizer) would shrink both numbers. I haven't committed that yet, so I'm not quoting a re-scored figure. What this benchmark doesn't tell you - It's English only. LibriSpeech test-clean says nothing about Polish, German or any other language. Parakeet v3's multilingual coverage and Whisper's language breadth are separate, unmeasured claims. - It's clean audiobook narration. Real dictation means a laptop mic, a room, hesitations, "no wait, I mean". WER will be higher for both engines. - It skips the capture path. Clips are decoded from files, so microphone behavior and voice-activity handling aren't tested. - It's one machine. An M4 MacBook Air. Older M1 machines will be slower, especially for Whisper. - It's a snapshot. The app has since moved to FluidAudio 0.15.8, which fixed dropped endings on some long recordings. Long dictations aren't in this sample at all, since LibriSpeech clips are short. Why Parakeet ended up the recommended default For push-to-talk dictation in English, latency decides a lot: a tenth of a second against two-thirds of a second, on clips of a few seconds. Accuracy on this sample is a tie within noise. Whisper is still there for the languages and accents where it does better, and because some people simply prefer its punctuation style. Run it yourself git clone conrader/plainsay cd plainsay git checkout 2bc1552b19a5ae4094bd81714461baf4ebaa7202 swift build --product BenchmarkCLI .build/debug/BenchmarkCLI Benchmark/data/librispeech-test-clean-manifest.json It prints per-clip progress, then WER, latency and real-time factor per engine, and writes results.json next to the manifest. The full method is in BENCHMARK.md, and the committed results are in Benchmark/data/. If you run it on a different Mac, I'd like to see the numbers, especially from M1/M2 machines. Open an issue or a discussion on the repo. Pull requests that add a better normalizer or a noisier test set are welcome.

Decision 3.0 now on Core ML 🍎 d3-lite triaging 1,000 pull requests into vllm-sr workgroups, live on an M5 Pro: 0.12 s per PR, 8.3 PRs/s, nothing leaves the Mac ⚡ lite, nano and mini are all converted and in one repo 👇 🤗 huggingface.co 🛠️ FluidInference/FluidUse t.co

We distilled clef-flash into a 0.6B model and supporting ANE option. 1,000 support tickets triaged on a M5 pro in 57 s: team, urgency and refund for each, 48 ms per ticket. Within a few points of the 9B on classification. 🤗 huggingface.co 💻 FluidInference/FluidUse t.co

We got EmbeddingGemma 2 running 100% on the Apple Neural Engine 🍎 M5 Pro 10,080 posts sorted into topics live, ~620/s on a MacBook, then any topic splits into subtopics in under a second. Fully on-device. 🤗 huggingface.co 💻 FluidInference/FluidUse t.co