NeMo

NVIDIA NeMo Speech: built for researchers and PyTorch developers working on Speech models (19k stars) What it does All text below is quoted directly from the project README. >A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech) Updates: - 2026-07: MagpieTTS v2607 has been released with support for 3 new languages (Ar, Ko, Pt) + 9 existing languages (En, Es, De, Fr, Vi, It, Zh, Hi, Ja). - 2026-06: Nemotron-3.5-ASR-Streaming-0.6B has been released with 40 languages supported, controllable latency 80ms-1s, and 240-2400 1xH100 concurrent streams. Built on cache-aware Fastconformer architecture. - 2026-04: Parakeet-unified-en-0.6b has been released with high-quality offline and streaming (with a minimum latency of 160ms) inference in one model for English language with punctuation and capitalization support. - 2026: This repo has pivoted to focus on audio, speech, and multimodal LLMs. - 2025-06: Canary-Qwen-2.5B has been released with record-setting 5.63% WER on English Open ASR Leaderboard. >uv pip install 'nemo-toolkit[asr,tts]' Repository NVIDIA-NeMo/Speech Repository stats |Metric|Value| |:-|:-| |Stars|18,655| |Forks|3.7k| |Last release|v3.0.0 (2026-08-07)| |Languages|Python 78.6%, Jupyter Notebook 20.4%, Other 1%| |Contributors|546| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|@claude (41 commits)| |AI co-authored commits|Claude: 42, Codex: 2, Cursor: 4, Copilot: 37| Main contributor |Field|Value| |:-|:-| |Name|Oleksii Kuchaiev| |GitHub|@okuchaiev| |Commits to this repo|883 (top contributor)| |Bio|Director of AI model post-training @ NVIDIA| |Company / Location|Santa Clara, CA| |Followers|317| ────────────────────────────── 📌 If you want to browse more repos like this, we've organized them by category and rankings here: bestgithubrepositories.com 🗂️ We're also archiving repos where possible, so useful projects don't disappear when they're removed from GitHub.

语音识别、合成、语音大模型各找一套框架？ 英伟达的 NeMo Speech 3.0 一把抓：ASR、TTS、语音大模型全在里面，刚发 v3.0.0。 适合做语音应用的研究和开发者，先从官方模型试起。 你会先试哪个，识别还是合成？ #语音 #英伟达 @grok NVIDIA-NeMo/Speech

Turn meeting recordings into speaker-labelled transcripts with Nvidia's Nemotron 3.5 Transcribe Originally published at aifrontierpost.com TL;DR: Nemotron 3.5 Transcribe, released October 28, 2026, is one open-weight model for transcription, speaker labels, word timestamps, translation, and audio questions — steered by prompts, not by chaining tools. This guide serves it with vLLM on one H100, prepares audio at 16 kHz mono, and turns its output into speaker-labelled notes and subtitles. Verified without a GPU where that was possible: audio preparation, the client, and both output formats against a mock server; the serve command itself is Nvidia's, quoted verbatim from the model card. - Who: Nvidia Nemotron 3.5 Transcribe (October 28, 2026) — 34B total parameters, about 4.3B active per token, 65,536-token context, recordings up to one hour, OpenMDW-1.1 licence. - What: a vLLM server on one H100, an OpenAI-compatible client, and two output builders: speaker-labelled Markdown notes and a timed SRT file. - When: verified October 29, 2026: ffmpeg 8.1.2 audio preparation, openai 3.41.0 client against a mock server, Markdown and SRT outputs byte-checked. - Requirement: one supported GPU (H100 80 GB reference; see the hardware table in Step 1), Linux or WSL2, Python 3.13, uv, ffmpeg. No API key for the local server. Why this matters now Meeting transcription used to mean chaining tools together: one model to turn speech into text, another to guess who was speaking, a third to translate, and a pile of glue code to reconcile three outputs that disagreed about where sentences ended. Every handoff was a place for errors to creep in — a speaker change lost between two passes, a timestamp that belonged to a different sentence than the one it labelled. On October 28, 2026, Nvidia released Nemotron 3.5 Transcribe, an open-weight speech model that collapses that chain into one promptable model. It is a speech-augmented language model: an audio encoder feeds a hybrid Mamba-Transformer mixture-of-experts language model, so the same recording can produce a verbatim transcript, a cleaned-up readable version, a speaker-labelled transcript, a translation, or an answer to a question about the audio — depending on the instruction you send with it. The model card puts the size at 34 billion total parameters with about 4.3 billion active per token, a 65,536-token context, and recordings up to one hour long. It transcribes 40 languages, translates between English and 31 other languages, and labels up to eight speakers in a recording. The weights are released under the OpenMDW-1.1 licence, which allows commercial use. That consolidation is the whole point of this tutorial. You will serve the model on one GPU with vLLM, talk to it through an OpenAI-compatible endpoint, and build the small amount of code that turns its raw output into two things people actually use: readable, speaker-labelled meeting notes and a subtitled file with real word timings. Along the way you will learn which of its tricks are free (prompt-steered style and translation) and which cost you something (word timestamps need a specific server setup, and the fastest serving numbers Nvidia quotes belong to a different, optimised build than the open one). One honest boundary before you start: the model itself needs a serious GPU — the reference deployment is one H100 with 80 GB of memory — and this tutorial was written and verified on a machine without one. The server command below is copied from Nvidia's model card, not run here, and the article says so wherever that matters. Everything that could be verified without the GPU was verified: the audio preparation, the client code, and the post-processing, all against a mock server that mimics the real endpoint's reply shape. Rent the GPU for the serving step, or try the hosted demo linked from the model card first, and you will know exactly what to expect when the real server answers. What you'll need - A GPU machine for serving. The model card's reference is one H100 80 GB, serving the BF16 checkpoint. Its hardware table also lists an A100 80 GB (BF16), an L40S 48 GB (FP8 only), and an RTX 5090 (NVFP4 only), plus Blackwell cards and DGX Spark. A rented cloud GPU or a workstation with one of these cards is the realistic route; this is not a laptop-CPU model. Pick your checkpoint precision from that table, not from habit: the wrong precision for your card simply will not be the supported path. - Linux (or WSL2 on Windows). The vLLM route below is a Linux recipe; the card notes Linux runtimes also run on Windows through WSL2. - Python 3.13, uv, and a Hugging Face account for the download. The NeMo Speech setup below uses uv venv --python 3.13. The checkpoint download is large — plan for a long download and enough disk for the weights plus a working margin, and do not start it on a metered connection without checking the checkpoint size on the model page first. - ffmpeg on the machine that prepares audio. Any recent ffmpeg works; this is the only step that runs comfortably on an ordinary laptop, and it is where your recordings get normalised before they ever reach the model. - No API key for the local server. The vLLM server speaks the OpenAI chat-completions format on your own machine, so the client uses a placeholder key. Keep that server on localhost or behind your own authentication if you expose it — it is not a public API with built-in access control. - Recordings you have the right to process. Speaker labels distinguish voices inside one recording; they do not identify real people, and the model card says so explicitly. Get consent where your jurisdiction or workplace requires it before you transcribe other people's meetings, and treat transcripts of sensitive meetings as sensitive documents. Step 1 — Choose your precision and your serving route Three decisions shape everything else, so make them before you download anything. First, precision: BF16 is the full-precision reference, FP8 is a quantised variant, and NVFP4 is the smallest. The model card's hardware table is the authority here — H100 takes BF16 or FP8, RTX 5090 takes NVFP4, L40S takes FP8 — and the checkpoint names follow the same pattern (NVIDIA-Nemotron-3.5-Transcribe-30B-A4B, plus -FP8 and -NVFP4 variants). If your card is not in the table, stop and check the card before downloading tens of gigabytes of weights you cannot serve. Second, serving route. This tutorial uses vLLM, because it is the only route that gives you everything: transcription, translation, speaker labels, and word timestamps. SGLang-Omni is simpler — one command, no NeMo — but the model card is explicit that it does not do word timestamps, speaker-attributed output, translation, or speculative decoding; it is a whole-file transcription endpoint. Running the model in-process with Hugging Face Transformers is the third route, useful for a notebook experiment, but it gives up the serving optimisations and the timestamp middleware. The decision table at the end of this guide summarises the trade-off; for meeting notes with speakers and timings, vLLM is the right default. Third, expectations about speed. The model card reports batched English ASR throughput on one H100 of roughly 410x real-time with two-token speculative decoding (MTP2) and 270x with eight-token decoding (MTP8) — and it warns, in the same breath, that these are rough batch estimates excluding startup, not single-request latency. Separately, Nvidia quotes an optimised NIM build transcribing up to 2,238 seconds of audio per second on an RTX PRO 6000. That NIM build is not the open-source release, and the card says the open vLLM integration is expected to be roughly two times slower. Do not budget a project on the NIM number; budget on the vLLM number, and measure your own first hour before promising anyone a turnaround time. Step 2 — Install NeMo Speech and vLLM, then download the checkpoint The model card's environment setup installs vLLM 0.32.0 and the latest NeMo Speech main branch into one environment with uv, then downloads the checkpoint. Run it exactly as written — the two extra packages at the end are not optional extras, they are fixes for gaps in the dependency sets: PyAV decodes compressed formats such as M4A without a system ffmpeg, and vLLM requires OpenCV, which NeMo's dependency settings exclude. git clone https://github.com/NVIDIA-NeMo/Speech.git nemo-speech cd nemo-speech uv venv --python 3.13 .venv UV_PROJECT_ENVIRONMENT=$PWD/.venv uv sync --extra asr --extra vllm --locked --no-install-project uv pip install --python .venv/bin/python --no-deps -e . # PyAV decodes compressed audio formats, such as M4A, without a system FFmpeg. uv pip install --python .venv/bin/python --no-deps av==19.0.2 # vLLM requires OpenCV, which NeMo's dependency settings exclude. uv pip install --python .venv/bin/python --no-deps opencv-python-headless==5.0.0.93 source .venv/bin/activate hf download nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B --local-dir ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B If you chose FP8 or NVFP4 in Step 1, substitute that checkpoint name in the download command. The hf command comes with the Hugging Face tooling in the environment; if it is not on your path after activation, install huggingface_hub into the same environment and retry before assuming the download failed. Expect this step to take a while: you are downloading a frontier-scale checkpoint, not a pip package, and a failed download halfway through is usually a disk-space or network problem, not a model problem. Check both before re-running. Step 3 — Start the vLLM server (the command, explained flag by flag) Here is the model card's server command, verbatim. The environment variables come first because they change behaviour the flags cannot: the plugin that teaches vLLM about this model, the one-hour audio limit (the default is 600 seconds), the 180 MB per-clip file-size cap, and the V1 model runner, which word timestamps require. export VLLM_PLUGINS=nemo_speechlm # Allow vLLM to decode recordings up to one hour long (the default is 600 seconds). export VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 export VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=180 # Word timestamps require the V1 model runner. export VLLM_USE_V2_MODEL_RUNNER=0 vllm serve ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --served-model-name nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.94 \ --max-model-len 65536 --max-num-batched-tokens 4096 --max-num-seqs 192 \ --enable-chunked-prefill --no-enable-prefix-caching \ --use-replayssm --mamba-backend flashinfer --mamba-cache-mode none \ --mm-processor-cache-gb 0 --no-async-scheduling \ --limit-mm-per-prompt '{"audio":{"count":6,"length":120000000}}' \ --speculative-config '{"method":"mtp","num_speculative_tokens":8}' \ --reasoning-parser nemotron_v3 \ --trust-request-mm-kwargs \ --middleware nemo.collections.speechlm2.vllm.salm.ctc_serving.ctc_timestamp_middleware What the flags you are most likely to touch actually do: --gpu-memory-utilization 0.94 lets vLLM use 94% of GPU memory — lower it if the card runs other work. --max-model-len 65536 matches the model's context window; a dense one-hour conversation can need about 20,000 output tokens, so do not shrink this casually. --no-enable-prefix-caching and the ReplaySSM flags are not tuning knobs to experiment with on day one; the card instructs you to keep them as written. The last two flags are the timestamp switch: --trust-request-mm-kwargs plus the CTC timestamp middleware are what let a request opt into word timestamps in Step 6. Leave them out and timestamps silently will not be available, which is the single most common setup mistake with this model. The server listens on 127.0.0.1:8000; use --host and --port to change that. The first requests after startup are slower while kernels compile and warm up — the card says so, and it is worth knowing before you judge the server on its first transcript. If you want the DFlash2 draft model instead of MTP speculative decoding, read the card's DFlash2 section first: it needs a small patch to vLLM 0.32.0's DFlash2 code, runs only on the V2 runner (so you unset the V1 variable), and word timestamps are not available with it. For meeting notes, stay with MTP and keep your timestamps. Step 4 — Prepare your recording: 16 kHz, mono, under one hour The model card's recipe uses 16 kHz audio, and the server caps a clip at one hour and 180 MB. Normalise before you send anything: one ffmpeg command handles the sample rate, the channel count, and the container in a single pass, whatever your recorder produced. ffmpeg -y -i meeting_raw.m4a -ar 16000 -ac 1 meeting_16k.wav ffprobe -v error -show_entries stream=sample_rate,channels,codec_name \ -show_entries format=duration -of default=noprint_wrappers=1 meeting_16k.wav Verified here on a generated test recording: a 44.1 kHz stereo file converted cleanly to 16,000 Hz mono PCM with its 5-second duration intact, which is exactly the shape the server expects. If your recording is longer than an hour, split it before transcribing — ffmpeg's -ss and -t options cut clean segments — and transcribe each part separately. One consequence to plan for: speaker tags restart with each request, so spk:0 in part two is not necessarily the same person as spk:0 in part one. Keep a mapping per part, or better, keep meetings under an hour per request whenever the agenda allows it. The same applies to the 180 MB cap: a long, high-bitrate original that survives as a WAV will trip it, which is another reason to normalise to 16 kHz mono first. Step 5 — Send your first transcript request The client below follows the model card's request function, with one change you should know about: the model name is a constant, so you cannot typo it differently in two places. Audio travels as a base64 data URL inside an OpenAI-style chat message, which is why the same code works against any OpenAI-compatible server, including the mock used for verification here. import base64 from pathlib import Path from openai import OpenAI MODEL = "nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B" client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="not-needed") def ask(audio_path, prompt, system_prompt="", reasoning=False, timestamps=False): audio = base64.b64encode(Path(audio_path).read_bytes()).decode() messages = [{"role": "system", "content": system_prompt}] if system_prompt else [] messages.append({"role": "user", "content": [ {"type": "text", "text": prompt}, {"type": "audio_url", "audio_url": {"url": f"data:audio/wav;base64,{audio}"}}, ]}) extra_body = { "chat_template_kwargs": {"enable_thinking": reasoning}, "skip_special_tokens": False, # keep speaker and event tags "spaces_between_special_tokens": False, "seed": 0, } if reasoning: extra_body.update(top_k=-1, thinking_token_budget=17408) extra_body["chat_template_kwargs"]["reasoning_budget"] = 16384 if timestamps: extra_body["mm_processor_kwargs"] = {"capture_ctc_timestamps": True} return client.chat.completions.create( model=MODEL, messages=messages, temperature=0.6 if reasoning else 0, top_p=0.95 if reasoning else 1.0, max_tokens=20480 if reasoning else 16384, extra_body=extra_body, ) response = ask("meeting_16k.wav", "Transcribe the following audio.") print(response.choices[0].message.content) Two settings in that function deserve emphasis because they are easy to get wrong. Transcription, translation, and language identification all use temperature 0 with reasoning off — deterministic output for a job where creativity is a bug. And skip_special_tokens stays False on purpose: setting it to True would strip the speaker and event tags that Step 6 depends on. The sampling rule flips only for audio understanding in Step 7, where reasoning goes on and temperature rises to 0.6 with top-p 0.95. Verification note, stated plainly: this client and everything after it were run against a mock server returning a canned two-speaker reply, because no H100 was available to this tutorial's verification machine. The mock proved the request shape (model name, temperature 0, the timestamp flag, and a base64 audio data URL), the speaker-tag parsing, and both output formats in Step 6 — but the transcript words in the examples below are illustrative, not model output. Treat your first real transcript as a test: run a two-minute clip you already know well, read it end to end, and only then hand the pipeline a real meeting. Step 6 — Get speakers and word timings, then turn them into notes and subtitles A plain transcript answers "what was said". A meeting note answers "who said it, and when" — and that needs two things at once: the speaker-tagged prompt from the model card's prompt guide, and the timestamp flag from Step 5. Ask for both together, then check the alignment error field before you trust a single timestamp. The card is explicit: timestamp requests accept one audio recording, stream=false, and n=1; a transcript without speaker tags is aligned as a single speaker; and when you are evaluating diarization itself, use the diarization segments, not the word timestamps. SPEAKER_PROMPT = ( "Transcribe this audio with punctuation and capitalization. " "Include all spoken words and verbal fillers. " "Write numbers in their standard written form. " "Put speaker tags <spk:0> <spk:1> <spk:2> <spk:3> on the transcription." ) response = ask("meeting_16k.wav", SPEAKER_PROMPT, timestamps=True) alignment = response.model_dump()["ctc_timestamps"] if alignment["error"] is not None: raise RuntimeError(alignment["error"]) for word in alignment["words"]: print(word["word"], word["start"], word["end"], word["speaker"]) The ctc_timestamps object gives you four things: words (word text, start and end in seconds, and the transcript's speaker tag), diarization (speaker activity segments from the audio), a mapping between transcript tags and diarization speakers, and the error field. The two small functions below turn the words into the two outputs people actually open. One subtitle cue per speaker turn, timed by its first and last word, keeps subtitles readable; grouping the same words by speaker with a start time gives you meeting notes you can skim. import re def fmt_ts(seconds): seconds = float(seconds) h, rem = divmod(int(seconds), 3600) m, s = divmod(rem, 60) ms = int(round((seconds - int(seconds)) * 1000)) return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}" def words_to_srt(words): cues, current = [], None for w in words: speaker = w.get("speaker") or "spk:0" if current and current["speaker"] == speaker: current["words"].append(w["word"]) current["end"] = w["end"] else: current = {"speaker": speaker, "words": [w["word"]], "start": w["start"], "end": w["end"]} cues.append(current) out = [] for i, cue in enumerate(cues, 1): label = f"Speaker {int(cue['speaker'].split(':')[1]) + 1}" out.append(f"{i}\n{fmt_ts(cue['start'])} --> {fmt_ts(cue['end'])}\n" f"[{label}] {' '.join(cue['words'])}") return "\n\n".join(out) + "\n" def words_to_markdown(words): lines, current = [], None for w in words: speaker = w.get("speaker") or "spk:0" if current and current["speaker"] == speaker: current["words"].append(w["word"]) else: current = {"speaker": speaker, "words": [w["word"]], "start": w["start"]} lines.append(current) out = [] for turn in lines: label = f"Speaker {int(turn['speaker'].split(':')[1]) + 1}" out.append(f"**{label}** [{fmt_ts(turn['start'])[:-4]}] " f"{' '.join(turn['words'])}") return "\n\n".join(out) + "\n" Run against the mock server, those functions produce exactly what you would expect from a two-speaker opening — Markdown reading Speaker 1 [00:00:00] Good morning, everyone. followed by Speaker 2 [00:00:01] Let's start with the budget., and an SRT file with matching cues (00:00:00,120 to 00:00:01,300, then 00:00:01,600 to 00:00:03,000). The words are canned mock data, so the point of the test is the shape, not the sentence: the same code paths, fed with a real server's ctc_timestamps, produce real notes and real subtitles without modification. Two details in the code are worth keeping: a word with no speaker tag falls back to spk:0 rather than crashing, and the Markdown timestamp drops the milliseconds for readability while the SRT keeps them, because subtitles need the precision and notes do not. Step 7 — Change the job with the prompt: style, vocabulary, translation, questions This is where one model replaces four tools, and the model card's prompt guide is the menu. Keep the system prompt empty for transcription and translation — that is the card's instruction, not a suggestion — and put the job in the user message. - Verbatim vs readable. For a record you may quote, ask for wording verbatim, including fillers and repetitions. For captions and searchable notes, ask it to omit fillers and accidental repetitions while keeping the remaining wording unchanged. Both prompts add punctuation, capitalisation, and "transcript only" as the output format, so the model does not summarise when you asked it to transcribe. - Domain vocabulary. Add context terms to the prompt — product names, drug names, internal jargon, the participants' names — instead of retraining anything. The card's contextual-transcription prompt is simply the transcription instruction plus Context terms: <names and terms>. This is the cheapest accuracy win in the whole tutorial, and the one most people skip. - Translation. Please translate the given speech to <target language>. That is the whole prompt. Translation runs at temperature 0 with reasoning off, like transcription, and covers English to and from 31 other languages. Check the card's language matrix for your pair before promising a translation: transcription covers 40 languages, but not every transcription language is a translation language. - Questions about the audio. Ask a specific question and turn reasoning on — that flips the sampling to temperature 0.6, top-p 0.95, a 20,480-token completion limit, and a 16,384-token reasoning budget, as in Step 5's function. This is the mode for "what did we decide about the launch date?" after you already have the transcript, not a replacement for reading it. Audio understanding was evaluated mainly in English, per the card, so treat non-English question-answering as experimental. - Only the nearest speaker (experimental). The card offers a dictation-style prompt that transcribes only the speaker nearest the microphone and ignores other voices. It is labelled experimental in the guide; use it for voice interfaces and dictation, not for meetings where every voice matters. Step 8 — Batch a folder, then run the quality checks that actually catch errors One meeting is a demo; a folder of meetings is a workflow. The full client script used for verification in this tutorial — transcribe.py, with --mode speaker|readable|verbatim|translate|qa, --format md|srt|txt, --context for vocabulary terms, and --out for the output file — wraps Steps 5–7 into one command per file, so a folder batch is a shell loop, not a new program: mkdir -p transcripts for f in recordings/*_16k.wav; do base=$(basename "$f" _16k.wav) python transcribe.py "$f" --mode speaker --format md --out "transcripts/$base.md" python transcribe.py "$f" --mode speaker --format srt --out "transcripts/$base.srt" done ls transcripts/ Then, before anyone relies on a transcript, run these checks — they are the ones that catch the failure modes the model card itself lists: substitutions, deletions, repetitions, hallucinated speech, wrong language choice, and speaker-attribution errors. - Check the error field first. If ctc_timestamps["error"] is not null, the timestamps are not usable, even if the text looks fine. Do not ship subtitles from a failed alignment. - Spot-check names, numbers, and speaker changes. Read the first two minutes and every speaker change against the audio. Names and domain terms are where transcription fails most visibly, and speaker changes are where diarization fails most quietly. If a name is consistently wrong, add it as a context term and re-run that file — that is what the vocabulary prompt is for. - Watch overlapping speech and long files. The card recommends targeted testing for long recordings, overlapping speakers, noise, names, and code-switching. A panel discussion with constant crosstalk is a harder test than a two-person interview; grade your pipeline on the hard file, not the easy one. - Collapse, but do not trust, repeated lines. A cheap guard in the verification script drops exact consecutive duplicate lines, which catches the visible symptom of a generation loop. It is not a cure: if a transcript repeats itself, re-run it and inspect the audio around the loop before using that section. - Stop the server when you are done. A rented GPU bills whether you are transcribing or not. The vLLM process holds the card's memory until you stop it, so make shutting it down part of the workflow, not an afterthought. Which approach should you use? The choice is not "which model" but "which serving route, for which job" — and the model card draws the lines clearly. | Route | Use it when | What you give up | |---|---|---| | vLLM + NeMo Speech (this tutorial) | You need speaker labels, word timestamps, translation, or speculative decoding from one server. | The heaviest setup: a GPU machine, the NeMo environment, and the full serve command. | | SGLang-Omni | You need whole-file transcription quickly and nothing else, on one GPU. | Word timestamps, speaker-attributed output, translation, and speculative decoding are all absent. | | Transformers, in-process | You are experimenting in a notebook and want the model as a Python object. | Serving optimisations and the timestamp middleware; you manage memory and batching yourself. | | The hosted demo linked from the model card | You want to hear what the model does on your audio before renting a GPU. | Your audio leaves your machine, and you get a demo, not a pipeline. | | A smaller local transcriber (e.g. faster-whisper) | You have no GPU budget and your meetings are single-speaker or speaker labels do not matter. | Accuracy on multi-speaker, multilingual, and long-form material — the model card's benchmark leads are for this model, not for the smaller one. | One more comparison, because it decides budgets: the model card's headline benchmark figures, captured October 12, 2026, are 4.71% mean word error rate on nine English sets, 4.62% across seven multilingual sets, and 15.80% mean cpWER on full multi-speaker sessions — each the best open-weight figure the card lists against its next-best open-weight rival. Those are the card's numbers, not measurements from this tutorial, and they describe the model, not your recordings. Your accent mix, your room noise, and your vocabulary will move them. The honest way to compare against your current tool is a side-by-side on ten of your own recordings, scored by hand on names, numbers, and speaker changes — the errors that cost you time, not the average that flatters a leaderboard. The takeaway Nemotron 3.5 Transcribe earns its place by removing glue code: one open-weight model, one prompt, and you choose between a verbatim record, a readable transcript, a speaker-labelled meeting note, a translation, or an answer about the audio. The price is a real GPU, a careful server setup where the timestamp flags are easy to forget, and the discipline to verify the first transcripts by ear before trusting the pipeline. Do the setup once, keep the prompt guide beside the client, add your vocabulary as context terms, and check the alignment error field every single run. The model does the transcribing; the workflow around it — normalised audio, the right prompt, and five minutes of spot-checking — is what makes the transcript trustworthy. Sources: Nvidia's model card for nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B on Hugging Face (architecture, 34B total / 4.3B active parameters, 65,536-token context, one-hour recordings, 40 transcription languages, OpenMDW-1.1 licence, hardware compatibility table, vLLM 0.32.0 + NeMo Speech environment setup, the vLLM serve command and its environment variables, the request function and sampling rules, the prompt guide, word-timestamp and diarization rules, SGLang-Omni limits, DFlash2 notes, and benchmark results captured October 12, 2026, all read October 29, 2026); launch coverage for the translation count (31 languages to or from English), speaker labelling (13 languages, up to eight speakers), and the NIM-versus-open-release speed distinction; and the verification runs quoted in Steps 4–6 and 8 (ffmpeg 16 kHz mono conversion, mock-server client runs with openai 3.41.0, Markdown and SRT outputs, and the flattened extra_body wire format). The vLLM serve command and the benchmark figures are the model card's, not runs performed here, and the article says so where it uses them.

Nemotron 3.5 Transcribe'ı Yerel Olarak Nasıl Çalıştırılır: Tek GPU, 1 Saatlik Dosyalar, API Faturası Yok Nemotron 3.5 Transcribe'ı Yerel Olarak Nasıl Çalıştırılır: Tek GPU, 1 Saatlik Dosyalar, API Faturası Yok NVIDIA'nın yeni Nemotron 3.5 Transcribe'ını tek bir GPU'da çalıştırabilir ve saatler süren kayıtları 40 dilde dakika başı ödeme yapmadan veya sesi kimseye göndermeden yazıya dökebilirsiniz. Hugging Face Open ASR Leaderboard'daki en iyi açık ağırlıklı modeldir, ticari kullanıma uygun OpenMDW-1.1 lisansı altında sunulur ve NVIDIA, çıkış günü olan 28 Ekim 2026'da BF16, FP8 ve NVFP4 ağırlıklarını yayınladı. {% cta apidog.com %} Bugün Apidog'u deneyin {% endcta %} Bu kılavuz, ihtiyacınız olan donanımı, onu çalıştırmanın üç yolunu (vLLM, SGLang-Omni ve Transformers), resmi model kartındaki tam komutları ve insanların takıldığı ayarları kapsar. Tüm komutlar 29 Ekim 2026'da karttan kopyalanmıştır. Modele yeni misiniz? ücretsiz kılavuzumuz ne yaptığını ve nasıl sıralandığını açıklar. GPU'nuz yok mu? barındırılan API dakika başına 0,002 $ tutar. !Architecture diagram of Nemotron 3.5 Transcribe: audio feeds a 600M ASR encoder and a speaker encoder, which feed a hybrid Mamba-2 and Transformer mixture-of-experts language model with 34B total and about 4.3B active parameters Neden yerel olarak çalıştırmalısınız? - Dakika başına fatura yok. DeepInfra'nın dakika başına 0,002 $ ücretiyle 10.000 saat 1.200 $ tutar. Kendi GPU'nuzda ise yalnızca elektrik maliyeti olur. - Gizlilik. Müşteri aramaları, tıbbi dikte ve hukuki kayıtlar ağınızdan asla ayrılmaz. - Tam özellik seti. vLLM yolu kelime zaman damgalarını, konuşmacı ayrıştırmayı, çeviriyi ve bağlam istemlerini destekler. Bazı barındırılan uç noktalar bunun yalnızca bir kısmını sunar. - İnce ayar. NVIDIA'nın lansman yazısı, tıbbi bir ince ayarın WER'i %30,8'den %11,8'e düşürdüğünü gösteriyor. Bunu yalnızca ağırlıklarla yapabilirsiniz. Donanım: onu ne çalıştırır Modelin toplamda 34B parametresi ve yaklaşık 4,3B aktif parametresi vardır, bu nedenle token başına hesaplama küçük olsa da ağırlıklar büyüktür. NVIDIA'nın referans hedefi BF16'da bir H100 80 GB'dır. Model kartındaki uyumluluk tablosu: | GPU | Hassasiyetler | Çalışma Zamanları | | --- | --- | --- | | H100, H200 | BF16, FP8 | vLLM 0.32.0, Transformers, SGLang-Omni 0.2.0 (H100) | | A100 80 GB | BF16 | vLLM 0.32.0, Transformers | | L40S 48 GB | FP8 | vLLM 0.32.0, Transformers | | B200, GB200, GB300 DGX Station | BF16, FP8, NVFP4 | vLLM 0.32.0, Transformers | | DGX Spark (GB10), Jetson Thor | BF16, FP8, NVFP4 | vLLM 0.32.0, Transformers | | RTX PRO 6000 Blackwell Server | BF16, FP8, NVFP4 | vLLM, Transformers, SGLang-Omni, GGML | | RTX 5090 (32 GB) | yalnızca NVFP4 | vLLM 0.32.0, Transformers, GGML | Kartınıza uygun kontrol noktasını seçin: - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B (BF16) - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-FP8 - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4 RTX 4090 gibi 24 GB'lık bir kart listede yok ve Apple Silicon desteklenmiyor. İşletim sistemleri Linux veya WSL2 üzerinden Windows'tur. GGML yerel Windows'ta da çalışır, ancak NVIDIA'nın lansman yazısı NeMo-Speech.cpp yerel derlemesinin hâlâ "yakında" olduğunu söylüyor. Seçenek 1: vLLM sunucusu (önerilir) vLLM size toplu işleme, bir saatlik dosyalar, spekülatif kod çözme ve kelime zaman damgalarıyla OpenAI uyumlu bir uç nokta sunar. Modelin vLLM eklentisini sağlayan vLLM 0.32.0 ile en son NeMo Speech ana dalına ihtiyaç duyar. !Comparison diagram of the three local run options for Nemotron 3.5 Transcribe: vLLM with NeMo Speech for full features, SGLang-Omni for plain transcription, and Transformers for scripts Kurulum Kart, uv ve Python 3.13 kullanır: git clone https://github.com/NVIDIA-NeMo/Speech.git nemo-speech cd nemo-speech uv venv --python 3.13 .venv UV_PROJECT_ENVIRONMENT=$PWD/.venv uv sync --extra asr --extra vllm --locked --no-install-project uv pip install --python .venv/bin/python --no-deps -e . uv pip install --python .venv/bin/python --no-deps av==19.0.2 uv pip install --python .venv/bin/python --no-deps opencv-python-headless==5.0.0.93 source .venv/bin/activate hf download nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B --local-dir ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B PyAV (av), sistem FFmpeg'i olmadan M4A ve diğer sıkıştırılmış biçimlerin kodunu çözmesini sağlar. OpenCV, vLLM ona ihtiyaç duyduğu ve NeMo'nun bağımlılık ayarları onu dışarıda bıraktığı için oradadır. Sunucuyu başlatın export VLLM_PLUGINS=nemo_speechlm export VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 export VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=180 export VLLM_USE_V2_MODEL_RUNNER=0 vllm serve ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --served-model-name nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.94 \ --max-model-len 65536 --max-num-batched-tokens 4096 --max-num-seqs 192 \ --enable-chunked-prefill --no-enable-prefix-caching \ --use-replayssm --mamba-backend flashinfer --mamba-cache-mode none \ --mm-processor-cache-gb 0 --no-async-scheduling \ --limit-mm-per-prompt '{"audio":{"count":6,"length":120000000}}' \ --speculative-config '{"method":"mtp","num_speculative_tokens":8}' \ --reasoning-parser nemotron_v3 \ --trust-request-mm-kwargs \ --middleware nemo.collections.speechlm2.vllm.salm.ctc_serving.ctc_timestamp_middleware Anlaşılması gereken bayraklar: - VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 vLLM'in ses sınırını 600 saniyeden bir saate yükseltir. Bunu unutursanız uzun dosyalar başarısız olur. - VLLM_USE_V2_MODEL_RUNNER=0 kelime zaman damgalarının ihtiyaç duyduğu V1 çalıştırıcısını korur. - --speculative-config sekiz taslak token ile çoklu token tahminini (MTP8) etkinleştirir. NVIDIA, bir H100'de MTP8 ile 270 ve MTP2 ile 410 toplu İngilizce RTFx ölçtü; bu nedenle gecikmeden çok aktarım hızını önemsiyorsanız "num_speculative_tokens":2 deneyin. - --middleware ... ctc_timestamp_middleware ve --trust-request-mm-kwargs, isteklerin kelime zaman damgası istemesini sağlar. - Ayarlar 80 GB'lık bir H100'ü hedefler. Diğer kartlarda --gpu-memory-utilization, --max-model-len veya --max-num-seqs değerlerini düşürün. Sunucu 127.0.0.1:8000 adresini dinler. Ağınızdaki diğer makinelerden erişmek için --host 0.0.0.0 ekleyin. Dosya gönderin import base64 from openai import OpenAI client = OpenAI(base_url="http://127.0.0.1:8000/v1", [REDACTED CREDENTIAL]) audio = base64.b64encode(open("standup.wav", "rb").read()).decode() response = client.chat.completions.create( model="nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B", messages=[{"role": "user", "content": [ {"type": "text", "text": "Transcribe the following audio."}, {"type": "audio_url", "audio_url": {"url": f"data:audio/wav;base64,{audio}"}}, ]}], temperature=0, extra_body={"chat_template_kwargs": {"enable_thinking": False}, "skip_special_tokens": False}, ) print(response.choices[0].message.content) Metin talimatını herhangi bir görev istemiyle değiştirin. Birkaç sesin olduğu bir toplantı için: Transcribe verbatim, including fillers. Write numbers as spoken, without punctuation or capitalization. Start every speaker turn with <spk:0>, <spk:1>, etc. Keep one stable tag per speaker. Output only the speaker-tagged transcript. Kelime zaman damgası isteklerinin kuralları vardır: istek başına bir kayıt, stream=false ve n=1. Yanıt daha sonra kelimeleri, saniye cinsinden başlangıç ve bitiş zamanlarını ve konuşmacı etiketlerini içeren bir ctc_timestamps nesnesi içerir. Seçenek 2: Düz transkripsiyon için SGLang-Omni Yalnızca tüm dosyalardan metin çıkarmanız gerekiyorsa, SGLang-Omni 0.2.0 daha basittir ve NeMo'ya ihtiyaç duymaz. BF16 veya FP8 kontrol noktasıyla bir H100 80 GB ve bir RTX PRO 6000 Blackwell üzerinde doğrulanmıştır. sgl-omni serve \ --model-path nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --model-name nemotron-transcribe \ --mem-fraction-static 0.85 \ --port 8000 curl http://localhost:8000/v1/audio/transcriptions \ -F model=nemotron-transcribe \ -F file=@audio.wav Bu /v1/audio/transcriptions rotası, OpenAI'nın Whisper API'sinin kullandığı şekille eşleşir, bu nedenle mevcut Whisper istemci kodu genellikle yalnızca yeni bir temel URL'ye ihtiyaç duyar. Ödünleşim: bu entegrasyonda kelime zaman damgaları, konuşmacı etiketleri, çeviri veya spekülatif kod çözme yoktur. Seçenek 3: Bir betikte Transformers Not defterleri ve tek seferlik işler için, kaynaktan yüklenen Hugging Face Transformers ile süreç içinde çalıştırın: pip install git+https://github.com/huggingface/transformers from transformers import AutoModelForMultimodalLM, AutoProcessor model_id = "nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto") [REDACTED PROMPT] "and verbal fillers. Put speaker tags <spk:0> <spk:1> <spk:2> <spk:3> on the transcription.") inputs = processor.apply_transcription_request(["podcast.mp3"], [REDACTED PROMPT] outputs = model.generate(**inputs, max_new_tokens=4096) print(processor.decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=False)) Başlamanın en kolay ve ölçekte çalıştırmanın en yavaş yoludur. Uzun dosyalar için max_new_tokens değerini artırın: yoğun bir saat yaklaşık 20.000 token üretebilir. Kendi GPU'nuzda ne kadar hızlı? NVIDIA'nın manşet rakamı, NVFP4 ağırlıkları ve DFlash2 spekülatif kod çözme ile bir RTX PRO 6000 Blackwell üzerinde saniyede 2.238 saniye sestir ve 30 saniyelik bir klibi döndürmek 0,13 saniye sürer. Dipnotu okuyun: bu rakamlar, sunum değişiklikleri henüz açık kaynak olmayan optimize edilmiş NIM'den gelir. Kart, açık vLLM entegrasyonunun yaklaşık iki kat daha yavaş olmasının beklendiğini söylüyor. H100'de BF16'da toplu İngilizce transkripsiyon için kart şu RTFx sayılarını listeler (hesaplama saniyesi başına ses saniyesi): | Spekülatif kod çözme | RTFx | | --- | --- | | MTP2 | 410 | | DFlash2, K=7 | 340 | | MTP8 | 270 | | DFlash2, K=15 | 260 | En yavaş satır bile toplu işte 15 saniyenin altında GPU süresinde bir saatlik sesi işler. DFlash2 taslak modelini istiyorsanız, vLLM 0.32.0'ın karttan küçük bir yamaya ihtiyaç duyduğunu, DFlash2'nin kelime zaman damgalarını devre dışı bıraktığını ve bunu MTP ile birleştirmemeniz gerektiğini unutmayın. Yaygın sorunlar ve çözümleri - Uzun dosyalar 10 dakikada kesiliyor. VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 ayarını atladınız. - Yanıtta zaman damgası yok. VLLM_USE_V2_MODEL_RUNNER=0, --middleware bayrağını ve isteğin stream=false ile tek kayıt içerdiğini kontrol edin. - Daha küçük kartlarda bellek yetersiz. FP8 veya NVFP4 kontrol noktasına geçin ve --max-num-seqs değerini düşürün. - Transkriptin sonunda tekrarlanan ifadeler. Kart, nihai çıktılar için bir üretim döngüsü koruması önerir. max_tokens değerini sınırlayın ve tekrarları kırpın. - Adların yanlış yazımı. İsteme Context terms: <names> ekleyin. - Transkriptte arka plan sesleri. Karttan deneysel yakın alan istemini deneyin; bu, NVIDIA'nın testinde karışık seste WER'i %27,32'den %4,78'e düşürdü. vLLM'e yeniyseniz, vLLM açıklayıcımız istekleri nasıl toplu işlediğini ve GPU belleğini nasıl yönettiğini anlatır. Kodlama aracınızın yerel sunucuyu Apidog CLI ile test etmesine izin verin Yerel bir model ancak uç noktası vLLM yükseltmeleri, istem ayarlamaları ve kontrol noktası değişimleri boyunca doğru kaldığı sürece kullanışlıdır. Kodlama aracınızın bunu Apidog CLI ile üstlenmesine izin verin ve aynı makineden http://127.0.0.1:8000 adresine karşı kontroller çalıştırın. npm install -g apidog-cli@latest apidog login --with-token <YOUR_TOKEN> npx -y skills add https://apidog.com # pick your agent when prompted Ardından aracınıza bir istem verin: In Apidog project <id>, add POST {{baseUrl}}/v1/chat/completions for my local Nemotron 3.5 Transcribe server. Create test cases that send samples/en.wav and samples/two-speakers.wav as base64 audio_url content with temperature 0, and assert status 200, a non-empty choices[0].message.content, and at least two distinct <spk:N> tags for the two-speaker file. Run them against the local environment and report failures. Aracı şemayı okur, yükünü doğrular, vakaları yazar ve çalıştırır: apidog cli-schema get test-case-create apidog cli-schema validate test-case-create --file case.json apidog test-case create --project <id> --file case.json apidog run --test-case <caseId> --project <id> -e <envId> -r cli,json Her sunucu yükseltmesinden sonra çalıştırın veya CI/CD iş akışında Apidog CLI bölümünde gösterildiği gibi apidog run ... -r junit komutunu CI'ya bağlayın. Aracı kurulumu için Claude Code'da Apidog CLI bölümüne bakın. SSS Nemotron 3.5 Transcribe'ı RTX 4090'da çalıştırabilir miyim? NVIDIA'nın uyumluluk listesinde yok. Listelenen en küçük tüketici kartı, NVFP4 kontrol noktasına sahip RTX 5090'dır. Mac'te çalışır mı? 29 Ekim 2026 itibarıyla hiçbir resmi çalışma zamanı Apple Silicon'ı desteklemiyor. Ticari ürünler için yerel kullanım ücretsiz mi? OpenMDW-1.1 lisansı ticari ve ticari olmayan kullanıma izin verir. Gönderim yapmadan önce atıf koşulları için lisans metnini okuyun. Canlı sesi yazıya dökebilir mi? Hayır. Kaydedilmiş dosyalar için çevrimdışı bir modeldir. NVIDIA, canlı kullanım için ayrı bir Nemotron 3.5 ASR Streaming modeli sunar. Hangisi daha hızlı, vLLM mi yoksa SGLang-Omni mi? NVIDIA baş başa bir karşılaştırma yayınlamıyor. Zaman damgalarına, konuşmacılara veya çeviriye ihtiyacınız olduğunda vLLM'i, en basit yalnızca metin sunucusu için SGLang-Omni'yi kullanın. Diğer yerel modellere karşı sıralaması nasıl? 28 Ekim Open ASR Leaderboard'da her açık ağırlıklı modelin önünde. En iyi açık kaynak konuşmadan metne modelleri listemiz onu Qwen3-ASR, Granite Speech, Canary-Qwen, Voxtral ve Whisper ile karşılaştırır. Çalıştırın GPU'nuza uyan kontrol noktasını indirin, vLLM'i bir saatlik ses sınırı ve zaman damgası ara katmanıyla başlatın ve OpenAI istemcisi aracılığıyla bir WAV dosyası gönderin. Ardından aracınızın yerel uç noktaya karşı Apidog kontrolleri çalıştırmasına izin verin, böylece her yükseltme, gerçek kayıtlarla güvenmeden önce hâlâ yazıya döktüğünü kanıtlasın. {% embed youtube.com %}

Nemotron 3.5 Transcribeをローカル実行する方法：GPU1台で1時間ファイル、API課金なし NVIDIAの新しいNemotron 3.5 Transcribeは、GPU 1台で実行でき、1時間の録音を40言語で文字起こしできます。分単位の課金も、音声を誰かに送信する必要もありません。Hugging Face Open ASR Leaderboardでトップのオープンウェイトモデルであり、商用利用しやすいOpenMDW-1.1ライセンスで提供されています。NVIDIAはリリース日の2026年10月28日にBF16、FP8、NVFP4の重みを公開しました。 {% cta apidog.com %} 今すぐApidogを試す {% endcta %} このガイドでは、必要なハードウェア、実行する3つの方法（vLLM、SGLang-Omni、Transformers）、公式モデルカードの正確なコマンド、そしてつまずきやすい設定について説明します。すべてのコマンドは2026年10月29日時点でモデルカードからコピーしたものです。 モデルは初めてですか？ 無料ガイドで、何ができるかとランキングでの位置を説明しています。GPUがない場合は、ホスト型APIが1分あたり0.002ドルです。 !Architecture diagram of Nemotron 3.5 Transcribe: audio feeds a 600M ASR encoder and a speaker encoder, which feed a hybrid Mamba-2 and Transformer mixture-of-experts language model with 34B total and about 4.3B active parameters なぜローカルで実行するのか - 分単位の課金なし。 DeepInfraの1分あたり0.002ドルでは、10,000時間で1,200ドルかかります。自分のGPUなら電気代だけです。 - プライバシー。 顧客通話、医療のディクテーション、法的記録がネットワークの外に出ません。 - 全機能セット。 vLLMの経路では、単語タイムスタンプ、話者ダイアリゼーション、翻訳、コンテキストプロンプトをサポートします。ホスト型エンドポイントの一部はその一部しか公開していません。 - ファインチューニング。 NVIDIAのローンチ記事では、医療向けファインチューニングでWERが30.8%から11.8%に下がったことが示されています。それは重みがなければできません。 ハードウェア：何で実行するか このモデルは合計34Bパラメータで、アクティブは約4.3Bです。そのため、トークンあたりの計算量は小さくても、重みは大きくなります。NVIDIAの参考ターゲットはBF16のH100 80 GB 1台です。モデルカードの互換性表は次のとおりです： | GPU | 精度 | ランタイム | | --- | --- | --- | | H100, H200 | BF16, FP8 | vLLM 0.32.0, Transformers, SGLang-Omni 0.2.0 (H100) | | A100 80 GB | BF16 | vLLM 0.32.0, Transformers | | L40S 48 GB | FP8 | vLLM 0.32.0, Transformers | | B200, GB200, GB300 DGX Station | BF16, FP8, NVFP4 | vLLM 0.32.0, Transformers | | DGX Spark (GB10), Jetson Thor | BF16, FP8, NVFP4 | vLLM 0.32.0, Transformers | | RTX PRO 6000 Blackwell Server | BF16, FP8, NVFP4 | vLLM, Transformers, SGLang-Omni, GGML | | RTX 5090 (32 GB) | NVFP4 のみ | vLLM 0.32.0, Transformers, GGML | カードに合ったチェックポイントを選んでください： - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B (BF16) - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-FP8 - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4 RTX 4090のような24 GBカードはリストに載っておらず、Apple Siliconはサポートされていません。オペレーティングシステムはLinux、またはWSL2経由のWindowsです。GGMLはネイティブWindowsでも動作しますが、NVIDIAのローンチ記事によると、NeMo-Speech.cppのローカルビルドはまだ「近日公開」だそうです。 オプション1：vLLMサーバー（推奨） vLLMは、バッチ処理、1時間のファイル、投機的デコーディング、単語タイムスタンプを備えたOpenAI互換エンドポイントを提供します。vLLM 0.32.0と、モデルのvLLMプラグインを提供する最新のNeMo Speechメインブランチが必要です。 !Comparison diagram of the three local run options for Nemotron 3.5 Transcribe: vLLM with NeMo Speech for full features, SGLang-Omni for plain transcription, and Transformers for scripts インストール モデルカードではuvとPython 3.13を使用しています： git clone https://github.com/NVIDIA-NeMo/Speech.git nemo-speech cd nemo-speech uv venv --python 3.13 .venv UV_PROJECT_ENVIRONMENT=$PWD/.venv uv sync --extra asr --extra vllm --locked --no-install-project uv pip install --python .venv/bin/python --no-deps -e . uv pip install --python .venv/bin/python --no-deps av==19.0.2 uv pip install --python .venv/bin/python --no-deps opencv-python-headless==5.0.0.93 source .venv/bin/activate hf download nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B --local-dir ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B PyAV（av）は、システムFFmpegなしでM4Aやその他の圧縮形式をデコードできるようにします。OpenCVは、vLLMがそれを必要とし、NeMoの依存関係設定では除外されているために含まれています。 サーバーを起動する export VLLM_PLUGINS=nemo_speechlm export VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 export VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=180 export VLLM_USE_V2_MODEL_RUNNER=0 vllm serve ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --served-model-name nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.94 \ --max-model-len 65536 --max-num-batched-tokens 4096 --max-num-seqs 192 \ --enable-chunked-prefill --no-enable-prefix-caching \ --use-replayssm --mamba-backend flashinfer --mamba-cache-mode none \ --mm-processor-cache-gb 0 --no-async-scheduling \ --limit-mm-per-prompt '{"audio":{"count":6,"length":120000000}}' \ --speculative-config '{"method":"mtp","num_speculative_tokens":8}' \ --reasoning-parser nemotron_v3 \ --trust-request-mm-kwargs \ --middleware nemo.collections.speechlm2.vllm.salm.ctc_serving.ctc_timestamp_middleware 理解しておく価値のあるフラグ： - VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 は、vLLMの音声制限を600秒から1時間に引き上げます。これを忘れると長いファイルが失敗します。 - VLLM_USE_V2_MODEL_RUNNER=0 はV1ランナーを維持します。単語タイムスタンプにはこれが必要です。 - --speculative-config は、8つのドラフトトークン（MTP8）によるマルチトークン予測を有効にします。H100では、NVIDIAはMTP8でバッチ英語のRTFx 270、MTP2で410を測定したので、レイテンシよりスループットを重視する場合は "num_speculative_tokens":2 を試してください。 - --middleware ... ctc_timestamp_middleware と --trust-request-mm-kwargs により、リクエストで単語タイムスタンプを要求できます。 - 設定は80 GBのH100を対象としています。他のカードでは、--gpu-memory-utilization、--max-model-len、--max-num-seqs を下げてください。 サーバーは 127.0.0.1:8000 で待ち受けます。ネットワーク上の他のマシンからアクセスするには、--host 0.0.0.0 を追加してください。 ファイルを送信する import base64 from openai import OpenAI client = OpenAI(base_url="http://127.0.0.1:8000/v1", [REDACTED CREDENTIAL]) audio = base64.b64encode(open("standup.wav", "rb").read()).decode() response = client.chat.completions.create( model="nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B", messages=[{"role": "user", "content": [ {"type": "text", "text": "Transcribe the following audio."}, {"type": "audio_url", "audio_url": {"url": f"data:audio/wav;base64,{audio}"}}, ]}], temperature=0, extra_body={"chat_template_kwargs": {"enable_thinking": False}, "skip_special_tokens": False}, ) print(response.choices[0].message.content) テキスト指示は任意のタスクプロンプトに置き換えられます。複数の声がある会議の場合： Transcribe verbatim, including fillers. Write numbers as spoken, without punctuation or capitalization. Start every speaker turn with <spk:0>, <spk:1>, etc. Keep one stable tag per speaker. Output only the speaker-tagged transcript. 単語タイムスタンプのリクエストにはルールがあります。1リクエストにつき1つの録音、stream=false、n=1 です。レスポンスには、単語、開始時刻と終了時刻（秒）、話者タグを含む ctc_timestamps オブジェクトが含まれます。 オプション2：単純な文字起こしにはSGLang-Omni ファイル全体からテキストだけが必要なら、SGLang-Omni 0.2.0の方が簡単で、NeMoも不要です。BF16またはFP8チェックポイントで、H100 80 GB 1台とRTX PRO 6000 Blackwell 1台で検証済みです。 sgl-omni serve \ --model-path nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --model-name nemotron-transcribe \ --mem-fraction-static 0.85 \ --port 8000 curl http://localhost:8000/v1/audio/transcriptions \ -F model=nemotron-transcribe \ -F file=@audio.wav この /v1/audio/transcriptions ルートは、OpenAIのWhisper APIが使う形と同じなので、既存のWhisperクライアントコードは多くの場合、新しいベースURLだけで済みます。トレードオフとして、この統合では単語タイムスタンプ、話者ラベル、翻訳、投機的デコーディングはありません。 オプション3：スクリプト内のTransformers ノートブックや単発のジョブでは、ソースからインストールしたHugging Face Transformersを使ってプロセス内で実行します： pip install git+https://github.com/huggingface/transformers from transformers import AutoModelForMultimodalLM, AutoProcessor model_id = "nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto") [REDACTED PROMPT] "and verbal fillers. Put speaker tags <spk:0> <spk:1> <spk:2> <spk:3> on the transcription.") inputs = processor.apply_transcription_request(["podcast.mp3"], [REDACTED PROMPT] outputs = model.generate(**inputs, max_new_tokens=4096) print(processor.decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=False)) 始めるには最も簡単な方法で、大規模に実行するには最も遅い方法です。長いファイルでは max_new_tokens を増やしてください。濃密な1時間で約20,000トークンを生成することがあります。 自分のGPUではどれくらい速いのか？ NVIDIAの見出しの数字は、RTX PRO 6000 Blackwell 1台で毎秒2,238秒の音声、NVFP4重みとDFlash2投機的デコーディングを使用し、30秒のクリップを0.13秒で返すというものです。脚注を読んでください。これらの数値は最適化されたNIMによるもので、そのサービングの変更はまだオープンソースではありません。モデルカードによると、オープンなvLLM統合は約2倍遅くなると予想されています。 BF16でのH100バッチ英語文字起こしについて、モデルカードには次のRTFx数値（計算1秒あたりの音声秒数）が記載されています： | 投機的デコーディング | RTFx | | --- | --- | | MTP2 | 410 | | DFlash2, K=7 | 340 | | MTP8 | 270 | | DFlash2, K=15 | 260 | 最も遅い行でも、バッチ処理では1時間の音声を15秒未満のGPU時間で処理します。DFlash2ドラフトモデルが必要な場合、vLLM 0.32.0にはモデルカードの小さなパッチが必要で、DFlash2は単語タイムスタンプを無効にし、MTPと組み合わせるべきではないことに注意してください。 よくある問題と修正 - 長いファイルが10分で切れる。 VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 をスキップしています。 - レスポンスにタイムスタンプがない。 VLLM_USE_V2_MODEL_RUNNER=0、--middleware フラグ、そしてリクエストに stream=false と録音1つがあることを確認してください。 - 小さいカードでメモリ不足。 FP8またはNVFP4チェックポイントに切り替え、--max-num-seqs を下げてください。 - 文字起こしの末尾でフレーズが繰り返される。 モデルカードは最終出力に生成ループのガードレールを推奨しています。max_tokens を制限し、繰り返しを削除してください。 - 名前のスペルが間違っている。 プロンプトに Context terms: <names> を追加してください。 - 文字起こしに背景の声が入る。 モデルカードの実験的な近接場プロンプトを試してください。NVIDIAのテストでは、混合音声のWERが27.32%から4.78%に下がりました。 vLLM自体が初めてなら、vLLM解説でリクエストのバッチ処理とGPUメモリ管理の方法を説明しています。 コーディングエージェントにApidog CLIでローカルサーバーをテストさせる ローカルモデルは、vLLMのアップグレード、プロンプトの調整、チェックポイントの交換を通じてエンドポイントが正しいままでなければ役に立ちません。コーディングエージェントにApidog CLIでそれを管理させ、同じマシンから http://127.0.0.1:8000 に対してチェックを実行させましょう。 npm install -g apidog-cli@latest apidog login --with-token <YOUR_TOKEN> npx -y skills add https://apidog.com # pick your agent when prompted 次に、エージェントに1つのプロンプトを与えます： In Apidog project <id>, add POST {{baseUrl}}/v1/chat/completions for my local Nemotron 3.5 Transcribe server. Create test cases that send samples/en.wav and samples/two-speakers.wav as base64 audio_url content with temperature 0, and assert status 200, a non-empty choices[0].message.content, and at least two distinct <spk:N> tags for the two-speaker file. Run them against the local environment and report failures. エージェントはスキーマを読み、ペイロードを検証し、ケースを作成して実行します： apidog cli-schema get test-case-create apidog cli-schema validate test-case-create --file case.json apidog test-case create --project <id> --file case.json apidog run --test-case <caseId> --project <id> -e <envId> -r cli,json サーバーをアップグレードするたびに実行するか、CI/CDパイプラインでのApidog CLIで示されているように apidog run ... -r junit をCIに組み込んでください。エージェントの設定については、Claude CodeでのApidog CLIを参照してください。 よくある質問 RTX 4090でNemotron 3.5 Transcribeを実行できますか？ NVIDIAの互換性リストにはありません。リストにある最小のコンシューマーカードは、NVFP4チェックポイントのRTX 5090です。 Macで動作しますか？ 2026年10月29日時点で、Apple Siliconをサポートする公式ランタイムはありません。 商用製品でのローカル利用は無料ですか？ OpenMDW-1.1ライセンスは商用利用と非商用利用を許可しています。出荷前に、帰属表示の条件についてライセンス本文を読んでください。 ライブ音声を文字起こしできますか？ いいえ。録音ファイル用のオフラインモデルです。NVIDIAはライブ用途向けに別のNemotron 3.5 ASR Streamingモデルを提供しています。 vLLMとSGLang-Omniではどちらが速いですか？ NVIDIAは直接比較を公開していません。タイムスタンプ、話者、翻訳が必要な場合はvLLMを、最も単純なテキスト専用サーバーが必要な場合はSGLang-Omniを使ってください。 他のローカルモデルと比べてどうランク付けされますか？ 10月28日のOpen ASR Leaderboardで、すべてのオープンウェイトモデルをリードしています。最高のオープンソース音声テキスト変換モデルのリストでは、Qwen3-ASR、Granite Speech、Canary-Qwen、Voxtral、Whisperと比較しています。 実行する GPUに合ったチェックポイントをダウンロードし、1時間の音声制限とタイムスタンプミドルウェアを使ってvLLMを起動し、OpenAIクライアント経由でWAVファイルを送信してください。次に、エージェントにローカルエンドポイントに対してApidogチェックを実行させ、実際の録音を任せる前に、アップグレードのたびにまだ文字起こしできることを証明しましょう。 {% embed youtube.com %}

Cách chạy Nemotron 3.5 Transcribe cục bộ: Một GPU, file 1 giờ, không tốn phí API Cách chạy Nemotron 3.5 Transcribe cục bộ: Một GPU, file 1 giờ, không tốn phí API Bạn có thể chạy Nemotron 3.5 Transcribe mới của NVIDIA trên một GPU và phiên âm các bản ghi dài hàng giờ bằng 40 ngôn ngữ mà không phải trả tiền theo phút hay gửi âm thanh cho bất kỳ ai. Đây là mô hình trọng số mở hàng đầu trên Bảng xếp hạng ASR mở của Hugging Face, được phát hành theo giấy phép OpenMDW-1.1 thân thiện với thương mại, và NVIDIA đã công bố trọng số BF16, FP8 và NVFP4 vào ngày phát hành, 28 tháng 10 năm 2026. {% cta apidog.com %} Dùng thử Apidog ngay hôm nay {% endcta %} Hướng dẫn này đề cập đến phần cứng bạn cần, ba cách chạy mô hình (vLLM, SGLang-Omni và Transformers), các lệnh chính xác từ thẻ mô hình chính thức, và những cài đặt khiến người ta gặp khó khăn. Tất cả các lệnh được sao chép từ thẻ vào ngày 29 tháng 10 năm 2026. Mới làm quen với mô hình? Hướng dẫn miễn phí của chúng tôi giải thích nó làm gì và xếp hạng ra sao. Không có GPU? API được lưu trữ có giá $0,002 mỗi phút. !Architecture diagram of Nemotron 3.5 Transcribe: audio feeds a 600M ASR encoder and a speaker encoder, which feed a hybrid Mamba-2 and Transformer mixture-of-experts language model with 34B total and about 4.3B active parameters Tại sao nên chạy cục bộ - Không hóa đơn theo phút. Với mức $0,002 mỗi phút của DeepInfra, 10.000 giờ tốn $1.200. Trên GPU của riêng bạn, chi phí chỉ là tiền điện. - Quyền riêng tư. Cuộc gọi khách hàng, ghi âm y tế và bản ghi pháp lý không bao giờ rời khỏi mạng của bạn. - Bộ tính năng đầy đủ. Đường dẫn vLLM hỗ trợ dấu thời gian từng từ, phân tách người nói, dịch và gợi ý ngữ cảnh. Một số điểm cuối được lưu trữ chỉ cung cấp một phần trong số đó. - Tinh chỉnh. Bài đăng ra mắt của NVIDIA cho thấy một bản tinh chỉnh y tế giảm WER từ 30,8% xuống 11,8%. Bạn chỉ có thể làm điều đó với trọng số. Phần cứng: những gì chạy được nó Mô hình có tổng cộng 34B tham số và khoảng 4,3B tham số hoạt động, vì vậy trọng số rất lớn mặc dù tính toán cho mỗi token nhỏ. Mục tiêu tham chiếu của NVIDIA là một H100 80 GB ở BF16. Bảng tương thích trên thẻ mô hình: | GPU | Độ chính xác | Môi trường chạy | | --- | --- | --- | | H100, H200 | BF16, FP8 | vLLM 0.32.0, Transformers, SGLang-Omni 0.2.0 (H100) | | A100 80 GB | BF16 | vLLM 0.32.0, Transformers | | L40S 48 GB | FP8 | vLLM 0.32.0, Transformers | | B200, GB200, GB300 DGX Station | BF16, FP8, NVFP4 | vLLM 0.32.0, Transformers | | DGX Spark (GB10), Jetson Thor | BF16, FP8, NVFP4 | vLLM 0.32.0, Transformers | | RTX PRO 6000 Blackwell Server | BF16, FP8, NVFP4 | vLLM, Transformers, SGLang-Omni, GGML | | RTX 5090 (32 GB) | chỉ NVFP4 | vLLM 0.32.0, Transformers, GGML | Chọn checkpoint cho card của bạn: - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B (BF16) - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-FP8 - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4 Card 24 GB như RTX 4090 không có trong danh sách, và Apple Silicon không được hỗ trợ. Hệ điều hành là Linux, hoặc Windows thông qua WSL2. GGML cũng chạy trên Windows gốc, nhưng bài đăng ra mắt của NVIDIA nói rằng bản dựng cục bộ NeMo-Speech.cpp vẫn "sắp ra mắt." Tùy chọn 1: Máy chủ vLLM (được khuyến nghị) vLLM cung cấp cho bạn một điểm cuối tương thích OpenAI với xử lý theo lô, tệp dài một giờ, giải mã suy đoán và dấu thời gian từng từ. Nó cần vLLM 0.32.0 cùng nhánh chính NeMo Speech mới nhất, cung cấp plugin vLLM của mô hình. !Comparison diagram of the three local run options for Nemotron 3.5 Transcribe: vLLM with NeMo Speech for full features, SGLang-Omni for plain transcription, and Transformers for scripts Cài đặt Thẻ sử dụng uv và Python 3.13: git clone https://github.com/NVIDIA-NeMo/Speech.git nemo-speech cd nemo-speech uv venv --python 3.13 .venv UV_PROJECT_ENVIRONMENT=$PWD/.venv uv sync --extra asr --extra vllm --locked --no-install-project uv pip install --python .venv/bin/python --no-deps -e . uv pip install --python .venv/bin/python --no-deps av==19.0.2 uv pip install --python .venv/bin/python --no-deps opencv-python-headless==5.0.0.93 source .venv/bin/activate hf download nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B --local-dir ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B PyAV (av) cho phép nó giải mã M4A và các định dạng nén khác mà không cần FFmpeg hệ thống. OpenCV có mặt vì vLLM cần nó và cài đặt phụ thuộc của NeMo bỏ sót nó. Khởi động máy chủ export VLLM_PLUGINS=nemo_speechlm export VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 export VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=180 export VLLM_USE_V2_MODEL_RUNNER=0 vllm serve ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --served-model-name nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.94 \ --max-model-len 65536 --max-num-batched-tokens 4096 --max-num-seqs 192 \ --enable-chunked-prefill --no-enable-prefix-caching \ --use-replayssm --mamba-backend flashinfer --mamba-cache-mode none \ --mm-processor-cache-gb 0 --no-async-scheduling \ --limit-mm-per-prompt '{"audio":{"count":6,"length":120000000}}' \ --speculative-config '{"method":"mtp","num_speculative_tokens":8}' \ --reasoning-parser nemotron_v3 \ --trust-request-mm-kwargs \ --middleware nemo.collections.speechlm2.vllm.salm.ctc_serving.ctc_timestamp_middleware Những cờ đáng hiểu: - VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 nâng giới hạn âm thanh của vLLM từ 600 giây lên một giờ. Quên nó và các tệp dài sẽ thất bại. - VLLM_USE_V2_MODEL_RUNNER=0 giữ trình chạy V1, thứ mà dấu thời gian từng từ cần. - --speculative-config bật dự đoán nhiều token với tám token nháp (MTP8). Trên H100, NVIDIA đo được RTFx tiếng Anh theo lô là 270 với MTP8 và 410 với MTP2, vì vậy hãy thử "num_speculative_tokens":2 nếu bạn quan tâm đến thông lượng hơn độ trễ. - --middleware ... ctc_timestamp_middleware và --trust-request-mm-kwargs cho phép các yêu cầu yêu cầu dấu thời gian từng từ. - Các cài đặt nhắm đến H100 80 GB. Trên các card khác, hãy giảm --gpu-memory-utilization, --max-model-len hoặc --max-num-seqs. Máy chủ lắng nghe trên 127.0.0.1:8000. Thêm --host 0.0.0.0 để truy cập từ các máy khác trong mạng của bạn. Gửi một tệp import base64 from openai import OpenAI client = OpenAI(base_url="http://127.0.0.1:8000/v1", [REDACTED CREDENTIAL]) audio = base64.b64encode(open("standup.wav", "rb").read()).decode() response = client.chat.completions.create( model="nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B", messages=[{"role": "user", "content": [ {"type": "text", "text": "Transcribe the following audio."}, {"type": "audio_url", "audio_url": {"url": f"data:audio/wav;base64,{audio}"}}, ]}], temperature=0, extra_body={"chat_template_kwargs": {"enable_thinking": False}, "skip_special_tokens": False}, ) print(response.choices[0].message.content) Thay hướng dẫn văn bản bằng bất kỳ gợi ý tác vụ nào. Đối với cuộc họp có nhiều giọng nói: Transcribe verbatim, including fillers. Write numbers as spoken, without punctuation or capitalization. Start every speaker turn with <spk:0>, <spk:1>, etc. Keep one stable tag per speaker. Output only the speaker-tagged transcript. Yêu cầu dấu thời gian từng từ có các quy tắc: một bản ghi cho mỗi yêu cầu, stream=false và n=1. Phản hồi sau đó bao gồm một đối tượng ctc_timestamps với các từ, thời gian bắt đầu và kết thúc tính bằng giây, và thẻ người nói. Tùy chọn 2: SGLang-Omni để phiên âm đơn giản Nếu bạn chỉ cần văn bản từ toàn bộ tệp, SGLang-Omni 0.2.0 đơn giản hơn và không cần NeMo. Nó đã được kiểm chứng trên một H100 80 GB và một RTX PRO 6000 Blackwell với checkpoint BF16 hoặc FP8. sgl-omni serve \ --model-path nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --model-name nemotron-transcribe \ --mem-fraction-static 0.85 \ --port 8000 curl http://localhost:8000/v1/audio/transcriptions \ -F model=nemotron-transcribe \ -F file=@audio.wav Route /v1/audio/transcriptions đó khớp với hình dạng mà Whisper API của OpenAI sử dụng, vì vậy mã máy khách Whisper hiện có thường chỉ cần một base URL mới. Đánh đổi: không có dấu thời gian từng từ, nhãn người nói, dịch hoặc giải mã suy đoán trong tích hợp này. Tùy chọn 3: Transformers trong một script Đối với notebook và các tác vụ một lần, hãy chạy nó trong tiến trình với Hugging Face Transformers được cài đặt từ nguồn: pip install git+https://github.com/huggingface/transformers from transformers import AutoModelForMultimodalLM, AutoProcessor model_id = "nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto") [REDACTED PROMPT] "and verbal fillers. Put speaker tags <spk:0> <spk:1> <spk:2> <spk:3> on the transcription.") inputs = processor.apply_transcription_request(["podcast.mp3"], [REDACTED PROMPT].device) outputs = model.generate(**inputs, max_new_tokens=4096) print(processor.decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=False)) Đây là cách dễ nhất để bắt đầu và là cách chạy chậm nhất ở quy mô lớn. Tăng max_new_tokens cho các tệp dài: một giờ dày đặc có thể tạo ra khoảng 20.000 token. Nó nhanh đến mức nào trên GPU của riêng bạn? Con số tiêu đề của NVIDIA là 2.238 giây âm thanh mỗi giây trên một RTX PRO 6000 Blackwell, với trọng số NVFP4 và giải mã suy đoán DFlash2, và 0,13 giây để trả về một clip 30 giây. Đọc chú thích: những con số đó đến từ NIM được tối ưu hóa, mà các thay đổi phục vụ của nó vẫn chưa mã nguồn mở. Thẻ nói rằng tích hợp vLLM mở dự kiến sẽ chậm hơn khoảng hai lần. Đối với phiên âm tiếng Anh theo lô trên H100 ở BF16, thẻ liệt kê các số RTFx sau (giây âm thanh mỗi giây tính toán): | Giải mã suy đoán | RTFx | | --- | --- | | MTP2 | 410 | | DFlash2, K=7 | 340 | | MTP8 | 270 | | DFlash2, K=15 | 260 | Ngay cả hàng chậm nhất cũng xử lý một giờ âm thanh trong dưới 15 giây thời gian GPU khi làm việc theo lô. Nếu bạn muốn mô hình nháp DFlash2, lưu ý rằng vLLM 0.32.0 cần một bản vá nhỏ từ thẻ, DFlash2 vô hiệu hóa dấu thời gian từng từ, và bạn không nên kết hợp nó với MTP. Các vấn đề thường gặp và cách khắc phục - Tệp dài bị cắt ở 10 phút. Bạn đã bỏ qua VLLM_MAX_AUDIO_DECODE_DURATION_S=3600. - Không có dấu thời gian trong phản hồi. Kiểm tra VLLM_USE_V2_MODEL_RUNNER=0, cờ --middleware, và rằng yêu cầu có stream=false và một bản ghi. - Hết bộ nhớ trên các card nhỏ hơn. Chuyển sang checkpoint FP8 hoặc NVFP4 và giảm --max-num-seqs. - Các cụm từ lặp lại ở cuối bản phiên âm. Thẻ khuyến nghị một rào chắn vòng lặp sinh trên đầu ra cuối cùng. Giới hạn max_tokens và cắt bỏ phần lặp lại. - Sai chính tả tên. Thêm Context terms: <names> vào gợi ý. - Giọng nói nền trong bản phiên âm. Hãy thử gợi ý trường gần thử nghiệm từ thẻ, giúp giảm WER trên âm thanh hỗn hợp từ 27,32% xuống 4,78% trong thử nghiệm của NVIDIA. Nếu bạn mới làm quen với chính vLLM, bài giải thích về vLLM của chúng tôi đề cập đến cách nó xử lý theo lô các yêu cầu và quản lý bộ nhớ GPU. Để tác nhân lập trình của bạn kiểm tra máy chủ cục bộ bằng Apidog CLI Một mô hình cục bộ chỉ hữu ích nếu điểm cuối của nó vẫn chính xác qua các nâng cấp vLLM, tinh chỉnh gợi ý và hoán đổi checkpoint. Hãy để tác nhân lập trình của bạn đảm nhận việc đó với Apidog CLI, chạy kiểm tra đối với http://127.0.0.1:8000 từ cùng một máy. npm install -g apidog-cli@latest apidog login --with-token <YOUR_TOKEN> npx -y skills add https://apidog.com # pick your agent when prompted Sau đó đưa cho tác nhân của bạn một gợi ý: In Apidog project <id>, add POST {{baseUrl}}/v1/chat/completions for my local Nemotron 3.5 Transcribe server. Create test cases that send samples/en.wav and samples/two-speakers.wav as base64 audio_url content with temperature 0, and assert status 200, a non-empty choices[0].message.content, and at least two distinct <spk:N> tags for the two-speaker file. Run them against the local environment and report failures. Tác nhân đọc schema, xác thực payload của nó, viết các ca kiểm thử và chạy chúng: apidog cli-schema get test-case-create apidog cli-schema validate test-case-create --file case.json apidog test-case create --project <id> --file case.json apidog run --test-case <caseId> --project <id> -e <envId> -r cli,json Chạy nó sau mỗi lần nâng cấp máy chủ, hoặc kết nối apidog run ... -r junit vào CI như được hiển thị trong Apidog CLI trong pipeline CI/CD. Để thiết lập tác nhân, xem Apidog CLI trong Claude Code. Câu hỏi thường gặp Tôi có thể chạy Nemotron 3.5 Transcribe trên RTX 4090 không? Nó không có trong danh sách tương thích của NVIDIA. Card tiêu dùng nhỏ nhất được liệt kê là RTX 5090 với checkpoint NVFP4. Nó có chạy trên Mac không? Không có môi trường chạy chính thức nào hỗ trợ Apple Silicon tính đến ngày 29 tháng 10 năm 2026. Sử dụng cục bộ có miễn phí cho sản phẩm thương mại không? Giấy phép OpenMDW-1.1 cho phép sử dụng thương mại và phi thương mại. Đọc văn bản giấy phép để biết điều khoản ghi công trước khi phát hành. Nó có thể phiên âm âm thanh trực tiếp không? Không. Đây là mô hình ngoại tuyến cho các tệp đã ghi. NVIDIA cung cấp một mô hình Nemotron 3.5 ASR Streaming riêng cho sử dụng trực tiếp. Cái nào nhanh hơn, vLLM hay SGLang-Omni? NVIDIA không công bố so sánh trực tiếp. Sử dụng vLLM khi bạn cần dấu thời gian, người nói hoặc dịch, và SGLang-Omni cho máy chủ chỉ văn bản đơn giản nhất. Nó xếp hạng như thế nào so với các mô hình cục bộ khác? Nó dẫn đầu mọi mô hình trọng số mở trên Bảng xếp hạng ASR mở ngày 28 tháng 10. Danh sách các mô hình chuyển giọng nói thành văn bản mã nguồn mở tốt nhất của chúng tôi so sánh nó với Qwen3-ASR, Granite Speech, Canary-Qwen, Voxtral và Whisper. Chạy nó Tải xuống checkpoint phù hợp với GPU của bạn, khởi động vLLM với giới hạn âm thanh một giờ và middleware dấu thời gian, và gửi tệp WAV qua máy khách OpenAI. Sau đó để tác nhân của bạn chạy kiểm tra Apidog đối với điểm cuối cục bộ để mỗi lần nâng cấp chứng minh nó vẫn phiên âm trước khi bạn tin tưởng nó với các bản ghi thực. {% embed youtube.com %}

Nemotron 3.5 Transcribe lokal ausführen: Eine GPU, 1-Stunden-Dateien, keine API-Kosten Sie können NVIDIas neues Nemotron 3.5 Transcribe auf einer GPU ausführen und stundenlange Aufnahmen in 40 Sprachen transkribieren, ohne pro Minute zu zahlen oder Audio an irgendjemanden zu senden. Es ist das beste Open-Weight-Modell im Hugging Face Open ASR Leaderboard, es wird unter der kommerziell nutzbaren OpenMDW-1.1-Lizenz veröffentlicht, und NVIDIA hat am Veröffentlichungstag, dem 28. Oktober 2026, BF16-, FP8- und NVFP4-Gewichte publiziert. {% cta apidog.com %} Testen Sie Apidog noch heute {% endcta %} Dieser Leitfaden behandelt die benötigte Hardware, die drei Möglichkeiten, es auszuführen (vLLM, SGLang-Omni und Transformers), die exakten Befehle aus der offiziellen Modellkarte und die Einstellungen, die Leute aus dem Tritt bringen. Alle Befehle wurden am 29. Oktober 2026 aus der Modellkarte kopiert. Neu beim Modell? Unser kostenloser Leitfaden erklärt, was es leistet und wie es abschneidet. Keine GPU? Die gehostete API kostet 0,002 $ pro Minute. !Architecture diagram of Nemotron 3.5 Transcribe: audio feeds a 600M ASR encoder and a speaker encoder, which feed a hybrid Mamba-2 and Transformer mixture-of-experts language model with 34B total and about 4.3B active parameters Warum lokal ausführen - Keine Abrechnung pro Minute. Bei DeepInfras 0,002 $ pro Minute kosten 10.000 Stunden 1.200 $. Auf Ihrer eigenen GPU kostet es Strom. - Datenschutz. Kundentelefonate, medizinische Diktate und juristische Aufnahmen verlassen niemals Ihr Netzwerk. - Vollständiger Funktionsumfang. Der vLLM-Pfad unterstützt Wort-Zeitstempel, Sprecherdiarisierung, Übersetzung und Kontext-Prompts. Einige gehostete Endpunkte bieten nur einen Teil davon. - Fine-Tuning. NVIDIAs Launch-Post zeigt ein medizinisches Fine-Tune, das die WER von 30,8 % auf 11,8 % senkt. Das können Sie nur mit den Gewichten machen. Hardware: Womit es läuft Das Modell hat insgesamt 34B Parameter und etwa 4,3B aktive, daher sind die Gewichte groß, auch wenn der Rechenaufwand pro Token klein ist. NVIDIAs Referenzziel ist eine H100 80 GB in BF16. Die Kompatibilitätstabelle auf der Modellkarte: | GPU | Präzisionen | Laufzeitumgebungen | | --- | --- | --- | | H100, H200 | BF16, FP8 | vLLM 0.32.0, Transformers, SGLang-Omni 0.2.0 (H100) | | A100 80 GB | BF16 | vLLM 0.32.0, Transformers | | L40S 48 GB | FP8 | vLLM 0.32.0, Transformers | | B200, GB200, GB300 DGX Station | BF16, FP8, NVFP4 | vLLM 0.32.0, Transformers | | DGX Spark (GB10), Jetson Thor | BF16, FP8, NVFP4 | vLLM 0.32.0, Transformers | | RTX PRO 6000 Blackwell Server | BF16, FP8, NVFP4 | vLLM, Transformers, SGLang-Omni, GGML | | RTX 5090 (32 GB) | NVFP4 only | vLLM 0.32.0, Transformers, GGML | Wählen Sie den Checkpoint für Ihre Karte: - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B (BF16) - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-FP8 - nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4 Eine 24-GB-Karte wie die RTX 4090 steht nicht auf der Liste, und Apple Silicon wird nicht unterstützt. Betriebssysteme sind Linux oder Windows über WSL2. GGML läuft auch unter nativem Windows, aber NVIDIAs Launch-Post sagt, dass der lokale NeMo-Speech.cpp-Build noch „coming soon" ist. Option 1: vLLM-Server (empfohlen) vLLM bietet Ihnen einen OpenAI-kompatiblen Endpunkt mit Batching, einstündigen Dateien, spekulativem Decoding und Wort-Zeitstempeln. Es benötigt vLLM 0.32.0 plus den neuesten NeMo Speech Main-Branch, der das vLLM-Plugin des Modells bereitstellt. !Comparison diagram of the three local run options for Nemotron 3.5 Transcribe: vLLM with NeMo Speech for full features, SGLang-Omni for plain transcription, and Transformers for scripts Installation Die Modellkarte verwendet uv und Python 3.13: git clone https://github.com/NVIDIA-NeMo/Speech.git nemo-speech cd nemo-speech uv venv --python 3.13 .venv UV_PROJECT_ENVIRONMENT=$PWD/.venv uv sync --extra asr --extra vllm --locked --no-install-project uv pip install --python .venv/bin/python --no-deps -e . uv pip install --python .venv/bin/python --no-deps av==19.0.2 uv pip install --python .venv/bin/python --no-deps opencv-python-headless==5.0.0.93 source .venv/bin/activate hf download nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B --local-dir ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B PyAV (av) ermöglicht das Dekodieren von M4A und anderen komprimierten Formaten ohne systemweites FFmpeg. OpenCV ist dabei, weil vLLM es benötigt und NeMOs Abhängigkeitseinstellungen es auslassen. Server starten export VLLM_PLUGINS=nemo_speechlm export VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 export VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=180 export VLLM_USE_V2_MODEL_RUNNER=0 vllm serve ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --served-model-name nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.94 \ --max-model-len 65536 --max-num-batched-tokens 4096 --max-num-seqs 192 \ --enable-chunked-prefill --no-enable-prefix-caching \ --use-replayssm --mamba-backend flashinfer --mamba-cache-mode none \ --mm-processor-cache-gb 0 --no-async-scheduling \ --limit-mm-per-prompt '{"audio":{"count":6,"length":120000000}}' \ --speculative-config '{"method":"mtp","num_speculative_tokens":8}' \ --reasoning-parser nemotron_v3 \ --trust-request-mm-kwargs \ --middleware nemo.collections.speechlm2.vllm.salm.ctc_serving.ctc_timestamp_middleware Die Flags, die man verstehen sollte: - VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 erhöht das Audio-Limit von vLLM von 600 Sekunden auf eine Stunde. Vergessen Sie es, und lange Dateien schlagen fehl. - VLLM_USE_V2_MODEL_RUNNER=0 behält den V1-Runner bei, den Wort-Zeitstempel benötigen. - --speculative-config aktiviert Multi-Token-Vorhersage mit acht Draft-Tokens (MTP8). Auf einer H100 hat NVIDIA eine gebatchte englische RTFx von 270 mit MTP8 und 410 mit MTP2 gemessen, probieren Sie also "num_speculative_tokens":2, wenn Ihnen Durchsatz wichtiger ist als Latenz. - --middleware ... ctc_timestamp_middleware und --trust-request-mm-kwargs ermöglichen es Anfragen, Wort-Zeitstempel anzufordern. - Die Einstellungen sind auf eine 80-GB-H100 ausgelegt. Senken Sie auf anderen Karten --gpu-memory-utilization, --max-model-len oder --max-num-seqs. Der Server lauscht auf 127.0.0.1:8000. Fügen Sie --host 0.0.0.0 hinzu, um ihn von anderen Maschinen in Ihrem Netzwerk zu erreichen. Datei senden import base64 from openai import OpenAI client = OpenAI(base_url="http://127.0.0.1:8000/v1", [REDACTED CREDENTIAL]) audio = base64.b64encode(open("standup.wav", "rb").read()).decode() response = client.chat.completions.create( model="nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B", messages=[{"role": "user", "content": [ {"type": "text", "text": "Transcribe the following audio."}, {"type": "audio_url", "audio_url": {"url": f"data:audio/wav;base64,{audio}"}}, ]}], temperature=0, extra_body={"chat_template_kwargs": {"enable_thinking": False}, "skip_special_tokens": False}, ) print(response.choices[0].message.content) Ersetzen Sie die Textanweisung durch einen beliebigen Task-Prompt. Für ein Meeting mit mehreren Stimmen: Transcribe verbatim, including fillers. Write numbers as spoken, without punctuation or capitalization. Start every speaker turn with <spk:0>, <spk:1>, etc. Keep one stable tag per speaker. Output only the speaker-tagged transcript. Anfragen mit Wort-Zeitstempeln haben Regeln: eine Aufnahme pro Anfrage, stream=false und n=1. Die Antwort enthält dann ein ctc_timestamps-Objekt mit Wörtern, Start- und Endzeiten in Sekunden sowie Sprecher-Tags. Option 2: SGLang-Omni für reine Transkription Wenn Sie nur Text aus ganzen Dateien benötigen, ist SGLang-Omni 0.2.0 einfacher und benötigt kein NeMo. Es ist auf einer H100 80 GB und einer RTX PRO 6000 Blackwell mit dem BF16- oder FP8-Checkpoint validiert. sgl-omni serve \ --model-path nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B \ --model-name nemotron-transcribe \ --mem-fraction-static 0.85 \ --port 8000 curl http://localhost:8000/v1/audio/transcriptions \ -F model=nemotron-transcribe \ -F file=@audio.wav Diese /v1/audio/transcriptions-Route entspricht der Form, die OpenAIs Whisper-API verwendet, sodass vorhandener Whisper-Client-Code oft nur eine neue Basis-URL benötigt. Der Kompromiss: keine Wort-Zeitstempel, Sprecherbezeichnungen, Übersetzung oder spekulatives Decoding in dieser Integration. Option 3: Transformers in einem Skript Für Notebooks und einmalige Jobs führen Sie es im Prozess mit aus dem Quellcode installierten Hugging Face Transformers aus: pip install git+https://github.com/huggingface/transformers from transformers import AutoModelForMultimodalLM, AutoProcessor model_id = "nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto") [REDACTED PROMPT]. Include all spoken words " "and verbal fillers. Put speaker tags <spk:0> <spk:1> <spk:2> <spk:3> on the transcription.") inputs = processor.apply_transcription_request(["podcast.mp3"], [REDACTED PROMPT] outputs = model.generate(**inputs, max_new_tokens=4096) print(processor.decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=False)) Es ist der einfachste Einstieg und die langsamste Art, es im großen Maßstab auszuführen. Erhöhen Sie max_new_tokens für lange Dateien: Eine dichte Stunde kann etwa 20.000 Tokens erzeugen. Wie schnell ist es auf Ihrer eigenen GPU? NVIDIAs Schlagzeile ist 2.238 Sekunden Audio pro Sekunde auf einer RTX PRO 6000 Blackwell, mit NVFP4-Gewichten und DFlash2-spezkulativem Decoding, sowie 0,13 Sekunden, um einen 30-Sekunden-Clip zurückzugeben. Lesen Sie die Fußnote: Diese Zahlen stammen vom optimierten NIM, dessen Serving-Änderungen noch nicht Open Source sind. Die Modellkarte sagt, dass die offene vLLM-Integration voraussichtlich etwa zweimal langsamer ist. Für gebatchte englische Transkription auf H100 in BF16 listet die Modellkarte diese RTFx-Zahlen auf (Sekunden Audio pro Sekunde Rechenzeit): | Spekulatives Decoding | RTFx | | --- | --- | | MTP2 | 410 | | DFlash2, K=7 | 340 | | MTP8 | 270 | | DFlash2, K=15 | 260 | Selbst die langsamste Zeile schafft eine Stunde Audio in unter 15 Sekunden GPU-Zeit bei gebatchter Arbeit. Wenn Sie das DFlash2-Draft-Modell möchten, beachten Sie, dass vLLM 0.32.0 einen kleinen Patch aus der Modellkarte benötigt, DFlash2 Wort-Zeitstempel deaktiviert und Sie es nicht mit MTP kombinieren sollten. Häufige Probleme und Lösungen - Lange Dateien werden bei 10 Minuten abgeschnitten. Sie haben VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 übersprungen. - Keine Zeitstempel in der Antwort. Prüfen Sie VLLM_USE_V2_MODEL_RUNNER=0, das --middleware-Flag und ob die Anfrage stream=false und eine Aufnahme hat. - Speichermangel auf kleineren Karten. Wechseln Sie zum FP8- oder NVFP4-Checkpoint und senken Sie --max-num-seqs. - Wiederholte Phrasen am Ende eines Transkripts. Die Modellkarte empfiehlt eine Guardrail gegen Generierungsschleifen bei den finalen Ausgaben. Begrenzen Sie max_tokens und kürzen Sie Wiederholungen. - Falsche Schreibweise von Namen. Fügen Sie Context terms: <names> zum Prompt hinzu. - Hintergrundstimmen im Transkript. Probieren Sie den experimentellen Near-Field-Prompt aus der Modellkarte, der in NVIDIAs Test die WER bei gemischtem Audio von 27,32 % auf 4,78 % senkte. Wenn Sie neu bei vLLM selbst sind, behandelt unser vLLM-Erklärer, wie es Anfragen batcht und GPU-Speicher verwaltet. Lassen Sie Ihren Coding-Agenten den lokalen Server mit Apidog CLI testen Ein lokales Modell ist nur nützlich, wenn sein Endpunkt über vLLM-Upgrades, Prompt-Anpassungen und Checkpoint-Wechsel hinweg korrekt bleibt. Lassen Sie Ihren Coding-Agenten das mit Apidog CLI übernehmen, indem er Prüfungen gegen http://127.0.0.1:8000 von derselben Maschine aus ausführt. npm install -g apidog-cli@latest apidog login --with-token <YOUR_TOKEN> npx -y skills add https://apidog.com # pick your agent when prompted Geben Sie Ihrem Agenten dann einen [REDACTED PROMPT] In Apidog project <id>, add POST {{baseUrl}}/v1/chat/completions for my local Nemotron 3.5 Transcribe server. Create test cases that send samples/en.wav and samples/two-speakers.wav as base64 audio_url content with temperature 0, and assert status 200, a non-empty choices[0].message.content, and at least two distinct <spk:N> tags for the two-speaker file. Run them against the local environment and report failures. Der Agent liest das Schema, validiert seine Payload, schreibt die Testfälle und führt sie aus: apidog cli-schema get test-case-create apidog cli-schema validate test-case-create --file case.json apidog test-case create --project <id> --file case.json apidog run --test-case <caseId> --project <id> -e <envId> -r cli,json Führen Sie es nach jedem Server-Upgrade aus, oder verdrahten Sie apidog run ... -r junit in CI, wie in Apidog CLI in einer CI/CD-Pipeline gezeigt. Zur Agenten-Einrichtung siehe Apidog CLI in Claude Code. FAQ Kann ich Nemotron 3.5 Transcribe auf einer RTX 4090 ausführen? Sie steht nicht auf NVIDIAs Kompatibilitätsliste. Die kleinste aufgeführte Consumer-Karte ist die RTX 5090 mit dem NVFP4-Checkpoint. Läuft es auf einem Mac? Stand 29. Oktober 2026 unterstützt keine offizielle Laufzeitumgebung Apple Silicon. Ist die lokale Nutzung für kommerzielle Produkte kostenlos? Die OpenMDW-1.1-Lizenz erlaubt kommerzielle und nichtkommerzielle Nutzung. Lesen Sie den Lizenztext zu Attributionsbedingungen, bevor Sie ausliefern. Kann es Live-Audio transkribieren? Nein. Es ist ein Offline-Modell für aufgezeichnete Dateien. NVIDIA bietet ein separates Nemotron 3.5 ASR Streaming-Modell für Live-Nutzung an. Was ist schneller, vLLM oder SGLang-Omni? NVIDIA veröffentlicht keinen direkten Vergleich. Verwenden Sie vLLM, wenn Sie Zeitstempel, Sprecher oder Übersetzung benötigen, und SGLang-Omni für den einfachsten reinen Textserver. Wie schneidet es im Vergleich zu anderen lokalen Modellen ab? Es führt jedes Open-Weight-Modell im Open ASR Leaderboard vom 28. Oktober an. Unsere Liste der besten Open-Source-Speech-to-Text-Modelle vergleicht es mit Qwen3-ASR, Granite Speech, Canary-Qwen, Voxtral und Whisper. Ausführen Laden Sie den Checkpoint herunter, der zu Ihrer GPU passt, starten Sie vLLM mit dem Ein-Stunden-Audio-Limit und der Zeitstempel-Middleware und senden Sie eine WAV-Datei über den OpenAI-Client. Lassen Sie dann Ihren Agenten Apidog-Prüfungen gegen den lokalen Endpunkt ausführen, damit jedes Upgrade beweist, dass es noch transkribiert, bevor Sie ihm echte Aufnahmen anvertrauen. {% embed youtube.com %}

NVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters NVIDIA releases Nemotron 3.5 Transcribe, an open 34B-total (~4.3B active) speech LLM for 40-language transcription under OpenMDW-1.1 (4.71% WER on English Open ASR) NVIDIA released Nemotron 3.5 Transcribe, an open-weights speech LLM built for transcription. One model handles 40 languages and recordings up to 1 hour, and a text instruction picks the output style per request. It can replace separate ASR, diarization and alignment tools. What it does: - Instruction-driven output: verbatim, written form, context terms, speaker tags or translation, with word timestamps on request - A frozen Nemotron 3 Diarization model (up to 8 speakers) feeds "who spoke when" into a 0.6B ASR encoder every 80 ms - LLM backbone is Nemotron 3.5 Lightning 30B-A3B, a hybrid Mamba-Transformer MoE with only 6 attention layers to keep the KV cache small on hour-long audio - Shared multi-token prediction layer for speculative decoding, plus a 50M CTC adapter for word timestamps - Weights in BF16, FP8, NVFP4 and GGUF; runs on vLLM, Transformers and SGLang-Omni Key numbers (self-reported by NVIDIA): - English Open ASR, 9 public sets: 4.71% mean WER vs 4.94% for Qwen3-ASR-1.7B - Multilingual Open ASR, 7 languages: 4.62% vs 6.49% for Qwen3-ASR-1.7B - Long-form ASR: 10.98% vs 13.37% for Parakeet-TDT-0.6B-v3 - Multi-speaker full sessions: 15.80% cpWER vs 16.83% for DiCoW v3.3 with DiariZen - Contextual ASR: an entity list cuts named-entity WER from 12.21% to 2.20% - BF16 fits 1x H100 80 GB; NVFP4 (~21 GB) runs on an RTX 5090 The catch: it is offline only, and commercial APIs still lead (OtterAI speech-v3.1 at 3.88% English, ElevenLabs Scribe v2 at 3.56% multilingual). Mandarin far-field meetings are weak at 34.95% cpCER on AliMeeting. The NIM serving optimizations are not open source yet, and NVIDIA expects the release-day vLLM path to be roughly 2x slower. Full analysis: marktechpost.com Model: huggingface.co Paper: research.nvidia.com Repo: NVIDIA-NeMo/Speech

Blog: huggingface.co Code: NVIDIA-NeMo/Speech

Run it today with NeMo Speech's vLLM 0.32.0 plugin, OpenAI-compatible chat completions and MTP speculative decoding. Deployment guide: github.com/NVIDIA-NeMo/Speech/blob/main/docs/sour…