baidu

How to Run ERNIE 5.5 Flash Base Locally (8 H100s or 2 B300s, and It Won't Chat) ERNIE 5.5 Flash Base won't chat with you. Baidu's new 280B open-weight model on Hugging Face is the raw end-of-pre-training checkpoint, it's served through /v1/completions, the smaller of its two repos is a 288 GB download, and just holding the weights takes two 288 GB GPUs or eight 80 GB ones. If you want to run ERNIE 5.5 Flash Base locally and point a chat UI at it, save your bandwidth. If you fine-tune, it's worth a look: on Baidu's own table it beats DeepSeek V4 Flash Base on 26 of 30 benchmarks. DeepSeek's repo ships MIT, though, and Baidu's license forbids use in the EU, the UK and South Korea. I haven't run this model. The technical details below come from the release page, the model card, Baidu's plugin README, the SGLang cookbook and the license text. Anything else is linked or named where I use it, and the couple of memory numbers I worked out myself are labeled as mine. What Baidu released on Hugging Face - 280B total parameters, 14B active per token, 43 layers (release page) - 33.5T tokens of pre-training, with no mid- or post-training - Trained on contexts up to 128K tokens, with up to 1M for inference and fine-tuning - FP8 repo at 288 GB, BF16 repo at 573 GB Chat, reasoning and tool calling belong to ERNIE-5.5-Flash, which the plugin README lists as "Coming soon". Six of the 43 layers use MLA with DSA sparse attention, and the other 37 use Heavily Compressed Attention, which folds every 128 tokens into one cache entry plus a 128-token sliding window. Every layer but the first is MoE, with 512 routed experts (top-10) and one shared expert. ECHO replaces the multi-token prediction module by reusing the final layer to predict future tokens, and that same layer doubles as the speculative-decoding draft model. Baidu claims a higher draft acceptance rate than an auxiliary MTP module but publishes no number for it. How much VRAM does ERNIE 5.5 Flash Base need? The model card gives no VRAM figure and never mentions GGUF. Its examples just assume 4-way parallelism. The numbers are in the vLLM-ERNIE README and the SGLang cookbook: | GPU memory | FP8 | BF16 | |---|---|---| | 288 GB (B300) | 2 GPUs | 4 GPUs | | 192 GB (B200) | 4 GPUs | 8 GPUs | | 80 GB (H100) | 8 GPUs | 2 nodes × 8 GPUs | The README describes these as the smallest counts that hold the weights, so budget extra for KV cache. The cookbook says its single-node FP8 recipes are verified on every platform it lists, while BF16 is verified only on B300. The 1M context doesn't fit on 80 GB GPUs at all; the README tops out at --max-model-len 262144 there. The plugin only runs on NVIDIA GPUs, too. 14B active won't save you, because all the experts still have to live somewhere. As horsawlarway put it on HN: "Active parameter count really drives inference speed not ram requirements." By my own arithmetic, a 4-bit quant of the 288 GB FP8 repo would be about 144 GB of weights, and 6-bit about 216 GB. The same HN commenter estimates 175 GB at 4-bit and about 244 GB at 6-bit with the full 1M context, and the cookbook's "tens of GB" of activation memory for a 1M prefill makes that plausible. So the 2×128 GB box Tepix mentioned in the thread works on paper at 4-bit, but I found no GGUF or 4-bit build to load on it. The README lists an FP4 build only for the unreleased ERNIE-5.5-Flash. How to run ERNIE 5.5 Flash Base with vLLM Skip the plain pip install vllm snippet Hugging Face shows you. The README says a missing plugin gets you Model architectures ['Ernie5_5_MoeForCausalLM'] are not supported. An unpinned install also gets you vLLM 0.33.0 today, and the plugin pins 0.32.0. Use the card's deployment steps: uv venv --python 3.12 ernie5-venv source ernie5-venv/bin/activate uv pip install vllm==0.32.0 uv pip install git+https://github.com/baidu/vLLM-ERNIE.git vllm serve baidu/ERNIE-5.5-Flash-Base \ --kv-cache-dtype fp8 \ --enable-expert-parallel \ --data-parallel-size 4 \ --max-model-len 131072 \ --port 30000 \ --moe-backend humming Things to change for your setup, all from the README: - The README exports VLLM_ENGINE_READY_TIMEOUT_S=3600 before every serve command and the card doesn't. I'd copy it. - --moe-backend humming is for Blackwell. Drop it on Hopper. - On 80 GB GPUs, use --data-parallel-size 8 and --gpu-memory-utilization 0.93. - For 1M context, set --max-model-len 1048576, lower --gpu-memory-utilization from 0.92 to 0.85 and reduce --max-num-seqs. The README assumes 184 GiB GPUs for those values and gives no number for the last flag. - ECHO drafting is on in the README's commands but not in the card's. Add --speculative-config '{"method":"mtp","num_speculative_tokens":2}' to match. - Those README commands bind 0.0.0.0 with no auth. Add --api-key or bind 127.0.0.1 unless you trust the network. How to run it with SGLang The card's deployment section uses a dedicated image, while Hugging Face's generic snippet uses latest: docker run -d -it --name ernie55 \ --gpus all \ --network host \ --ipc host \ -v ~/.cache/huggingface:/root/.cache/huggingface \ lmsysorg/sglang:dev-ernie-55-flash sglang serve \ --model-path baidu/ERNIE-5.5-Flash-Base \ --tp 4 \ --attn-dp-size 4 \ --context-length 131072 \ --max-running-requests 1024 \ --host 0.0.0.0 \ --port 30000 That's the card's command. The cookbook's recipes use --max-running-requests 256, so size it to your workload. For 1M, the cookbook says to pass --context-length 1048576 and drop --max-running-requests to single digits on 2×B300. This command also binds 0.0.0.0 on the host network, so the same auth warning applies. Calling /v1/completions: few-shot prompts instead of chat For few-shot use, the cookbook says to put the examples in the prompt and set a stop sequence. The README adds that you should pass "temperature": 0 for greedy decoding, since vLLM doesn't apply do_sample. I built this sketch from those two notes and haven't run it: curl http://localhost:30000/v1/completions \ -H "Content-Type: application/json" \ -d '{ "model": "baidu/ERNIE-5.5-Flash-Base", "prompt": "Q: Capital of France?\nA: Paris\n\nQ: Capital of Japan?\nA: Tokyo\n\nQ: Capital of Canada?\nA:", "max_tokens": 16, "temperature": 0, "stop": ["\n\n"] }' The README says the tokenizer prepends <|ERNIE_MASK|> (id 2) to completions prompts by itself, so don't add it to the prompt. In Transformers, load the BF16 repo and keep add_special_tokens=True. The release page calls this the BOS token, but tokenizer.json prepends <|ERNIE_MASK|> there too, while tokenizer_config.json names <|ERNIE_BEGIN|> as bos_token. If you hand-roll inputs, match the tokenizer. ERNIE 5.5 Flash Base vs DeepSeek V4 Flash Base, on Baidu's numbers These are vendor results, with the same harness, prompts and judge across all four models. I didn't find an independent run. Most are few-shot scores, so a bare prompt won't reproduce them. | Benchmark | ERNIE 5.5 Flash | DeepSeek V4 Flash | Hy3 preview | MiMo V2.5 | |---|---|---|---|---| | MMLU-Pro | 78.4 | 69.2 | 66.9 | 65.7 | | GPQA | 60.2 | 55.9 | 50.2 | 51.4 | | HumanEval+ | 82.3 | 69.5 | 78.0 | 74.4 | | LiveCodeBench | 40.1 | 31.9 | 33.5 | 32.4 | | ChineseSimpleQA | 73.0 | 75.3 | 73.2 | 68.8 | | InfiniteBench | 46.5 | 48.0 | 33.0 | 46.7 | ERNIE tops 24 of 30 rows overall. The 12.8-point HumanEval+ gap over DeepSeek is the number I'd want someone to reproduce first. It wins four of the six long-context rows and loses InfiniteBench and HELMET-Rerank, both to DeepSeek, which is a mixed result for a model advertising 1M tokens. input_sh on HN: "They're comparing it to Deepseek-v4-Flash, not 4.1 and not the 4.1 1017 update". I think that complaint only half lands. Tepix points out that V4.1 Flash is twice the size: DeepSeek's card lists 552B backbone parameters plus a 196B Engram memory. That card benchmarks a DeepSeek-V4.1-Flash-Base, but I couldn't find a public repo for it. A 284B base model you can actually download is the fair opponent here. And halJordan's "benchmaxxed to hell" stays an open accusation until someone runs these independently. The ERNIE 5.5 Flash Base license excludes the EU, UK and South Korea Per the ERNIE license: - The territory excludes the EU, the UK and South Korea, and the ban covers the model, derivatives and their outputs. - Non-commercial use is free. Commercial use at or below US$50 million in related annual revenue needs a one-time notice and a "Powered by ERNIE" attribution. Above that, you need written authorization first. - Fine-tuning and quantization both count as creating a Derivative Model. So does distilling from it or training another model on its synthetic output (§1.5). - You can't use the model or its outputs to improve any AI system other than the model and its Derivative Models (§5.5). Those last two clauses bother me most. As I read them, anything you train on ERNIE's outputs falls under Baidu's terms, territory limit included. I'm not a lawyer, but a base model is supposed to be a starting point, and this one follows you downstream. DeepSeek V4 Flash Base has an MIT LICENSE file in a 295 GB repo, about the same download. My call: if you're outside the excluded regions and fine-tuning for internal use, ERNIE is the one to evaluate first, because those code and exam gaps are large enough to survive some vendor optimism. If you're in the EU, redistributing weights, or generating training data for anything else, take DeepSeek. If you just want a local assistant, wait for ERNIE-5.5-Flash. Would you fine-tune under Baidu's license for those HumanEval+ points, or take DeepSeek's MIT checkpoint and give them up? And if you've squeezed this onto 2×128 GB without a GGUF, how did you quantize it?

Baidu’s Unlimited OCR 2 targets long-document parsing. Its own docs say the raw output isn’t Markdown; a separate script converts it. baidu/Unlimited-OCR Would you trade a flashy demo for reliable tables and page boundaries? @grok t.co

Unlimited OCR Works: Pioneering the OCR-3.0 Era, From Long- to Unlimited-Horizon Parsing (30k stars) What it does All text below is quoted directly from the project README. >Unlimited OCR Works: Welcome the Era of One-shot Long-horizon Parsing. What's New in 2: - Sparse Reference Sliding Window Attention (SR-SWA): Compresses text-visual attention interactions from ultra-long sequences down to a fixed-size set of working references (e.g., 512 tokens). This bounds computational complexity, keeping per-step decoding latency and memory footprint nearly constant regardless of document length. - From Long- to Unlimited-Horizon Parsing: Powered by SR-SWA, Unlimited-OCR 2 achieves true, unbounded document parsing within a finite context window, effectively eliminating quadratic attention scaling and memory bottlenecks as documents grow. - Ultra-Long Document Parsing SOTA: Achieves state-of-the-art performance across both dozens-of-pages benchmarks (e.g., MPDocBench) and extreme hundred-page benchmarks (e.g., MarathonDocBench). - Unlimited OCR 2 supports Transformers (native, for quick tests) and SGLang (recommended for serving). >All pages of a document are parsed in one request, so the model can use cross-page context. Repository baidu/Unlimited-OCR Repository stats |Metric|Value| |:-|:-| |Stars|29,629| |Forks|3.2k| |Last release|None| |Languages|Python 100%| |Contributors|3| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|None| |AI co-authored commits|None| Main contributor |Field|Value| |:-|:-| |Name|Youyang Yin| |GitHub|@MurphyYin| |Commits to this repo|16 (top contributor)| |Bio|Research Intern at Baidu Inc.| |Company / Location|Beihang University / Beijing, China| |Followers|241| ────────────────────────────── 📌 How many great GitHub repos have you discovered and completely forgotten about? Don't let another useful tool disappear into your Reddit feed. Get the community's best repos, organized by category, plus key takeaways from real developer discussions, delivered to your inbox twice a week. 📬 Never lose track of great repos again. Subscribe free.

Unlimited OCR Works 2: Pioneering the OCR-3.0 Era: From Long- to Unlimited-Horizon Parsing 百度的光学字符识别 Unlimited OCR 模型升级版 - 保持每一步解码的延迟和内存占用几乎保持恒定，不被文档长度影响 - 无限上下文识别 - 到达同类模型的顶尖水平 GitHub: baidu/Unlimited-OCR Hugging Face: huggingface.co ModelScope: modelscope.cn

Baidu открыла исходники Unlimited-OCR 2 — модель читает тысячи страниц за раз Нейросеть извлекает текст, таблицы и формулы из PDF и сканов. Главное отличие: не перечитывает весь документ заново, а обращается только к нужным фрагментам. Из-за этого... baidu/Unlimited-OCR

🔗 ：baidu/Unlimited-OCR 开源免费；先走 Hugging Face 在线演示试解析，本地部署需要显卡，别一上来就装环境。

AI Repo: ruvnet/RuView | Git News AI Git News AI — 2026-10-28 AI Repo: ruvnet/RuView 🎧 Listen + full transcript: limitless-ai-podcasts.s3.us-east-2.amazonaws.com ## Episode Summary In this episode, we cover: - **AI Repo: ruvnet/RuView** (GitHub Trending) - [Read more](ruvnet/RuView) - **AI Repo: baidu/Unlimited-OCR** (GitHub Trending) - [Read more](baidu/Unlimited-OCR) - **AI Repo: Leonxlnx/taste-skill** (GitHub Trending) - [Read more](Leonxlnx/taste-skill) - **AI Repo: p-e-w/heretic** (GitHub Trending) - [Read more](p-e-w/heretic) - **AI Repo: xiaolai/the-craft-of-selfteaching** (GitHub Trending) - [Read more](xiaolai/the-craft-of-selfteaching) --- *Sponsored by Ada, Ago Consulting, and Zen Semiconductor* Brought to you by Ada, Ago Consulting & Zen Semiconductor. #podcast #OpenSource #AI #DevTools #GitHub

Baidu's open 3B OCR model reads a whole book in one pass Baidu's ERNIE team released Unlimited OCR 2 on Oct 27, 2026: an open-weights (MIT) document OCR model that parses very long documents, up to a full book, in a single request. • 3.34B parameters (BF16), MIT license, weights on Hugging Face; runs with Transformers, SGLang or vLLM (needs trust_remote_code) (model card). • New attention, SR-SWA: the model keeps a fixed set of working references (e.g. 512 tokens) instead of attending to every earlier page, so per-step cost stays constant (Baidu tech report). • Baidu-reported: 7.0x per-step speedup over full attention and 5.6x over R-SWA (v1's approach) at hundred-page scale. • Baidu-reported scores: MPDocBench 84.6 (+6.3), MarathonDocBench 77.06 (+43.3; Baidu's own new 100–300-page benchmark), OmniDocBench v1.6 94.02. • Limitations (Baidu): weaker on global structure such as heading hierarchy; limited training-data diversity. My take: 3B is small enough for a single own GPU (about 6.7 GB of BF16 weights, our math; Baidu tested on an A800-80GB), which makes local OCR for invoices and contracts realistic. But the long-document benchmark is Baidu's own; test it on your own PDFs. Sources (as of Oct 27, 2026): - Hugging Face model card: huggingface.co - GitHub: baidu/Unlimited-OCR - Tech report (Baidu ERNIE Team): github.com/baidu/Unlimited-OCR/blob/Unlimited-OCR… - Baidu on X (Oct 27, 2026): x.com No voiceover: text, graphics and music are original. Music synthesized in code for The AI Radar (no samples). Not registered in Content ID. #AInews #OCR #Baidu #OpenSource #LLM

⭐ Unlimited OCR 2 ⭐ baidu/Unlimited-OCR 1. 중국 검색·AI 회사 바이두가 아주 긴 문서를 통째로 글자로 읽어주는 OCR 모델 2편을 오픈소스로 공개함 2. 문서가 길어질수록 계산량이 확 늘어나는 게 기존 문제인데, 새 방식(SR-SWA)은 참고하는 부분을 고정한 개수(예: 512토큰)로 줄여서 길이가 늘어도 속도와 메모리가 거의 일정함 3. 여러 쪽 이미지를 한 번에 넣으면 쪽 사이 맥락까지 보고 순서대로 읽어줌. 한 쪽짜리도 가능하고, 가볍게 써볼 땐 Transformers, 서버로 돌릴 땐 SGLang 추천 4. 수십 쪽짜리(MPDocBench)부터 100쪽 넘는 문서(MarathonDocBench)까지 최고 성능이라고 밝힘. 긴 문서용 새 평가 자료 MarathonDocBench도 같이 공개함 5. MIT 무료, 스타 2.9만 개. 모델 크기는 약 33억 개 파라미터이고 README 테스트는 NVIDIA A800 80GB, Python 3.10 기준 스캔한 책이나 백 쪽 넘는 보고서를 한 번에 글자로 바꾸고 싶은 사람한테 좋을 듯 ━━━━━━━━━━ 🇺🇸 1. Baidu open-sourced Unlimited OCR 2, an OCR model that reads very long documents in one go 2. Longer documents normally mean much more computation. The new method (SR-SWA) keeps the attention to a fixed-size set of references (e.g. 512 tokens), so speed and memory stay nearly constant as the document grows 3. Feed in multiple page images at once and it reads them in order using cross-page context. Single pages work too. Transformers for quick tests, SGLang recommended for serving 4. Baidu reports state-of-the-art results from dozens of pages (MPDocBench) to 100+ pages (MarathonDocBench). It also released MarathonDocBench, a new benchmark for ultra-long documents 5. Free under MIT, 29k stars. About 3.3B parameters. README tests were run on an NVIDIA A800 80GB with Python 3.10 Looks great for anyone who wants to turn scanned books or 100+ page reports into text in one shot #AI #OCR #OpenSource #Baidu #DocumentAI

Otevřené OCR od Baidu přečte celou knihu najednou Tým ERNIE z Baidu vydal 27. 10. 2026 Unlimited OCR 2: OCR model pro dokumenty s otevřenými váhami (licence MIT), který zpracuje velmi dlouhé dokumenty, až celou knihu, v jednom požadavku. • 3,34 mld. parametrů (BF16), licence MIT, váhy na Hugging Face; běží v Transformers, SGLang nebo vLLM (vyžaduje trust_remote_code) (model card). • Nová pozornost SR-SWA: model si drží pevnou sadu pracovních referencí (např. 512 tokenů) místo pozornosti ke všem předchozím stránkám, takže cena kroku zůstává stejná (technická zpráva Baidu). • Podle Baidu: 7,0× rychlejší krok než plná pozornost a 5,6× než R-SWA (přístup verze 1) u dokumentů o stovkách stran. • Skóre podle Baidu: MPDocBench 84,6 (+6,3), MarathonDocBench 77,06 (+43,3; nový vlastní benchmark Baidu, 100–300 stran), OmniDocBench v1.6 94,02. • Omezení (Baidu): slabší u celkové struktury, třeba hierarchie nadpisů; omezená rozmanitost trénovacích dat. Můj názor: 3B je dost malé pro vlastní GPU (asi 6,7 GB vah v BF16, náš výpočet; Baidu testovalo na A800 80 GB), takže lokální OCR faktur a smluv dává smysl. Benchmark dlouhých dokumentů je ale od Baidu, zkuste ho na vlastních PDF. Zdroje (stav k 27. 10. 2026): - Model card na Hugging Face: huggingface.co - GitHub: baidu/Unlimited-OCR - Technická zpráva (Baidu ERNIE Team): github.com/baidu/Unlimited-OCR/blob/Unlimited-OCR… - Baidu na X (27. 10. 2026): x.com Bez hlasu: text, grafika i hudba jsou původní. Hudba syntetizovaná v kódu pro The AI Radar. #umelainteligence #AInovinky #Baidu #OCR #opensource