MiniCPM-o

MiniCPM-V: See, hear, and interact on your local devices (27k stars) What it does All text below is quoted directly from the project README. >MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in 35B-A3B and 1B variants. The 35B-A3B variant is the series' first MoE model. Model highlights: - A new performance–efficiency Pareto frontier. The 35B-A3B and 1B variants each advance the performance–efficiency Pareto frontier for multimodal large language models. In Instruct mode, 35B-A3B achieves a higher overall score in less inference time than Qwen3.6 35B-A3B; the 1B variant further improves on MiniCPM-V 4.6's core capabilities and substantially outperforms Qwen3.5 0.8B in both overall score and inference efficiency. - The first open-source MoE to run smoothly on-device without compromising performance. Powered by the latest Lookahead Expert Scheduling (LES) inference acceleration technique, MiniCPM-V 4.7 35B-A3B becomes the first open-source MoE model to run smoothly on-device without compromising performance, achieving a decoding speed of 13+ tokens/s with just 3 GB of memory on iPhone 17 Pro Max. - Broad platform support and easy customization. MiniCPM-V 4.7 supports iOS, Android, and HarmonyOS, and supports chip platforms from Intel, AMD, Arm, Rockchip, and Houmo, covering a diverse range of mobile devices and local computing scenarios. We also open-source Self-Replay Fine-Tuning (SRFT), helping developers extend model capabilities at low cost while reducing forgetting of existing capabilities. - Vision capability approaching Gemini 2.5 Flash. It scores 77.6 on average across 8 popular OpenCompass benchmarks. With only 9B parameters, its visual understanding surpasses GPT-4o and Gemini 2.0 Pro, and a single model supports both instruct and thinking modes. - Strong speech capability. It supports bilingual real-time speech conversation with configurable voices in English and Chinese, making conversations more natural, expressive, and stable. A simple reference audio clip is enough for voice cloning and role play, and its voice cloning performance surpasses strong TTS tools such as CosyVoice2. - Full-duplex, proactive live interaction. It process real-time, continuous video and audio input streams simultaneously while generating concurrent text and speech output streams in an end-to-end fashion, without mutual blocking. This allows MiniCPM-o 4.5 to see, hear, and speak simultaneously, enabling smooth, real-time omnimodal conversations. >MiniCPM-o 4.5 is the latest model in the MiniCPM-o series, a 9B end-to-end omnimodal model. It can continuously receive video and audio while generating speech and text, enabling real-time interactions that see, hear, and speak simultaneously. Repository OpenBMB/MiniCPM-V Repository stats |Metric|Value| |:-|:-| |Stars|26,888| |Forks|2,111| |Last release|20250527 (2025-05-27)| |Languages|Python 83.2%, Vue 14.1%, JavaScript 1.7%| |Contributors|37| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|cursoragent: 3| |AI co-authored commits|Cursor: 3| Main contributor |Field|Value| |:-|:-| |Name|Tianyu Yu| |GitHub|@yiranyyu| |Commits to this repo|231 (top contributor)| |Company / Location|Tsinghua University / Beijing| |Followers|174| |Public repos|20| |On GitHub since|2017-04-27| ────────────────────────────── 📌 How many great GitHub repos have you discovered and completely forgotten about? Don't let another useful tool disappear into your Reddit feed. Get the community's best repos, organized by category, plus key takeaways from real developer discussions, delivered to your inbox twice a week. 📬 Never lose track of great repos again. Subscribe free.

MiniCPM-V 4.7 35B-A3B，直接跑在 iPhone 上。 35B 的 MoE 视觉模型，激活参数约 3B，完全本地运行，只占 3 GB 内存。一张旧窗户的照片，一个问题，它真把照片读懂了🧵 GitHub: OpenBMB/MiniCPM-V HF: huggingface.co t.co

MiniCPM-V 4.7 (OpenBMB/MiniCPM-V) brings a 35B-A3B multimodal MoE model to an iPhone at 13+ tokens/s. Its key trick is Lookahead Expert Scheduling: predict which experts the next layer will need and preload their weights while the current layer computes. #MiniCPMV #MultimodalAI

🚀 MiniCPM-V 4.7 hits #3 on @huggingface Trending! Huge thanks to the community for the support! 💗 📱 35B-A3B MoE runs smoothly on iOS and Android with ~3GB memory. The 1B model delivers strong OCR, math, and video understanding performance. 🔗 GitHub: OpenBMB/MiniCPM-V 🤗 Hugging Face: huggingface.co

OpenBMB выпустили MiniCPM-V 4.7 — мультимодальные модели для смартфонов и планшетов Две версии: 35B-A3B и компактная 1B. Работают локально на iPad и iPhone, понимают изображения и длинные видео,... OpenBMB/MiniCPM-V huggingface.co

MiniCPM-V 4.7 35B-A3B, running on an iPhone. A 35B MoE vision model with ~3B active, fully on-device with just 3 GB of memory. One photo of a weathered window, one question — and it actually read the photo 🧵 GitHub: OpenBMB/MiniCPM-V HF: huggingface.co t.co

How much multimodal work fits in 1B parameters? I tested MiniCPM-V 4.7 1B on 29 fixed tests (OCR, charts, math diagrams, multi-image, video) against its 35B-A3B sibling. Same API, temp 0, 3 runs each. 1B: 18.7/29 35B-A3B: 28/29 Try it yourself: GitHub: OpenBMB/MiniCPM-V Hugging Face: huggingface.co Lookahead Expert Scheduling: llama.cpp-omni/docs/les/lookahead-expert-scheduling.md at master · tc-mb/llama.cpp-omni

A 35B multimodal model running on a phone with ~3GB memory? MiniCPM-V 4.7 demo is pretty wild Just saw this demo from OpenBMB for MiniCPM-V 4.7, and the on-device capability is pretty impressive. A 35B-A3B MoE vision-language model running locally on mobile devices is already interesting, but the use cases are what caught my attention: - 📷 Reading real-world signs - 📄 Understanding complex documents - 🧠 Reasoning over visual information - 🗺️ Planning with real-world constraints The key piece seems to be LES (Lookahead Expert Scheduling) — instead of just making the model smaller, they optimize MoE inference by predicting and preloading experts to reduce bottlenecks. They also released a smaller MiniCPM-V 4.7 1B model for more resource-constrained devices. Another interesting addition is Self-Replay Fine-Tuning (SRFT), which aims to let developers customize the model for specific tasks while keeping general capabilities. The direction is interesting: instead of sending every image/question to the cloud, we're getting closer to having capable multimodal assistants running directly on phones. Links: GitHub: OpenBMB/MiniCPM-V Hugging Face: huggingface.co SRFT details: openbmb.cn

Bloomery 0.3.2 릴리즈에 모델 지원 추가해봤습니다. 빠릿빠릿한게 일도 좀 시켜봐야겠네요. 지원모델 확장중입니다. 관심있으신분 와서 작업해주세요! github.com/midagedev/bloomery/releases t.co

🔗 GitHub: OpenBMB/MiniCPM-V 🤗 Hugging Face: huggingface.co Lookahead Expert Scheduling: Making On-Device MoE Inference Faster: - Expert prefetch and compute scheduling - Optimized weight access github.com/tc-mb/llama.cpp-omni/blob/master/docs/…