OmniParser: Screen Parsing tool for Pure Vision Based GUI Agent (26k stars)
What it does
All text below is quoted directly from the project README.
>A simple screen parsing tool towards pure vision based GUI agent
OmniParser is a comprehensive method for parsing user interface screenshots into structured and easy-to-understand elements, which significantly enhances the ability of GPT-4V to generate actions that can be accurately grounded in the corresponding regions of the interface.
News:
- [2026/7] We add a YOLOv9-E interactive region detector.
- [2025/3] We are gradually adding multi agents orchstration and improving user interface in OmniTool for better experience.
- [2025/2] We release OmniParser V2 checkpoints.
- [2025/2] We introduce OmniTool: Control a Windows 11 VM with OmniParser + your vision model of choice.
- [2024/11] We release an updated version, OmniParser V1.5 which features 1) more fine grained/small icon detection, 2) prediction of whether each screen element is interactable or not.
- [2024/09] OmniParser achieves the best performance on Windows Agent Arena!
>To run gradio demo, simply run: python gradio_demo.py
Repository
microsoft/OmniParser
Repository stats
|Metric|Value|
|:-|:-|
|Stars|25,581|
|Forks|2.2k|
|Last release|v.2.0.1 (2025-09-12)|
|Languages|Jupyter Notebook 49.6%, Python 37.5%, Shell 8.1%|
|Contributors|11|
|AI contributor accounts (Claude / Codex / Cursor / Copilot)|@Copilot (1 commits)|
|AI co-authored commits|Copilot: 1|
Main contributor
|Field|Value|
|:-|:-|
|GitHub|@yadong-lu|
|Commits to this repo|47 (top contributor)|
|Followers|66|
──────────────────────────────
📌 How many great GitHub repos have you discovered and completely forgotten about?
Don't let another useful tool disappear into your Reddit feed. Get the community's best repos, organized by category, plus key takeaways from real developer discussions, delivered to your inbox twice a week.
📬 Never lose track of great repos again. Subscribe free.
2 points
Mentions
OmniParser: Screen Parsing tool for Pure Vision Based GUI Agent (26k stars) What it does All text below is quoted directly from the project README. >A simple screen parsing tool towards pure vision based GUI agent OmniParser is a comprehensive method for parsing user interface screenshots into structured and easy-to-understand elements, which significantly enhances the ability of GPT-4V to generate actions that can be accurately grounded in the corresponding regions of the interface. News: - [2026/7] We add a YOLOv9-E interactive region detector. - [2025/3] We are gradually adding multi agents orchstration and improving user interface in OmniTool for better experience. - [2025/2] We release OmniParser V2 checkpoints. - [2025/2] We introduce OmniTool: Control a Windows 11 VM with OmniParser + your vision model of choice. - [2024/11] We release an updated version, OmniParser V1.5 which features 1) more fine grained/small icon detection, 2) prediction of whether each screen element is interactable or not. - [2024/09] OmniParser achieves the best performance on Windows Agent Arena! >To run gradio demo, simply run: python gradio_demo.py Repository microsoft/OmniParser Repository stats |Metric|Value| |:-|:-| |Stars|25,581| |Forks|2.2k| |Last release|v.2.0.1 (2025-09-12)| |Languages|Jupyter Notebook 49.6%, Python 37.5%, Shell 8.1%| |Contributors|11| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|@Copilot (1 commits)| |AI co-authored commits|Copilot: 1| Main contributor |Field|Value| |:-|:-| |GitHub|@yadong-lu| |Commits to this repo|47 (top contributor)| |Followers|66| ────────────────────────────── 📌 How many great GitHub repos have you discovered and completely forgotten about? Don't let another useful tool disappear into your Reddit feed. Get the community's best repos, organized by category, plus key takeaways from real developer discussions, delivered to your inbox twice a week. 📬 Never lose track of great repos again. Subscribe free.
OmniParser：让 GUI agent"摸得到"屏幕。 𝐘𝐎𝐋𝐎𝐯𝟗-𝐄 检测可交互区域并给包围盒，𝐅𝐥𝐨𝐫𝐞𝐧𝐜𝐞 给每个图标写描述、判可操作性。截图进去，带坐标和语义的元素出来。 V2 在 ScreenSpot Pro 拿下 39.5% SOTA，配 OmniTool 可操控 Win11 虚拟机。 microsoft/OmniParser t.co