SoMarkAI

Errors as Data: How RSI Helps OXR Achieve SOTA Accuracy and Speed in Document Parsing --- title: "Errors as Data: How RSI Helps OXR Achieve SOTA Accuracy and Speed in Document Parsing" description: "What if every document a model gets wrong told you what to train on next? OXR's RSI loop turns those failures into targeted training data." tags: opensource, ai, machinelearning, ocr canonical_url: somark.ai series: Inside OXR --- > Summary > > What if every document a model gets wrong told you what to train on next? That's the loop behind OXR, a 1.3B model that ranks #1 on three document parsing benchmarks. Start with what usually happens when a model misreads a document. Someone corrects that one result, ships it, and moves on. Next week a similar document arrives, and the model probably makes the same mistake. A better next version means picking out bad cases, working out what went wrong, preparing data, and checking that the data is usable, while new failures keep showing up every day. We handed that pipeline to the system. OXR-1.0 uses recursive self-improvement (RSI) in post-training. Each round, the system finds the samples the model misreads or can't settle on in unlabeled documents, analyzes what went wrong, plans the next round of training, and generates targeted training data. To our knowledge, this is the first time RSI has been used for document parsing. RSI is a training-time design. OXR does not learn from your documents while you use it. Most of the judging, attribution, planning, and data generation is done by a stronger general-purpose multimodal model (the General VLM in our diagrams). Disagreement across OXR's own repeated predictions also helps flag uncertain samples, and a separate model handles clustering. When we say "no manual work," we mean that the training data RSI generates needs no human annotation or correction. It does not mean the whole training process runs without people. OXR-1.0 ranks #1 on OmniDocBench (97.18), olmOCR-Bench (87.84), and DocParsingBench (87.76). In speed tests on an RTX PRO 6000, it also ranks #1 among open-source document parsing models. Why document parsing suits learning from errors Document parsing has a property few other tasks share: every input has a correct answer. Characters, table rows, columns and merged cells, and formula symbols can all be checked item by item. Quality can be measured objectively, and most mistakes fall into a specific kind of problem. That property has mostly been used for evaluation, not for learning. The catch is that the images and PDFs arriving from real use come with no human answers, so they can't serve as training data directly. OXR first has to find its weak spots in those documents, then build labeled new data around them. The three stages below do exactly that. The RSI loop in three stages !OXR's RSI loop: three stages in sequence, with training results flowing back into OXR. The stages are Prediction & Classification, Attribution & Training Planning, and Data Generation. The Train line at the bottom, leading back to OXR, is the "recursive" part: data that passes validation in one round becomes training input for the next. Stage 1: Find the misreads and the shaky samples OXR first parses unlabeled documents. Then the General VLM looks at the original image, the prediction, and the judging criteria, and sorts each sample into one of three classes: - Correct: judged right. It stops here for this round. - Wrong: judged wrong. It goes on to analysis. - Uncertain: can't be judged reliably, or the prediction is unstable. It also goes on to analysis. Keeping Uncertain is deliberate. These documents have no human ground truth, and judging rules can't cover every case in advance. With only Correct and Wrong, the judge's hesitation gets hidden in a label that looks certain and travels all the way into the training data. In our experiments, adding the third class made the split between Correct and Wrong cleaner. That is a qualitative observation; we haven't measured it systematically. An uncertain sample is one whose prediction is unstable and that sits near the edge of the training data distribution. There is a second way to find them: we branch off multiple rollouts from OXR and compare the predictions for the same sample. One correct answer doesn't mean the model has the sample down. If repeated runs differ clearly, the sample deserves a look. The General VLM checks whether a result is wrong, and the rollouts show whether OXR's answer is stable, so the two signals complement each other. !Prediction & Classification: parallel identification paths, three-way classification, and perturbation analysis of high-disagreement samples. For samples where the rollouts disagree, we ask one more question: why is the prediction unstable? We apply small perturbations to the image, such as rescaling, blurring, compression, or a slight shift, without changing the content, and run the rollouts again. If the disagreement grows, the model is sensitive to visual changes that have nothing to do with content. The perturbation itself then becomes an important reason, and it carries into Stage 2. Stage 2: Turn scattered errors into "what to add next" A Wrong verdict says the result is incorrect. It doesn't say where, and the original document still has no reliable label. So the General VLM analyzes each sample's original image and prediction and writes down possible reasons. One sample can have several. For uncertain samples, it analyzes what makes the prediction unstable, without assuming the sample is wrong. !Attribution & Training Planning: possible reasons from multiple samples are encoded and clustered into data coverage directions. These natural-language reasons are diagnostic hypotheses, but the ones that keep recurring deserve attention. We encode the reason text with SigLIP 2 and cluster it, grouping similar reasons into reason clusters. Here is an example built only to show the mechanism; the documents and strings in the figure are illustrative, not OXR measurements. A research report is judged Wrong, and one reason is that the capital M in "Model" was transcribed as lowercase. A technical manual is judged Uncertain because its predictions disagree on the capitalization of "Model." The two documents come from different domains, but both reasons land in the same cluster: case confusion. The next stage generates brand-new documents around that cluster to cover this kind of capitalization difficulty. That is Training Planning: turning scattered diagnoses into the problems the next round of training should cover. Stage 3: Generate data that targets the weakness and still looks like a real document Reason clusters give the generator a clear target, but counting samples hides two problems: 1. Self-repetition. A thousand generated samples may share one set of layout habits and phrasing, like one person talking to themselves. The count goes up while the distribution stays narrow. 2. Overemphasized difficulty. Without real-world style references, the generator may draw a structure too neat and too prominent just to make the point. A model can learn those exercises and still fail to recognize the same structure in natural documents. So Stage 3 takes two kinds of input. The reason clusters say which difficulties to cover, and real samples supply layout and visual style. Each generation run draws a different sample from the real pool, so a cluster is never tied to one template. In the capitalization example, the difficulty shows up in different real layouts: "Model selection" in a research article, and "MODEL" and "model_id = 07" in a technical spec. A reason that came from perturbation goes in more directly. If the model is sensitive to slight rescaling or compression, describing the problem isn't enough, so the generated documents have to carry those changes again and again. !Data Generation: one structural difficulty is combined with different real-world styles to generate documents, which are validated before training. The General VLM writes code that draws, typesets, and renders documents, and a program runs it to produce the images and labels. The difficulty has to be a feature visible in the image, not just a line in the prompt. Then comes validation. Targeted generation tends to overdo the difficulty: cells made too small, formulas written with nonstandard symbols, the wrong delimiters where text and formulas mix. Output like that can render fine and still break the standard. Only data that passes rule-based validation and filtering goes into the next round of training. How we know the next round improved Data that passes filtering trains the next round of OXR. The updated model then faces real documents again, and its new predictions and classifications become the starting point of the next round. That is why RSI sits in post-training and runs repeatedly: one round's results change the next round's inputs. We track the repair rate. We re-run the previous round's Wrong samples through the updated OXR, have the General VLM judge them again, and divide the number that turn Correct by the number originally labeled Wrong. This is the share of the same samples the VLM now judges correct. It is not accuracy on an independent test set, and nobody checked each case by hand. Overall model accuracy is what the three public benchmarks above report. In theory RSI can iterate without limit, but iterating doesn't guarantee progress. Four things to watch when it keeps running: - Find problems with real documents. The input for finding and attributing weaknesses has to be real documents, not synthetic ones, so the weaknesses it finds don't lean toward cases the generator created itself. - Use enough samples per round. With too few documents, you only see scattered bad cases. With enough, reason clusters have more support and Training Planning has a firmer basis. - Weigh quantity and diversity together. Cover different sources, time periods, document types, and layouts. A large set of near-identical documents keeps exposing the same problem. - Build solid base capabilities first. RSI-generated samples are a small share of the overall training data, so they can't carry all the optimization. RSI strengthens specific weaknesses on top of a solid foundation. What you can take from this Looking back, the design relies on document parsing having correct answers. Because right and wrong can be judged automatically, the verdicts can be turned into directions for building data. Because exercises can be generated by a program, training data doesn't depend on human annotation. Because generation quality has executable standards, automated attempts can run at scale. Together with the training that follows, the three stages make up one RSI round. The model's size follows from this too. With the data distribution better covered, the 1.3B OXR doesn't have to lean on parameter count for accuracy, and its small size plus a task-level architecture design gives it speed. We think it can get smaller still. The approach may not stop at document parsing. Any task with a stable, automatically checkable success criterion could automate the same chain: find weaknesses, build supervision, validate data. Document parsing is the case we've gotten working. Try OXR on your own documents, and tell us where it still reads them wrong. --> About OXR and SoMark OXR is a 1.3B open-source document parsing model from the team behind SoMark. It ranks #1 on olmOCR-Bench, OmniDocBench, and DocParsingBench and delivers SOTA parsing speed. The model uses a recursive self-improvement (RSI) loop during post-training. SoMark is a production-grade document parsing engine for agents, powered by an independent model. It supports more file formats, finer-grained recognition of document elements, and faster processing. It also offers extensive post-processing features, custom development, and model training services for different business needs. **If OXR saves you time, give it a ⭐ on GitHub: SoMarkAI/OXR**

我们开源了 OXR： 1.3B 文档解析模型，全球三榜第一，提供免费 API 首先先自我介绍一下，我是 SoMark 团队的 CTO 。 我们最近开源了一个新的文档智能解析模型—— OXR （ Optical Everything Recognition ）。 它可以把 PDF 和文档图片转成 Markdown / JSON ，支持表格、公式、化学式、代码块等 13 类元素，并且解析结果中包含版面坐标信息。 OXR 的主模型只有 1.3B ，在全球三项文档解析基准测试中其总分都排到了第一：OmniDocBench 97.18 、olmOCR-Bench 87.84 、DocParsingBench 87.76 。 速度也做了同条件实测：单张 RTX PRO 6000 Blackwell （ 96GB ），解析 100 页从 OmniDocBench 里随机抽取的文档，OXR 平均用时约 101 秒，在参测的 21 个开源模型中最快，其中包括 Paddle 和 MinerU 。 OXR 创新采用 RSI 训练范式，结合巧妙的架构设计，达到了精度与速度双 SOTA 。同时 OXR 的版面分析模块独立、可插拔。大家可以直接用我们的内置模型，也可以按照业务需求进行替换。 OXR 项目地址：[SoMarkAI/OXR]( SoMarkAI/OXR) 下面向大家介绍一下 OXR 的具体使用方法。 一、不部署，先跑一份文档 想先看看效果，可以直接打开 [在线 Demo]( free-oxr.somark.cn)，上传 PDF 或文档图片，对照原文查看解析出的 Markdown 和版面结果。 为了方便大家调用，我们也开放了免费 API ，不用注册，也不用申请 Key 。装好 CLI ，就能把自己的 PDF 转成 Markdown 和 JSON： pip install somark somark --base-url https://free-oxr.somark.cn/api/v1 --api-key dummy \ parse document.pdf --formats md,json --out ./parsed-output/ 把 document.pdf 换成你的文件路径，结果会保存到 ./parsed-output/。其中 dummy 只是工具参数的占位值，不是真实 Key 。 我们将持续提供免费 API 服务，直至我们的服务器顶不住压力。 ⚠️ 注意：不建议将免费 API 用于线上环境，我们没有稳定性保证，也设置了排队机制，速率略有浮动。 二、本地部署 OXR 本地部署的配置要求如下，准备好后下载我们提供的代码和模型，即可通过 Docker 或源码启动服务。 |项目|环境要求| |---|---| |操作系统|Linux| |GPU|支持 BF16 的 NVIDIA GPU| |驱动|兼容 CUDA 13.0 的 NVIDIA 驱动| |容器引擎|Docker （仅 Docker 启动需要）| |GPU 容器支持|NVIDIA Container Toolkit （仅 Docker 启动需要）| |Python|Python 3.12 （用于下载模型和源码启动）| |显存（起步建议）|单卡 24GB ，用于低并发试用| |显存（推荐配置）|单卡 48GB 及以上，为长文档和较高并发预留空间| 1. 下载代码和模型。权重可从 [Hugging Face]( huggingface.co) 或 [ModelScope]( modelscope.cn) 获取，这里以 Hugging Face 为例： git clone https://github.com/SoMarkAI/OXR.git cd OXR pip install -U huggingface_hub hf download SoMarkAI/OXR-1.0 --local-dir ./models/OXR-1.0 2. 启动服务。推荐使用 docker 配置服务，一次性启动所有服务。首次启动可能较慢，默认启动等待上限为 900 秒。 docker build -f docker/Dockerfile.allinone -t oxr-1.0 . docker run --rm --gpus all --ipc=host \ -p 127.0.0.1:8000:8000 \ -v "$PWD/models/OXR-1.0:/models/oxr-model:ro" \ oxr-1.0 3. 打开本地 Demo ，或通过 API 调用。 等待启动日志显示服务就绪后，打开 http://localhost:8000/demo 上传文档即可查看解析结果；接口文档在 http://localhost:8000/docs。 也可以将前面 CLI 命令的 --base-url 改成 http://localhost:8000/v1，调用本地服务。 源码启动方式、并发和显存参数等详细说明见 [README]( github.com/SoMarkAI/OXR/blob/main/README_CN.md)。 三、配套的 SoMark 生态工具 此前我们团队围绕核心产品 「 SoMark 文档智能」开发了一系列生态工具，OXR 作为我们的独立开源模型，同样适配： ||一句话简介|项目地址| |---|---|---| |SoMarkDown|SoMark 、OXR 官方的 Markdown 解析与渲染库|[SoMarkAI/SoMarkDown]( SoMarkAI/SoMarkDown)| |SoMarkDownViewer|SoMarkDown 的实时编辑应用|[SoMarkAI/SoMarkDownViewer]( SoMarkAI/SoMarkDownViewer)| |SoPDF|高性能、宽松开源许可的 PDF 处理库|[SoMarkAI/SoPDF]( SoMarkAI/SoPDF)| |Docision|面向文档场景的多模态 Jev|[SoMarkAI/docision]( SoMarkAI/docision)| |somark-py|服务 SoMark 、OXR 的官方 Python SDK \& CLI|[SoMarkAI/somark-py]( SoMarkAI/somark-py)| |somark-js|服务 SoMark 、OXR 的官方 JS / TS SDK \& CLI|[SoMarkAI/somark-js]( SoMarkAI/somark-js)| 欢迎拿你们手头难处理的文档来试试。哪里没解析对，可以 [点击这里]( free-oxr.somark.cn) 向我们反馈。 如果对大家有帮助，一定要给我们的项目点个 🌟 Star 支持一下～😉 相关链接 - GitHub：[SoMarkAI/OXR]( SoMarkAI/OXR) - Hugging Face：[huggingface.co]( huggingface.co) - ModelScope：[modelscope.cn]( modelscope.cn) - 在线 Demo：[free-oxr.somark.cn]( free-oxr.somark.cn) - 免费 API：[free-oxr.somark.cn]( free-oxr.somark.cn) - OXR 官网：[somark.cn]( somark.cn)

OXR delivers top accuracy on 3 major benchmarks while running 6.8x faster than Surya-OCR-2 on the same GPU. 683.1s → 101.0s. Less time, better document parsing. SoMarkAI/OXR

Signal: OXR. 1.3B, open source. olmOCR-Bench 87.84, rank 1. OmniDocBench 97.18, rank 1. DocParsingBench 87.76, rank 1. Speed rank 1, RTX PRO 6000 x1. vs MinerU2.5-Pro: 2.52x. Edge: tables. Layout: swappable. Arch: recognition ∥ order. Training: RSI. SoMarkAI/OXR

The Long-Overlooked Performance Cost in Document Parsing: Reading Order --- description: "Every document parser has to decide reading order, and the usual designs pay for it in either speed or labeling. OXR moves it into recognition and makes layout analysis a pluggable module." canonical_url: somark.ai series: Inside OXR --- > Summary > > Every document parser has to decide reading order, and the usual designs pay for it in either speed or labeling. OXR moves it into recognition and makes layout analysis a pluggable module. Where should reading order live in a document parsing pipeline? Most of the attention in document parsing goes to recognition. But where you put reading order decides how fast the pipeline runs and how much data you have to label to adapt it to your business. That question shaped OXR, a 1.3B open-source document parsing model from the team behind SoMark. It turns PDFs and document images into Agent-Friendly data, in Markdown or JSON. OXR runs reading order together with recognition, in the same model and the same stage, and keeps layout analysis as a separate, lighter model that you can swap out. Together with its recursive self-improvement (RSI) training, this task-level design is part of why OXR ranks #1 on olmOCR-Bench, OmniDocBench, and DocParsingBench, and ranks #1 among open-source document parsing models in speed tests on an RTX PRO 6000. The rest of this post explains why we designed it this way and how it compares with the other options. Every parser runs the same three steps Over the past two years, document intelligence has filled with new players, and most of them started with parsing. The industry has settled on three steps: 1. Layout analysis: detect the position and type of each element on the page. 2. Element recognition: process each element according to its type. 3. Reading order: arrange the elements in natural reading order. !A typical document parsing pipeline: layout analysis detects regions, each region is recognized by type, then reading order assembles the results into Markdown. Every one of these tasks has the same shape. Text is image to text, a formula is image to LaTeX, a table is image to an HTML table, and layout analysis is image to a string of coordinates. They all turn an image into a language sequence, so in principle one vision-language model (VLM) can do all of them. What differs between systems is how the work is split across models and stages, and that split is where the reading order cost comes from. Four designs, four different bills !Model and stage structure of DotsOCR and DeepSeekOCR, Dolphin and MinerU 2.5, MonkeyOCR, PaddleOCR-VL, and OXR. Timings are simulated and relative, not measured runtimes. 1 model, 1 stage DotsOCR, DeepSeekOCR, and MinerU-VLM do layout analysis, reading order, and recognition in one pass. It is simple and slow, because the whole page comes out one token at a time. 1 model, 2 stages Dolphin and others run layout analysis and reading order first, then recognize all elements in parallel. That speeds up a page several times over, and MinerU went back to this approach in versions 2.0 and 2.5. A VLM still emits layout boxes one at a time, though, orders of magnitude slower than a conventional detector that outputs hundreds at once. The field accepted this because the order in which the VLM emits boxes is the reading order. Moving layout analysis out of the VLM would mean solving reading order with another model. 3 models, 3 stages MonkeyOCR pulled everything apart: layout analysis, then recognition, then reading order. Layout analysis became very fast, but reading order turned into its own sequential stage with a heavy model. LayoutReader is the only open-source reading order model, and it is large and slow. 2 models, 2 stages PaddleOCR-VL merged layout analysis and reading order into one small specialized model. That let layout analysis leave the VLM and return to detector speed, and it became the standard answer almost overnight. Many models released in 2026, including GLM-OCR and Falcon-OCR, use its layout-and-order model directly and build only their own recognition model. So is that the best we can do? The merged design has a price. Every new layout annotation needs a matching reading order annotation, which doubles the labeling work. Training it also means leaving the familiar PyTorch ecosystem: PaddleOCR is well engineered, but you lose the community's toolchain. Layout rules differ by business, so layout analysis should stand alone How a document is written depends on the organization, the language, and the author, so there is no universal definition of layout. The same page can be divided in different valid ways: - Element type: is a text box a title plus text, or one table? - Region boundary: is a dense group of pictures one region, or one region per picture? - Content grouping: is a picture full of text one figure, or a figure plus text elements? !Element type: the same project delivery page annotated under Standard A and Standard B !Region boundary: the same leaf morphology page annotated under Standard A and Standard Bn !Content grouping: the same rainwater recycling page annotated under Standard A and Standard B Each answer serves a different need. For high-quality parsing, the need to customize layout analysis never goes away, so it helps to make that part independent. OXR's answer: reading order goes with recognition Layout Analysis → Recognition + Reading Order Reading order benefits from understanding the content, so it belongs with recognition. Layout definitions depend on the business, so layout analysis stands on its own. OXR is also two models in two stages, with the split in a different place. A lightweight layout model returns only region types and bounding boxes. OXR then recognizes each region and, in the same stage, predicts the reading order from the full-page image plus each region's ID, type, and boundaries. !OXR architecture: a pluggable Layout Analysis module feeds the OXR model, which handles Recognition and Reading Order. Faster and more accurate Parallel layout detection plus parallel element recognition already put two-model, two-stage designs far ahead of the others on speed. Reading order takes far less time than recognition, so running them together removes the time that reading order used to add to the pipeline. Layout analysis no longer carries reading order either, so a smaller layout model can reach the same accuracy. Reading order also becomes content-aware. It is predicted from the full page, with content and layout context together, so OXR can work out how columns, headings, captions, and body text relate, which a purely visual ordering model can miss. Pluggable layout analysis Layout analysis is now a fully independent model and task, and OXR doesn't restrict its form or categories. You can pick any approach from the object detection community, which has matured over decades, and plug it into the pipeline. In practice: - Start with the built-in layout model. It's in the layout_analysis folder of the OXR weights on Hugging Face. - Bring your own detector if your business has its own layout rules. Adapt the interface, then re-evaluate reading order on your data. - Fine-tune the layout model on your own annotations. Reading order is already trained into OXR, so you don't need to label it. The built-in model is based on D-FINE, a recent SOTA object detector. We trained and optimized it, and OXR ships the PyTorch model. D-FINE's authors don't provide deployment support, so we also optimized its operators and exported it for ONNX and TensorRT. That engineering code isn't open source yet because of commercial contracts we've already delivered. Where reading order belongs Beyond model capability, how tasks fit together can be what separates one pipeline from another. Combining tasks or splitting them each has benefits and costs. Reading order rarely makes benchmark headlines, but it decides how fast your pipeline runs and how much labeling it takes to adapt a model to your documents. With reading order handled inside OXR, you can spend your effort on the layout rules that are specific to your business. Real documents are more varied than any design anticipates. Try OXR on a page with a complex layout. If you find a page where OXR's reading order breaks, please share it through our feedback form. Those are exactly the cases we want to learn from. --- About OXR and SoMark OXR is a 1.3B open-source document parsing model from the team behind SoMark. It ranks #1 on olmOCR-Bench, OmniDocBench, and DocParsingBench and delivers SOTA parsing speed. The model uses a recursive self-improvement (RSI) loop during post-training. **If OXR saves you time, give it a ⭐ on GitHub: SoMarkAI/OXR**

OXR wins on both axes. 100 pages, single RTX PRO 6000: OXR 101.0 s | TeleOCR 735.8 s olmOCR-Bench · OmniDocBench · DocParsingBench, first on each, biggest lead on tables Pluggable layout analysis 1.3B, RSI, parallel recognition and ordering. SoMarkAI/OXR

If you’re building RAG pipelines or agents that read PDFs, this is the one I’d test first. It’s open source and outputs structured Markdown or JSON, so complex documents come out ready to use. Explore the GitHub repo: SoMarkAI/OXR Try the online demo: free-oxr.somark.ai

Hi HN,I'm Binyang.founder of SoMark.Our team developed OXR. We believe a small model can go a long way with a systematic way to get the right training data-that's why we built OXR's RSI loop. Happy to answer your quetions! if OXR helps you.consider giving it a Star on github SoMarkAI/OXR

OXR in plain words: a free model turning PDFs into clean data, molecules included. Tops olmOCR-Bench, OmniDocBench, DocParsingBench. 6x+ Surya-OCR-2's speed on an RTX PRO 6000. 1.3B. Layout is yours to replace. RSI, ordering in step with reading. SoMarkAI/OXR

OXR – An RSI-trained 1.3B open-source SOTA document parsing model