Running OCR entirely in the browser (and making scanned PDFs editable)
A few months ago I needed to change one line in a scanned contract. Every "free" PDF site I tried wanted me to upload the file to their server first. One let me edit for ten minutes and then asked me to pay to download the result.
That's a strange deal for a contract, a passport scan or a payslip. So I started building PDFkind, a PDF editor where everything happens in the browser. The hardest part by far was OCR: reading the text out of a scan, then turning it into real text you can edit.
This post is about how that works, what broke along the way, and the numbers I got. Everything runs client-side; the server only hands out static files.
The constraint: no server, no exceptions
The rule I set myself was simple: the document never leaves the device. No "we delete it after an hour", no API call to a cloud OCR service. That rules out the easy options and leaves two questions:
1. Which OCR engine can run in a browser tab with acceptable quality?
2. How big a download can I ask people to accept?
First try: Tesseract.js
Tesseract.js is the obvious choice. It's mature, easy to use, and works well on clean scans. On my test set it was fine for flat, well-lit documents.
Then I tried phone photos: slightly tilted pages, shadows, uneven lighting. That's what people actually have when they "scan" something. Tesseract mostly fell apart:
| Document type | Tesseract.js | PaddleOCR (what I ended up with) |
|---|---|---|
| Poor-quality scans | 3.5 to 6.4% of characters wrong | about 1% |
| Phone photos | unusable | 10 to 14% |
(Character error rate on my own small test set of French and English documents. Not a benchmark, but the gap was big enough to decide.)
The difference comes from how the text is found. Tesseract relies on classical layout analysis, which struggles once the page isn't flat and evenly lit. PaddleOCR uses a neural network to find text lines first, and that detector copes much better with real-world photos.
PaddleOCR in the browser with ONNX Runtime Web
PaddleOCR is normally a Python library, but its models can be exported to ONNX and run with onnxruntime-web in WebAssembly. I use the PP-OCRv5 mobile models:
- a detection model that finds text lines
- a recognition model for Latin-script languages (English, French, Spanish, German and so on)
Together with the WASM runtime that's about 20 MB. It's only downloaded the first time someone uses OCR, and it's cached for 30 days:
// next.config.ts
async headers() {
return [{
source: '/ocr/:file*',
headers: [{ key: 'Cache-Control', value: 'public, max-age=2592000, stale-while-revalidate=86400' }],
}];
}
One setting worth knowing about: I run ONNX Runtime with numThreads = 1. Multi-threaded WASM needs SharedArrayBuffer, which in turn needs the page to be cross-origin isolated (COOP/COEP headers). Those headers break a lot of third-party embeds. One thread is slower, but a page of OCR still takes a few seconds, which is fine.
OCR happens in two steps: find the lines, then read each line.
Step 1: finding the text lines
The detector (a DBNet model) takes the page image and returns a probability map: for every pixel, how likely it is to be text.
Preparing the input has a couple of traps. The image size must be a multiple of 32, and PaddleOCR models expect BGR channel order, not RGB, normalized with the ImageNet mean and standard deviation:
const MEAN = [0.485, 0.456, 0.406], STD = [0.229, 0.224, 0.225];
// resize so the longest side is at most 1600 px, rounded to multiples of 32
const k = Math.min(1, 1600 / Math.max(canvas.width, canvas.height));
const W = Math.max(32, Math.round((canvas.width * k) / 32) * 32);
const H = Math.max(32, Math.round((canvas.height * k) / 32) * 32);
const px = getPixels(canvas, W, H); // RGBA from a resized canvas
const plane = W * H, input = new Float32Array(3 * plane);
for (let i = 0, p = 0; p < plane; i += 4, p++) {
input[p] = (px[i + 2] / 255 - MEAN[0]) / STD[0]; // B
input[plane + p] = (px[i + 1] / 255 - MEAN[1]) / STD[1]; // G
input[2 * plane + p] = (px[i] / 255 - MEAN[2]) / STD[2]; // R
}
const out = await session.run({ [session.inputNames[0]]: new ort.Tensor('float32', input, [1, 3, H, W]) });
The probability map isn't a list of boxes yet. Turning it into lines is the post-processing that the Python library normally does for you, so I had to rewrite it in TypeScript:
1. Threshold the map (pixels above 0.3 count as text).
2. Flood fill to group connected text pixels into regions.
3. Drop regions that are too small or whose average score is below 0.6.
4. Compute each region's convex hull, then the minimum-area rotated rectangle around it. This is what makes tilted lines work.
5. Expand ("unclip") the rectangle, because the model predicts a slightly shrunk core of each line.
The expansion distance comes from the DBNet paper: area × ratio ÷ perimeter.
const d = (r.w * r.h * UNCLIP) / (2 * (r.w + r.h)); // UNCLIP = 1.6
const w = r.w + 2 * d, h = r.h + 2 * d;
The bug that cost me a day: my first version found the four corners of each box by sorting them by y coordinate. That works on a flat scan. On a tilted photo, the top-right corner of a line can be lower than its bottom-left corner, so lines got read upside down or mirrored. Recognition on photos was garbage and I couldn't figure out why.
The fix was to stop guessing corners from coordinates and derive them from the rectangle's angle instead. I keep the angle within ±45° (swapping width and height when needed), so the width always runs along the text line. Then the corners come out in a known order: top-left, top-right, bottom-right, bottom-left.
Step 2: reading each line
The recognizer expects one straight, horizontal strip of text, 48 pixels high. So each detected line, which may be tilted, has to be cut out and straightened.
Canvas can do this with a single affine transform. Map three corners of the detected box to the corners of the output strip, invert the matrix, and draw the page through it:
function crop(src: HTMLCanvasElement, quad: [number, number][], boxW: number, boxH: number, outH = 48) {
const w = Math.max(8, Math.round((boxW / boxH) * outH)), h = outH;
const c = document.createElement('canvas');
c.width = w; c.height = h;
const g = c.getContext('2d')!;
const [p0, p1, , p3] = quad; // top-left, top-right, bottom-left
// matrix mapping strip coordinates to page coordinates...
const a = (p1[0] - p0[0]) / w, b = (p1[1] - p0[1]) / w;
const cc = (p3[0] - p0[0]) / h, d = (p3[1] - p0[1]) / h;
// ...inverted, so drawing the page lands the line flat in the strip
const det = a * d - b * cc;
const ia = d / det, ib = -b / det, ic = -cc / det, id = a / det;
g.setTransform(ia, ib, ic, id, -(ia * p0[0] + ic * p0[1]), -(ib * p0[0] + id * p0[1]));
g.drawImage(src, 0, 0);
return g.getImageData(0, 0, w, h);
}
The recognizer outputs, for each horizontal position in the strip, a probability for every character in its dictionary. Turning that into text is CTC greedy decoding: take the most likely character at each step, merge repeats, drop the "blank" symbol.
// dictionary: blank first, space last (PaddleOCR convention)
const dict = ['', ...chars, ' '];
let text = '', prev = -1, conf = 0, n = 0;
for (let t = 0; t < steps; t++) {
let best = 0, max = -Infinity;
for (let k = 0; k < classes; k++) {
const v = out[t * classes + k];
if (v > max) { max = v; best = k; }
}
if (best !== 0 && best !== prev) { text += dict[best]; conf += max; n++; }
prev = best;
}
The average probability of the kept characters gives a confidence score. I skip lines below 0.4: they're usually logos, stamps or noise.
Arabic needed a different approach
I'm from Morocco, and a lot of documents here mix French and Arabic, sometimes in the same line. PaddleOCR's Arabic recognizer was disappointing in my tests: it dropped digits and letters often enough to be useless for things like dates and ID numbers.
What worked was a hybrid:
- keep PaddleOCR's detector to find the lines, since it's robust to photos
- read Arabic lines with Tesseract in single-line mode, on a cleaned-up crop of each line
The cleanup makes a big difference with Tesseract. For each line strip I estimate the background with a max filter (the paper is the brightest thing around), blur it, divide the image by it to flatten shadows and uneven light, then stretch the contrast between two percentiles. Tesseract then sees near-black text on near-white paper.
Two more details:
- Line joining. The detector often splits a line at wide gaps, for example after a colon. For Arabic I join pieces that sit on the same row and are close together (gap under 2.5× the line height), then order them right to left. The gap limit matters: without it, a French heading on the left and an Arabic heading on the right got merged into one nonsense line.
- A Tesseract setting that silently drops spaces. I had preserve_interword_spaces: '1' enabled. On Arabic it removed the spaces between words entirely ("شهادةعمل" instead of "شهادة عمل"). Turning it off fixed it. If your Arabic OCR output has words glued together, check this first.
On my Arabic test documents the hybrid gets about 2 to 4% of characters wrong on scans and around 4% on phone photos. Not perfect, but very usable.
From "searchable" to "editable"
Recognizing text is half the job. The classic output of OCR is a searchable PDF: the recognized text is added as an invisible layer behind the image, so you can select and copy it while the page looks unchanged.
That's useful, but it doesn't answer the question people actually ask: "how do I change the text in my scan?" So the editor has an option to go further. Each recognized line gets:
1. a cover in the color of the paper, hiding the original pixels
2. a real text box on top, with the same words, roughly the same font, size, weight, slant and ink color
Getting that to look right is mostly careful measuring.
Paper and ink color. The paper color is the most common color along the edges of the line strip (bucketed to 5 bits per channel so noise doesn't matter). The ink color is the average of the darkest pixels inside the letters. Edge pixels blend with the paper, so averaging all of them would make the text look washed out.
Font size. I measure the height of the ink from the top of the letters to the baseline, then ask the browser how tall the same text is in a candidate font:
ctx.font = `100px ${family}`;
const m = ctx.measureText(text);
const size = inkHeightAboveBaseline / (m.actualBoundingBoxAscent / 100);
Using actualBoundingBoxAscent for the exact text matters. A line with no capitals or tall letters ("one more") is shorter than "Total", and a fixed cap-height ratio gets the size wrong.
Which font? I can't identify arbitrary typefaces, but I can tell serif from sans serif. For each candidate (Helvetica, Times), I draw the recognized text into an offscreen canvas, stretched to the same length as the original, and measure how well it overlaps the original's letter shapes (intersection over union). The best overlap wins. Since a page is usually set in one typeface, a family that clearly wins most lines also gets a bonus on close calls.
Bold. My first attempt compared stroke thickness, and it failed on blurry scans, where everything looks bold. What works is comparing the total amount of ink in a line with how much ink the regular-weight font would use for the same text. Blur spreads ink out but doesn't change the total. And because a faded scan affects every line alike, I compare each line with the page's median rather than an absolute threshold.
Italic. I shear the letter shapes by different amounts and keep the shear at which the vertical strokes line up best (the sum of squared column counts peaks). Upright text peaks at 0; italic peaks at around 0.1 to 0.25.
Width. Finally, a little letter spacing makes the new text exactly as long as the original, so it doesn't overlap whatever comes after it.
The result isn't pixel-perfect. Unusual fonts come out as Helvetica or Times, and handwriting is left alone. But for the common case (fix a date, a name, an amount on a scanned letter) it's close enough that the change is hard to spot.
What I'd tell someone trying this
- Test on phone photos early. Clean scans hide almost every problem.
- ONNX Runtime Web is production-ready, but you have to rewrite the pre- and post-processing that Python libraries normally hide. Budget time for it.
- Cache the models aggressively and load them only when the feature is used. Nobody should download 20 MB just to merge two PDFs.
- Measure with real documents and track a simple error rate. Several of my "obvious" improvements made things worse.
Try it
Everything above is live in PDFkind. It's free, with no signup. Open a scanned PDF in the editor, tick "Make scanned text editable", and you can check in DevTools that your file never goes over the network.
I'd love feedback, especially from anyone who has worked on OCR or PDF internals. Where does it break for you?
2 reactions · 7 comments
Mentions
5. 사진 찍은 서류나 표 많은 PDF에서 글자와 표를 정리된 문서로 뽑아줍니다. 수식, 도장, 차트까지 읽어내요. PaddlePaddle/PaddleOCR
Try paddleocr.dev Select PP-OCRv6 small. Choose an image. Open JSON and Download for coordinates and confidence scores. Independent demo using the official SDK at github.com/PaddlePaddle/PaddleOCR/tree/main/paddl…
5 FREE tools to pull text from scanned PDFs (on your PC) A scanned PDF is an image, so you can't search or copy it. We review what each open-source project promises and where it falls short, depending on your task. Chapters: 00:00 Hook 00:18 What OCR is 00:45 OCRmyPDF 01:01 docTR 01:20 docTR in its README 01:36 RapidOCR 01:53 Language limit 02:08 MarkItDown 02:29 MarkItDown caveat 02:44 PaddleOCR 03:10 Language comparison 03:34 How a page gets read 03:52 Choose by task 04:17 What we don't know 04:42 Privacy 05:07 Declared adoption 05:23 Answer 05:41 Closing Official source: PaddlePaddle/PaddleOCR Subscribe to RadarIA to understand every AI release in a few minutes. Avatar, voice and illustrations generated with AI; the official page captures are real. Original music by RadarIA. Sound effects: Pixabay Content License. #OCR #AI #PDF
Choose a parser before feeding PDFs to AI | 6 open-source choices Bad parsing can ruin good AI answers. Six options, with different use cases and adoption checks. Docling, PaddleOCR, olmOCR, Tesseract, EasyOCR, Unstructured. Explore the original cards: channel profile → AI Open Source Radar → Collections. Choose the collection for this task and inspect the project cards. useaistation.com Sources checked 23 Oct 2026. This is a project-selection guide, not hands-on testing or a live ranking. These are different choices, not a verified integrated stack. Software licenses, model weights, hardware and external service costs must be checked separately. Community/core and enterprise features may differ. Developer disclosure: we build AI Open Source Radar, not the featured projects. Narration: Microsoft free Aria neural voice. Project and License sources (observed branch references, subject to change): Docling: docling-project/docling License: raw.githubusercontent.com PaddleOCR: PaddlePaddle/PaddleOCR License: raw.githubusercontent.com olmOCR: allenai/olmocr License: raw.githubusercontent.com Tesseract: tesseract-ocr/tesseract License: raw.githubusercontent.com EasyOCR: JaidedAI/EasyOCR License: raw.githubusercontent.com Unstructured: Unstructured-IO/unstructured License: raw.githubusercontent.com #OpenSource #AITools #DocumentAI
[Paper] Virtual Head Attention >The design of attention mechanisms in long-context large language models faces an ``impossible triangle'' among KV cache size, training and prefill FLOPs, and model quality. GQA achieves strong quality but incurs high KV cache overhead; alternative approaches such as MLA and MFA can compress the cache, but MLA suffers from quality loss while MFA substantially increases training and prefill compute. We propose Virtual Head Attention (VHA), which simultaneously improves all three vertices through two lightweight linear operations: Q Premix applies a near-identity transformation to queries within each KV group to recover virtual head diversity from halved physical query heads, and Linear Postmix fuses inter-head features via a low-rank I+AB⊤ residual structure. Since VHA introduces no nonlinear operations, Premix can be folded into the query projection and Postmix into the output projection at inference time, reducing the model to a standard GQA-2 architecture that directly reuses all existing GQA inference optimizations. With only two KV heads, the group size in large models readily reaches 64, fully exploiting Tensor Core parallelism during decoding. Across four scales with 4K-context 50B-token pretraining, VHA consistently surpasses the standard GQA baseline at every scale (+0.05\% to +0.81\%), while reducing KV cache by 4× vs.\ GQA-8 at dense scales and 2× vs.\ GQA-4 at the MoE scale, and lowering training and prefill FLOPs by 7.9\% (1.7B, S=4096). On an industrial 30B-A3B MoE model trained on 500B tokens at 8K context, VHA outperforms both MLA and GQA-4 by +0.70\,/\,+2.60\,\% on a 12-benchmark base-model evaluation suite; in a separate matched 10B-A1B MoE comparison after long-context mid-training, VHA attains an average RULER score of 84.4 over 8K--128K versus 73.0 for GQA, while using two rather than four KV heads in the full-attention layers. Please read below items as above text's formatting is mess. - arXiv : arxiv.org - PDF : arxiv.org - GitHub : PaddlePaddle/VHA
[Paper] FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention >Long-context training of large language and multimodal models couples structured attention masks (document, prefix-LM, share-question, sliding-window, block-sparse) with multi-node context parallelism, making the mask a dominant bottleneck. FlashMask's column-wise sparse mask compresses them from O(n2) to O(n) via four query-axis boundary vectors, covering broad structured patterns. Implemented for Ampere GPUs, it neither exploits FlashAttention-3's TMA/WGMMA warp-specialized pipeline on Hopper nor supports distribution. The column-wise sparse mask is indexed by key positions but stores query-axis interval boundaries. Under any multi-chunk context-parallel assignment, it reduces per-rank mask localization to a provably zero-communication \emph{clip--shift--max} on the boundary vectors, compatible with the single-GPU kernel. FlashMask-3 introduces a mask-aware Hopper kernel with three-role warp specialization atop FlashAttention-3's TMA/WGMMA pipeline, plus a mask-aware distributed extension: zero-communication mask localization, a sparsity-aware load balancer (HCC-IPO), and a mask-aware compute-communication overlap with mask-driven communication pruning, whose forward all-gather uses topology-aware hierarchical routing. On a single H100 across 12 masks at 4K--128K context, it improves achieved TFLOPs over FlashMask, FlexAttention, and MagiAttention by 45--141%, 34--71%, and 2--44%. At context-parallel degrees 4--32 with 8K sequence per device, it attains 31--75% compute-communication overlap efficiency and improves throughput over Megatron-Core CP ring, Megatron-Core CP all-gather, and MagiAttention by 1.80×--3.71×, 1.71×--2.92×, and 1.09×--1.46×. In end-to-end training of a 118B-A14B MoE at 128K on 128 H100 GPUs, it raises throughput by 7.3--11.0% over a Dual Chunk all-gather baseline, with attention forward+backward 24.5--44.9% faster. - arXiv : arxiv.org - PDF : arxiv.org - GitHub : PaddlePaddle/FlashMask - Project : enigmatisms.github.io
@VikParuchuri How it is faster than PaddleOCR: PADDLEPADDLE/PADDLEOCR Im using Paddle, but faster is better.
Excited to announce that FlashMask-3 has been accepted at #NeurIPS2026! GitHub: PaddlePaddle/FlashMask Paper: arxiv.org Project: enigmatisms.github.io
Running OCR entirely in the browser (and making scanned PDFs editable) A few months ago I needed to change one line in a scanned contract. Every "free" PDF site I tried wanted me to upload the file to their server first. One let me edit for ten minutes and then asked me to pay to download the result. That's a strange deal for a contract, a passport scan or a payslip. So I started building PDFkind, a PDF editor where everything happens in the browser. The hardest part by far was OCR: reading the text out of a scan, then turning it into real text you can edit. This post is about how that works, what broke along the way, and the numbers I got. Everything runs client-side; the server only hands out static files. The constraint: no server, no exceptions The rule I set myself was simple: the document never leaves the device. No "we delete it after an hour", no API call to a cloud OCR service. That rules out the easy options and leaves two questions: 1. Which OCR engine can run in a browser tab with acceptable quality? 2. How big a download can I ask people to accept? First try: Tesseract.js Tesseract.js is the obvious choice. It's mature, easy to use, and works well on clean scans. On my test set it was fine for flat, well-lit documents. Then I tried phone photos: slightly tilted pages, shadows, uneven lighting. That's what people actually have when they "scan" something. Tesseract mostly fell apart: | Document type | Tesseract.js | PaddleOCR (what I ended up with) | |---|---|---| | Poor-quality scans | 3.5 to 6.4% of characters wrong | about 1% | | Phone photos | unusable | 10 to 14% | (Character error rate on my own small test set of French and English documents. Not a benchmark, but the gap was big enough to decide.) The difference comes from how the text is found. Tesseract relies on classical layout analysis, which struggles once the page isn't flat and evenly lit. PaddleOCR uses a neural network to find text lines first, and that detector copes much better with real-world photos. PaddleOCR in the browser with ONNX Runtime Web PaddleOCR is normally a Python library, but its models can be exported to ONNX and run with onnxruntime-web in WebAssembly. I use the PP-OCRv5 mobile models: - a detection model that finds text lines - a recognition model for Latin-script languages (English, French, Spanish, German and so on) Together with the WASM runtime that's about 20 MB. It's only downloaded the first time someone uses OCR, and it's cached for 30 days: // next.config.ts async headers() { return [{ source: '/ocr/:file*', headers: [{ key: 'Cache-Control', value: 'public, max-age=2592000, stale-while-revalidate=86400' }], }]; } One setting worth knowing about: I run ONNX Runtime with numThreads = 1. Multi-threaded WASM needs SharedArrayBuffer, which in turn needs the page to be cross-origin isolated (COOP/COEP headers). Those headers break a lot of third-party embeds. One thread is slower, but a page of OCR still takes a few seconds, which is fine. OCR happens in two steps: find the lines, then read each line. Step 1: finding the text lines The detector (a DBNet model) takes the page image and returns a probability map: for every pixel, how likely it is to be text. Preparing the input has a couple of traps. The image size must be a multiple of 32, and PaddleOCR models expect BGR channel order, not RGB, normalized with the ImageNet mean and standard deviation: const MEAN = [0.485, 0.456, 0.406], STD = [0.229, 0.224, 0.225]; // resize so the longest side is at most 1600 px, rounded to multiples of 32 const k = Math.min(1, 1600 / Math.max(canvas.width, canvas.height)); const W = Math.max(32, Math.round((canvas.width * k) / 32) * 32); const H = Math.max(32, Math.round((canvas.height * k) / 32) * 32); const px = getPixels(canvas, W, H); // RGBA from a resized canvas const plane = W * H, input = new Float32Array(3 * plane); for (let i = 0, p = 0; p < plane; i += 4, p++) { input[p] = (px[i + 2] / 255 - MEAN[0]) / STD[0]; // B input[plane + p] = (px[i + 1] / 255 - MEAN[1]) / STD[1]; // G input[2 * plane + p] = (px[i] / 255 - MEAN[2]) / STD[2]; // R } const out = await session.run({ [session.inputNames[0]]: new ort.Tensor('float32', input, [1, 3, H, W]) }); The probability map isn't a list of boxes yet. Turning it into lines is the post-processing that the Python library normally does for you, so I had to rewrite it in TypeScript: 1. Threshold the map (pixels above 0.3 count as text). 2. Flood fill to group connected text pixels into regions. 3. Drop regions that are too small or whose average score is below 0.6. 4. Compute each region's convex hull, then the minimum-area rotated rectangle around it. This is what makes tilted lines work. 5. Expand ("unclip") the rectangle, because the model predicts a slightly shrunk core of each line. The expansion distance comes from the DBNet paper: area × ratio ÷ perimeter. const d = (r.w * r.h * UNCLIP) / (2 * (r.w + r.h)); // UNCLIP = 1.6 const w = r.w + 2 * d, h = r.h + 2 * d; The bug that cost me a day: my first version found the four corners of each box by sorting them by y coordinate. That works on a flat scan. On a tilted photo, the top-right corner of a line can be lower than its bottom-left corner, so lines got read upside down or mirrored. Recognition on photos was garbage and I couldn't figure out why. The fix was to stop guessing corners from coordinates and derive them from the rectangle's angle instead. I keep the angle within ±45° (swapping width and height when needed), so the width always runs along the text line. Then the corners come out in a known order: top-left, top-right, bottom-right, bottom-left. Step 2: reading each line The recognizer expects one straight, horizontal strip of text, 48 pixels high. So each detected line, which may be tilted, has to be cut out and straightened. Canvas can do this with a single affine transform. Map three corners of the detected box to the corners of the output strip, invert the matrix, and draw the page through it: function crop(src: HTMLCanvasElement, quad: [number, number][], boxW: number, boxH: number, outH = 48) { const w = Math.max(8, Math.round((boxW / boxH) * outH)), h = outH; const c = document.createElement('canvas'); c.width = w; c.height = h; const g = c.getContext('2d')!; const [p0, p1, , p3] = quad; // top-left, top-right, bottom-left // matrix mapping strip coordinates to page coordinates... const a = (p1[0] - p0[0]) / w, b = (p1[1] - p0[1]) / w; const cc = (p3[0] - p0[0]) / h, d = (p3[1] - p0[1]) / h; // ...inverted, so drawing the page lands the line flat in the strip const det = a * d - b * cc; const ia = d / det, ib = -b / det, ic = -cc / det, id = a / det; g.setTransform(ia, ib, ic, id, -(ia * p0[0] + ic * p0[1]), -(ib * p0[0] + id * p0[1])); g.drawImage(src, 0, 0); return g.getImageData(0, 0, w, h); } The recognizer outputs, for each horizontal position in the strip, a probability for every character in its dictionary. Turning that into text is CTC greedy decoding: take the most likely character at each step, merge repeats, drop the "blank" symbol. // dictionary: blank first, space last (PaddleOCR convention) const dict = ['', ...chars, ' ']; let text = '', prev = -1, conf = 0, n = 0; for (let t = 0; t < steps; t++) { let best = 0, max = -Infinity; for (let k = 0; k < classes; k++) { const v = out[t * classes + k]; if (v > max) { max = v; best = k; } } if (best !== 0 && best !== prev) { text += dict[best]; conf += max; n++; } prev = best; } The average probability of the kept characters gives a confidence score. I skip lines below 0.4: they're usually logos, stamps or noise. Arabic needed a different approach I'm from Morocco, and a lot of documents here mix French and Arabic, sometimes in the same line. PaddleOCR's Arabic recognizer was disappointing in my tests: it dropped digits and letters often enough to be useless for things like dates and ID numbers. What worked was a hybrid: - keep PaddleOCR's detector to find the lines, since it's robust to photos - read Arabic lines with Tesseract in single-line mode, on a cleaned-up crop of each line The cleanup makes a big difference with Tesseract. For each line strip I estimate the background with a max filter (the paper is the brightest thing around), blur it, divide the image by it to flatten shadows and uneven light, then stretch the contrast between two percentiles. Tesseract then sees near-black text on near-white paper. Two more details: - Line joining. The detector often splits a line at wide gaps, for example after a colon. For Arabic I join pieces that sit on the same row and are close together (gap under 2.5× the line height), then order them right to left. The gap limit matters: without it, a French heading on the left and an Arabic heading on the right got merged into one nonsense line. - A Tesseract setting that silently drops spaces. I had preserve_interword_spaces: '1' enabled. On Arabic it removed the spaces between words entirely ("شهادةعمل" instead of "شهادة عمل"). Turning it off fixed it. If your Arabic OCR output has words glued together, check this first. On my Arabic test documents the hybrid gets about 2 to 4% of characters wrong on scans and around 4% on phone photos. Not perfect, but very usable. From "searchable" to "editable" Recognizing text is half the job. The classic output of OCR is a searchable PDF: the recognized text is added as an invisible layer behind the image, so you can select and copy it while the page looks unchanged. That's useful, but it doesn't answer the question people actually ask: "how do I change the text in my scan?" So the editor has an option to go further. Each recognized line gets: 1. a cover in the color of the paper, hiding the original pixels 2. a real text box on top, with the same words, roughly the same font, size, weight, slant and ink color Getting that to look right is mostly careful measuring. Paper and ink color. The paper color is the most common color along the edges of the line strip (bucketed to 5 bits per channel so noise doesn't matter). The ink color is the average of the darkest pixels inside the letters. Edge pixels blend with the paper, so averaging all of them would make the text look washed out. Font size. I measure the height of the ink from the top of the letters to the baseline, then ask the browser how tall the same text is in a candidate font: ctx.font = `100px ${family}`; const m = ctx.measureText(text); const size = inkHeightAboveBaseline / (m.actualBoundingBoxAscent / 100); Using actualBoundingBoxAscent for the exact text matters. A line with no capitals or tall letters ("one more") is shorter than "Total", and a fixed cap-height ratio gets the size wrong. Which font? I can't identify arbitrary typefaces, but I can tell serif from sans serif. For each candidate (Helvetica, Times), I draw the recognized text into an offscreen canvas, stretched to the same length as the original, and measure how well it overlaps the original's letter shapes (intersection over union). The best overlap wins. Since a page is usually set in one typeface, a family that clearly wins most lines also gets a bonus on close calls. Bold. My first attempt compared stroke thickness, and it failed on blurry scans, where everything looks bold. What works is comparing the total amount of ink in a line with how much ink the regular-weight font would use for the same text. Blur spreads ink out but doesn't change the total. And because a faded scan affects every line alike, I compare each line with the page's median rather than an absolute threshold. Italic. I shear the letter shapes by different amounts and keep the shear at which the vertical strokes line up best (the sum of squared column counts peaks). Upright text peaks at 0; italic peaks at around 0.1 to 0.25. Width. Finally, a little letter spacing makes the new text exactly as long as the original, so it doesn't overlap whatever comes after it. The result isn't pixel-perfect. Unusual fonts come out as Helvetica or Times, and handwriting is left alone. But for the common case (fix a date, a name, an amount on a scanned letter) it's close enough that the change is hard to spot. What I'd tell someone trying this - Test on phone photos early. Clean scans hide almost every problem. - ONNX Runtime Web is production-ready, but you have to rewrite the pre- and post-processing that Python libraries normally hide. Budget time for it. - Cache the models aggressively and load them only when the feature is used. Nobody should download 20 MB just to merge two PDFs. - Measure with real documents and track a simple error rate. Several of my "obvious" improvements made things worse. Try it Everything above is live in PDFkind. It's free, with no signup. Open a scanned PDF in the editor, tick "Make scanned text editable", and you can check in DevTools that your file never goes over the network. I'd love feedback, especially from anyone who has worked on OCR or PDF internals. Where does it break for you?
PaddleOCR 3.7.0 업데이트에서 PP-OCRv6가 공개되었네요. 34.5M 파라미터의 미디엄 티어 모델로 대형 VLM을 능가하는 정확도를 보여주며, CPU와 애플 M4에서도 빠르게 추론할 수 있어 온프레미스 문서 파싱에 유용해 보입니다. #AI #개발 github.com/PaddlePaddle/PaddleOCR/blob/main/readm…