101arrowz

Sharing a Symfony project in a URL, without storing anything --- title: Sharing a Symfony project in a URL, without storing anything published: true tags: php, symfony, webdev, showdev canonical_url: apprendre.forelse.fr cover_image: apprendre.forelse.fr --- "Can you show me your code?" On Slack, the answer is usually a forty-line paste, without the Twig template that goes with it, without the route, and with no way to run it. Whoever wants to help copies it all into a local project, notices a file is missing, and the conversation fizzles out. I added a playground to apprendre.forelse.fr: a blank Symfony 8 or Laravel 13 project that runs in the browser. You edit a controller, the preview updates, and a "Share" button gives you a link. Whoever opens it gets the project as it is, ready to run. The interesting part isn't the button, it's what isn't behind it: no database, no storage, no share ID. All the code travels in the URL. Here is how, and the two questions it raises: size, and security. The project already runs in the browser I described the mechanics in a previous article: PHP 8.4 compiled to WebAssembly, a real Symfony project with its vendor/, and a Service Worker serving the preview. The server only delivers static files. The playground reuses exactly that machinery. To the platform, it's an exercise with no objective and no test: a starting project, a list of editable files, and that's it. The editor, the console (bin/console, php artisan) and the preview are the same as in the exercises. That left one question: how do you hand your copy of the project to someone else? Why not server-side storage The classic answer, the one 3v4l and CodePen use, is to save the code and hand out a short ID: /s/a8Fk2. It's clean, but it has a cost: - a table that grows forever, full of content nobody moderates; - a GDPR question with every share, since the code can contain anything; - links that die the day you clean up. For a learning playground, I wanted none of that. And the browser offers a perfect place to carry data without sending it to the server: the URL fragment , everything after the #. The fragment, the part of the URL the server never sees When you open https://apprendre.forelse.fr/en/playground/symfony#code=…, the browser asks the server for /en/playground/symfony and keeps the rest to itself. The fragment shows up neither in the HTTP request, nor in the server logs, nor in the Referer header. Only the page's JavaScript can read it, through location.hash. That's exactly what's needed: the public page stays the same for everyone (one URL to index, one canonical), and the code only exists in the link and in the browsers of the people who open it. One exception to watch: analytics. Some trackers send the full URL, fragment included. The site uses Umami, which has an option for that (tag simplified): <script defer src="/mesure/traceur.js" data-website-id="…" data-exclude-hash="true"></script> Without it, every visitor's code would have ended up in the stats. Fitting a project in a link A full Symfony project weighs several megabytes, vendor/ included. But everything that doesn't change is already known: it's the starting project, the same for everyone. So the link only carries the difference, that is the files changed or created. export function changedFiles(current: Record<string, string>, starting: Record<string, string>): Record<string, string> { return Object.fromEntries(Object.entries(current).filter(([path, content]) => starting[path] !== content)); } Those files are turned into JSON, compressed with deflate (the fflate library, already there to unpack the PHP environment), then encoded in base64url, the base64 variant that only uses URL-safe characters: export function encodeFragment(files: Record<string, string>): string { const bytes = deflateSync(strToU8(JSON.stringify({ v: 1, files })), { level: 9 }); let binary = ''; for (let i = 0; i < bytes.length; i += 0x8000) binary += String.fromCharCode(...bytes.subarray(i, i + 0x8000)); return 'code=' + btoa(binary).replace(/\+/g, '-').replace(/\//g, '\_').replace(/=+$/, ''); } The v: 1 costs a few bytes, but it will let the format change without breaking links already shared. In practice, it's very reasonable. The starting project's controller and Twig template, slightly edited, come to about 1,500 characters. Compressed and encoded, the fragment is a little over 1,000 characters. PHP code compresses well: lots of repeated keywords, use statements and indentation. One unexpected constraint came from Bluesky, which counts the whole URL in its 300 characters. A shared project quickly goes over: in that case the Bluesky button disappears, and you still have copy link, the phone's share sheet, and LinkedIn and X, which display a shortened address. Reading a link you didn't write Decoding needs more care, because anyone can craft a link. Three rules: Cap the size, before and after decompression. A few kilobytes of deflate can inflate to hundreds of megabytes. The fragment is rejected beyond 200,000 characters, and decompression runs chunk by chunk, stopped as soon as the total exceeds 1 MB: const inflate = new Inflate((chunk) => { size += chunk.length; if (size > MAX_BYTES) throw new RangeError('too-large'); chunks.push(chunk); }); inflate.push(bytes, true); Reject paths that leave the project. A file named ../../etc/passwd or /root.php is ignored. It would be fairly harmless in PHP's in-memory file system, but there is no reason to accept it. Only accept editable files. The link can only write where the visitor could write themselves: for Symfony, src/*/*.php and the Twig templates; for Laravel, the routes, the code under app/ and the Blade views. The framework configuration isn't part of it: a shared link has no business touching config/packages/security.yaml. On opening, the link's code becomes the visitor's draft (replacing any they had left), and the fragment is removed from the address bar with history.replaceState. Otherwise, reloading the page would force the link's code back over the changes made since. Running someone else's code That was the first question I got: is it really reasonable to send someone a link that runs code in their browser? Let's start with what's safe. PHP runs in a Web Worker, in WebAssembly, with no network access: the worker's Content Security Policy only allows fetching the environments and the site's own build files. It can't read the visitor's files or call an API. The HTML and JavaScript the project produces are shown in an iframe served from another origin , with no cookie and no access to the platform. It's the same isolation as for the exercises, where learners already run their own code. But "isolated" doesn't mean "harmless". The page shown in the preview can look like anything: a fake login page, for instance. And an infinite loop can make the CPU spin. So I chose to never automatically run code received through a link. The project loads and the code can be read in the editor, but the preview first shows a warning: > This link contains code written by someone else. Read it in the editor before running it. It runs isolated from this site, but the page it shows can imitate anything: never type a password into it. The preview only starts when you click "Run this code". That one click is deliberate friction, and the right trade-off: someone sharing a bug wants it read, not run blindly. One last security choice: the playground offers Symfony and Laravel, but not Nuxt, even though the platform can simulate it. In the Nuxt simulator, part of the code (the server/ routes) runs on the platform's origin. That's fine for your own code, not for a stranger's. What I take away The URL fragment is an underrated tool. It lets you share a complete state with no backend, no account and nothing kept, as long as the state stays reasonably small. For a code playground, that's exactly the case: you almost never share more than a few files. The flip side is that a link becomes untrusted input like any other. Treat it like a form sent by a stranger: cap, filter, and ask before running. You can try the Symfony playground or the Laravel playground, no account needed. And next time someone asks for your code on Slack, send the link. The engine is open source (AGPL-3.0): the code from this article is on GitHub, in playground/src/app/scratch.ts and Playground.ts.

They Charge Me to Export My Own WhatsApp Contacts. I Shipped a Free Alternative in One Day 🔓 --- description: "Most WhatsApp export extensions stop at 10 to 30 contacts on the free plan. WAVault is free, open source and unlimited: contacts, group members, chats with media and statuses to Excel, CSV, vCard, Google and Outlook." --- By Md. Habibullah Sharif — Full-Stack Engineer | CSE, Northern University Bangladesh --- > 633 contacts in my WhatsApp Business account. > The "free" exporters let me take 10. > The other 623 had a price tag. My own contacts. From my own account. Already loaded in my own browser tab. And someone wanted a monthly subscription to hand them back to me. --- The Paywall My account had grown the way every long-lived WhatsApp account grows: years of phones, several synced devices, customers, suppliers, groups. 957 raw contact records for 633 actual people. I needed them in a spreadsheet. WhatsApp has no export button. That part is well known. So I checked the Chrome Web Store. Pages of exporters, polished icons, confident promises. I tried them one after another, and every one I tried followed the same script: > Free plan: 10 contacts. Some stretch it to 30. > Everything else: upgrade to Premium. I build backends and data pipelines for a living. I know exactly what sits behind that "Premium" button: the data is already in the page's memory, and serialising it to CSV is a few dozen lines. There is no server cost to cover and no expensive API behind it. The paywall isn't covering a cost. It's a toll on your own data. That was the part I refused to accept. I closed the tab and started building. WAVault shipped the same day. --- !WAVault demo: filtering contacts, loading group members, backing up chats, saving statuses --- What You Get, For Nothing WAVault is a free, open-source browser extension for WhatsApp Web and WhatsApp Business. No premium tier. No contact limit. No account. No server. It adds a dashboard to WhatsApp Web with four sections. 1. Contacts, every one of them One row per real person, merged from the address book, chats and groups. 15 one-click lists: Saved · Unsaved · Invite list · Chatted · Unread · Read · With username · Without username · Business · Personal · Group members · Hidden number · Blocked Combine them with Any/Yes/No filters, WhatsApp Business labels and group membership. Search works across scripts (Bangla, Arabic, Hindi, emoji) and across number formats: +44 7700 900123, 07700900123 and 447700900123 resolve to the same person. !Contacts filtered to Business, with filters open and rows selected 2. Group members The member list of every group you belong to, with roles (Owner, Admin, Member). Excel output gives one sheet per group. This is the feature paid tools tend to lock hardest. !WhatsApp groups with member counts and roles 3. Full chat backup, with media Whole conversations, loaded page by page, with photos, videos, voice notes and documents decrypted locally. The output is a ZIP with a folder per chat, an index page, and an HTML viewer that reads like WhatsApp itself: bubbles, replies, inline media, search. It opens offline, in any browser, on any machine. !A backed-up chat in the offline HTML viewer 4. Status downloader Every current status, grouped by contact. Save one, or select many and get a ZIP with a gallery page. Formats | Format | Built for | |---|---| | Excel (.xlsx) | Spreadsheets; numbers stored as text so + survives | | CSV | CRMs and tools; UTF-8 with BOM so non-Latin names render in Excel | | Google Contacts CSV | Direct import at contacts.google.com | | Outlook CSV | Outlook and Microsoft 365 | | vCard 3.0 / 4.0 | iPhone, Android, iCloud | | JSON | Automation | | Numbers list / HTML / PDF | Quick lists, sharing, printing | --- The Architecture WhatsApp Web exposes no public API for any of this. It doesn't need to: the data is already in the tab. The engineering problem is reading it safely, read-only, and without a single byte leaving the machine. [Launcher tab / toolbar] │ ▼ [Content script — React 19 dashboard inside a Shadow DOM] │ window.postMessage RPC (progress events, transferable ArrayBuffers) ▼ [Bridge script — injected into the page's main world] │ read-only calls ▼ [WhatsApp Web's internal modules: contacts, chats, groups, messages, media] | Decision | Reason | |---|---| | WXT | One codebase, native builds for Chrome, Edge, Brave, Opera, Firefox and Safari | | Shadow DOM | Full style isolation in both directions | | Main-world bridge | Content scripts run in an isolated world and can't reach page objects | | postMessage RPC | Typed requests, streamed progress, cancellation, binary-safe media | | Custom .xlsx writer (~130 lines) | A spreadsheet library would outweigh the entire extension | | fflate | Streaming ZIP, so large chat backups don't need the whole archive in memory | Permissions: web.whatsapp.com and storage. Nothing else. No analytics, no telemetry, no remote code. --- Five Problems the Paid Tools Don't Talk About Exporting ten rows is easy. Exporting everything, correctly is where it gets interesting. 1. One person, two identities WhatsApp now assigns every user a private LID alongside their phone number. The same person can appear as …@c.us and as …@lid, and in many groups members appear only as a LID with the number hidden. A naive exporter gives you duplicates and calls it a feature. WAVault first builds the full LID ↔ phone map, then folds the address book, chats, group participants and block list into a single record per person. 957 raw records → 633 real people. That number is the difference between a list and a clean list. 2. WhatsApp changes its internals without notice Mid-build, the function that pages through chat history changed its signature from (chat) to ({ chat, msgCollection, ... }). Users update WhatsApp Web at different times, so both shapes have to work. The resolver tries each and caches the winner: let loadEarlierStyle = null; // 'object' | 'positional' async function loadEarlier(loader, chat) { const styles = loadEarlierStyle ? [loadEarlierStyle] : ['object', 'positional']; let lastError; for (const style of styles) { try { const r = style === 'object' ? await loader.loadEarlierMsgs({ chat, msgCollection: chat.msgs }) : await loader.loadEarlierMsgs(chat); loadEarlierStyle = style; return r; } catch (e) { lastError = e; if (!(e instanceof TypeError) || loadEarlierStyle) break; } } throw lastError; } Every internal access goes through one module with fallbacks. When WhatsApp moves something, WAVault reports which part is unavailable instead of failing silently. That design choice matters more than any single feature. 3. Media decryption expects instrumentation WhatsApp's media pipeline now requires a performance-logging object on every download. WAVault logs nothing, so it passes a self-returning no-op that satisfies any call chain: const noopLogger = new Proxy({}, { get: (_, key) => (key === 'then' ? undefined : () => noopLogger), }); The then guard keeps it from being treated as a thenable. Verified against a live account: decrypted size matches the declared size byte for byte, JPEG header ff d8 ff e0. 4. Spreadsheets are hostile to phone numbers Open a naive CSV in Excel and +447700900123 becomes 4.477E+11. Names in Bangla turn into রহিম. And a contact named =HYPERLINK("http://…") becomes a live formula on someone else's machine. WAVault writes real .xlsx with numbers as text, adds a UTF-8 BOM to CSV, and neutralises formula injection while leaving genuine phone numbers untouched. Contact data is user-controlled input; it gets treated that way. 5. Your build artifacts can leak what git never saw Firefox Add-ons requires a sources zip, and build tools zip the project folder by default, including local test output that .gitignore keeps out of the repo. A release built on a developer machine can quietly ship screenshots and logs from real accounts. WAVault's build excludes every local artifact explicitly. Before release, every published file (git history and all four release zips) was scanned against the 1,111 real phone numbers in my own backup. Zero matches. Every screenshot here uses fictional data from number ranges reserved for film and TV. > 💡 For anyone shipping a data tool: .gitignore protects the repo, not the artifact. Run unzip -l on every zip before it goes public. --- Is It Safe? - Read-only. It never sends messages and never modifies chats, contacts or groups. - Local. No server, no account, no network requests of its own. - No automation. It reads what your tab has already loaded; it doesn't drive WhatsApp. - Auditable. MIT licensed, every line on GitHub. It is an independent project, not affiliated with WhatsApp or Meta. Use it on your own account. --- Install in Two Minutes 1. Download the zip for your browser from the latest release and unzip it. 2. Open chrome://extensions (or edge://, brave://), enable Developer mode, click Load unpacked, and select the folder. Firefox: about:debugging → Load Temporary Add-on. 3. Open web.whatsapp.com and click the WAVault tab on the right edge. Guides for every feature, including importing into Google Contacts, iPhone, Android and Outlook, are in the docs. To run it without logging in, there's a demo mode with generated data: git clone https://github.com/md8-habibullah/wavault-whatsapp-web-backup.git cd wavault-whatsapp-web-backup npm ci && npm run preview # http://127.0.0.1:4789 --- Why It Stays Free Your contacts were never a premium feature. They're yours. Charging people to read data that already sits in their own browser is a business model built on the assumption that users can't see how simple the work behind the paywall is. Now there's a free, open alternative that does more than the paid ones, and anyone can verify every line of it. WAVault will not get a premium tier. That is the one roadmap item that never changes. --- Coming next: Chrome Web Store, Edge and Firefox Add-ons listings, incremental backups, and a Bangla interface. Issues and pull requests are open on GitHub. If WAVault saved you a subscription, a ⭐ on the repo is how other people find it. And if you've hit the same 10-contact wall, tell me which tool in the comments. This post was written with AI assistance and edited by me.

Why your generated Anki image occlusion cards import as plain cloze (and the one-line fix) I've been building AnkiForge, a web app that turns lecture PDFs and labelled diagrams into Anki cards. For diagrams it makes image occlusion cards: every label on, say, a heart diagram gets hidden, one card per label. Getting the masks in the right place took most of the work. Getting Anki to treat them as real image occlusion took one line of config, and finding out what that line actually does took reading Anki's source twice. Here it is, along with the parts that are still broken. A note type named "Image Occlusion" isn't image occlusion Anki has had image occlusion built in since 23.10. Before that it was an add-on (Image Occlusion Enhanced) with its own note type, and plenty of shared decks still use that old format. The Anki forums have a steady trickle of threads like "Anki deck with image occlusion shows no images" from people importing them. If you generate decks in code, the obvious approach is to copy the templates and field names from the built-in IO note type. You end up with a cloze note type that happens to be called "Image Occlusion". The cards even review fine, because the template draws the masks itself. The difference shows up when someone opens a card to fix a mask.This is how Anki desktop decides whether to show its mask editor (aqt/editor_legacy.py in the 26.9 package): def current_notetype_is_image_occlusion(self) -> bool: if not self.note: return False return ( self.note_type().get("originalStockKind", None) == StockNotetype.OriginalStockKind.ORIGINAL_STOCK_KIND_IMAGE_OCCLUSION ) Not the name, not the fields, not the template. One value in the note type's config, and ORIGINAL_STOCK_KIND_IMAGE_OCCLUSION is 6. Without it the image occlusion button stays hidden and the editor shows plain text fields, with the raw {{c1::image-occlusion:rect:...}} string where the masks should be. In a schema 11 collection, note types live as JSON in the col table, so this is what mine looks like: { name: 'Image Occlusion (AnkiForge)', type: 1, // cloze originalStockKind: 6, // the line that matters flds: [ field('Occlusion', 0, { tag: 0, preventDeletion: true }), field('Image', 1, { tag: 1, preventDeletion: true }), field('Header', 2, { tag: 2, preventDeletion: true }), field('Back Extra', 3, { tag: 3, preventDeletion: true }), field('Comments', 4, { tag: 4, preventDeletion: false }), ], // templates copied from Anki's stock IO note type } The field tags and preventDeletion flags are copied from the stock note type too, so mine matches the built-in one field for field. The occlusion field is a string, and I copied it exactly Each mask is a cloze deletion with a shape inside it: {{c1::image-occlusion:rect:left=.2325:top=.1104:width=.1371:height=.0412}}<br> Coordinates are fractions of the image size. Anki formats them with a small function called floatToDisplay: four decimals, then strip leading and trailing zeros, so 0.2325 becomes .2325. "Hide all, guess one" mode adds :oi=1 to each shape. I matched that output character for character, so a note from my export and the same note after a round trip through Anki's editor are identical. I haven't tested how forgiving Anki's parser is about other formats, and I'd rather not find out from a user. Testing with Anki itself I didn't want to check this by opening Anki and clicking around after every change. pip install anki gives you the same Rust backend the desktop app runs, so the test imports the generated .apkg into a fresh collection and asks Anki for the occlusions with get_image_occlusion_note, the call the mask editor uses to load a note. While writing this post I removed the flag from a test deck to show the failure. The test still passed. The backend parses occlusions out of any note with the right field, flag or no flag: >>> m['originalStockKind'] None >>> col.get_image_occlusion_note(note_id=nid) note: 3 occlusions, image 'af-test.png' Only the desktop UI reads the flag, so a backend-only test can't see it missing. The test now checks it directly: col = Collection(os.path.join(tmp, "collection.anki2")) col.import_anki_package(ImportAnkiPackageRequest( package_path=path, options=ImportAnkiPackageOptions())) for nt in col.models.all_names_and_ids(): m = col.models.get(nt.id) nids = col.find_notes(f"mid:{nt.id}") if nids and m["flds"][0]["name"] == "Occlusion": assert m.get("originalStockKind") == 6, "editor won't treat this as IO" io = col.get_image_occlusion_note(note_id=nids[0]) assert io.WhichOneof("value") == "note", io The same script renders the cloze cards and checks that the question side hides the deletion. It runs on every change to the note types or the packer. Lesson for me: a test that passes before and after you break the thing isn't testing the thing. Building the .apkg in the browser An .apkg is a zip with a SQLite database (collection.anki2), a media file mapping "0", "1"... to real filenames, and the media files themselves, named 0, 1, .... I build it in the browser with sql.js (SQLite compiled to WebAssembly) and fflate for the zip. sql.js is about 1 MB of wasm, so it's only loaded when someone clicks Download. The point is that the deck never exists on my server. Diagrams do go through the server to the OCR step, but they aren't stored, and the finished deck is assembled on the user's machine. A few details that took more than one try: - Note GUIDs come from the content, using the same base91 scheme genanki uses. Re-exporting the same card gives the same GUID, so importing an updated deck updates notes instead of duplicating them. - Deck IDs come from the deck name, for the same reason. - Note type IDs are fixed and never change in place. If I change a template, it ships as a new note type with a new name and a new ID. Changing a published note type under the same ID means people who imported the old one get a conflict on their next import. Masks: OCR was the easy part OCR returns lines of text with bounding boxes. Turning those lines into one mask per label is where the work was. Labels on anatomy diagrams wrap across lines ("Superior / vena cava"), so you have to merge lines. But labels also stack in columns ("Nose / Nasal cavity / Pharynx"), and a column of short labels looks exactly like one wrapped label if all you have is geometry. Merge too little and a two-line label becomes two masks. Merge too much and one card shows five answers at once. To measure it I built an eval set: diagrams from Wikimedia Commons with every label written down by hand. For each label the question is whether a student gets exactly one mask over it, switched on, without touching anything. I call that the one-click rate. My first version merged by proximity and scored 64%. For comparison I ran NIOBIUM, a free CLI that does the same job with a 10 px proximity merge, through the same scorer. It got 67%. Both of us found about 93% of the labels. The whole gap was merging: 43 merged masks for NIOBIUM against 2 for my current version. Here's one diagram from the set, same OCR engine, before and after: ![Respiratory system diagram with OCR masks. Left, merging by position only: 14 of 48 labels get their own mask, and whole columns like "Nose / Nasal cavity / Nasal conchae" sit under one red mask. Right, merging only when the text continues: 46 of 48 labels get their own mask.] !Image description Diagram: "Respiratory system complete" by LadyofHats (Mariana Ruiz), public domain, via Wikimedia Commons. What fixed it was making a merge depend on the text as well as the boxes. Two lines merge only if they line up and the text reads as a continuation: the next line starts lowercase or with (, the previous line ends on something like "of", or the next line is a single modifier word. Then the exceptions that the eval found: - subclavius n. above phrenic n. is two nerves, not one label. A line ending in a period ends the label, even if the next line starts lowercase. - musculo- above cutaneous n. is one word split by a hyphen and should join. - On lecture slides, the title, bullet points and page number are text too. They now get masks that are off by default. On the 9 diagrams I tuned against (163 labels), the two OCR engines I use score 95% and 92%. On 7 diagrams I kept aside, the latest rule changes moved them from 91% to 98% and from 87% to 94%. I tuned those rule changes after looking at the failures on that held-out set, so it isn't really held out any more. The next round needs new images. One thing I didn't expect: the newer document-parsing OCR models were much worse at this. Two of them scored 7% and 25%. They see an anatomy diagram as one figure block and don't read the text inside it. They're built to turn scanned papers into Markdown, and for that they're probably great. The cloze side, briefly The text-to-cloze part uses an LLM, and it got its own eval, which I'll write up separately. Two things worth knowing if you're doing something similar: - The model I use reasons before it answers, even at the lowest reasoning setting, and those tokens count against max_tokens. A sibling project had every response cut off at 400 tokens before anyone noticed. I now give it 16,000 and keep the reasoning out of the response. - Models love putting a list under one cloze number: {{c1::mitral}} and {{c1::tricuspid}}. That's a card asking you to recall a set, which is the thing spaced repetition is worst at. There's a prompt rule against it now, and the eval still catches it now and then. What's still broken - Numbered diagrams. If the figure has "1, 2, 3" on it and a legend elsewhere, the mask hides "3" and the answer on the back is "3". The real answer is in the legend. I need to pair numbers with legend entries, and that isn't built yet. - Tiny old scans. A 409×500 plate from the 1918 Gray's Anatomy defeats every engine I've tried. Upscaling before OCR got one engine from 0 to 5 of 11 labels. - Phones. Anki's own library says the cards are image occlusion. I haven't yet opened them on a real phone in AnkiMobile or AnkiDroid. If you're generating decks yourself and Anki's mask editor won't open your IO notes, check originalStockKind first. The tool is at ankiforge.app

I built a free tool that turns AI-generated Markdown tables into Excel Tables from ChatGPT, Claude or Gemini are written in Markdown. Paste one into Excel and every row ends up in column A, pipes and all. So I built a free online tool that converts Markdown tables straight into Excel: md-to.com !Markdown tables into Excel Features - Paste a whole AI answer or upload a .md file, and every table is extracted automatically - Several tables become several sheets, named after the heading above each table - 1,234.50 and 12.5% are saved as real numbers; IDs longer than 15 digits stay as text so Excel can't round them - Bold, frozen header row and auto-sized columns - Download an .xlsx, or copy with one click and paste into Excel - Runs entirely in the browser: no uploads, free, no signup How it's built An .xlsx file is a zip archive with a few XML parts. Instead of SheetJS, the tool uses a small hand-written XLSX writer and zips the parts with fflate, with no new dependencies: import { strToU8, zipSync } from 'fflate'; files['xl/styles.xml'] = strToU8(styles.toXml()); files['xl/sharedStrings.xml'] = strToU8(strings.toXml()); return zipSync(files, { level: 6 });

Client-Side PDF Tools in JavaScript: Merge, Compress and Convert Without Uploading Most "free online PDF tools" work the same way: you upload a contract, a bank statement or a scan of your passport, a server does something to it, and you download the result. That's a lot of trust for merging two files. We recently built a set of 18 free PDF tools that do the opposite: the file is read, changed and saved on your device, and its contents never reach a server. The browser can do most of this with two open-source libraries and a <canvas>. Here's how the pieces fit together, with the code that matters and the limits we ran into. The building blocks - pdf-lib edits the structure of a PDF: copy pages between documents, rotate them, read and write metadata, draw text and images. It doesn't render anything. - PDF.js is Firefox's PDF renderer. It turns pages into pixels and gives you the text layer. - The canvas decodes and re-encodes images, which is what you need for PDF to image and for compression. Everything starts the same way: a file input, turned into bytes. const input = document.querySelector('input[type="file"]'); input.addEventListener("change", async () => { const files = await Promise.all( [...input.files].map(async (file) => new Uint8Array(await file.arrayBuffer())), ); // files is an array of Uint8Array, nothing has left the device }); And everything ends the same way: bytes, turned back into a download. function download(bytes, name, type = "application/pdf") { const url = URL.createObjectURL(new Blob([bytes], { type })); const link = Object.assign(document.createElement("a"), { href: url, download: name }); link.click(); setTimeout(() => URL.revokeObjectURL(url), 1000); } Structural edits with pdf-lib Merging is the classic example. You create an empty document and copy every page of every source into it: import { PDFDocument } from "pdf-lib"; export async function mergePdfs(files) { const output = await PDFDocument.create(); for (const bytes of files) { const source = await PDFDocument.load(bytes, { updateMetadata: false }); const pages = await output.copyPages(source, source.getPageIndices()); pages.forEach((page) => output.addPage(page)); } return output.save({ useObjectStreams: true }); } Split and extract are the same function with a different list of page indices. Two details are worth copying: - updateMetadata: false stops pdf-lib from rewriting the producer and dates of the file you just opened. Set them deliberately when you save instead. - Encrypted PDFs throw on load. Catch that and say so in plain words, otherwise the user sees a stack trace for a file that opens fine in their viewer. Rotation has one trap: pages can already carry a rotation, so you add to it instead of overwriting it. import { degrees } from "pdf-lib"; function rotatePage(page, angle) { const current = page.getRotation().angle; const next = (((Math.round((current + angle) / 90) * 90) % 360) + 360) % 360; page.setRotation(degrees(next)); } Without that, rotating a scanned page that was already stored sideways turns it the wrong way, and you get bug reports you can't reproduce with your own test files. Rendering pages with PDF.js PDF to PNG or JPEG means rendering each page onto a canvas. PDF.js does the hard part, but its setup decides whether it works under a strict Content Security Policy. PDF.js parses documents in a web worker and fetches extra data at runtime: character maps for CJK text, the standard fonts, and in recent versions WebAssembly image decoders and ICC color profiles. We self-host all of them next to the worker, so the CSP can stay at worker-src 'self' and no third-party origin ever sees a request. import * as pdfjs from "pdfjs-dist"; const assets = `/vendor/pdfjs/${pdfjs.version}`; pdfjs.GlobalWorkerOptions.workerSrc = `${assets}/pdf.worker.min.mjs`; export async function openPdf(bytes) { return pdfjs.getDocument({ // PDF.js transfers the buffer to its worker, so pass a copy and keep the original usable. data: bytes.slice(), cMapUrl: `${assets}/cmaps/`, cMapPacked: true, standardFontDataUrl: `${assets}/standard_fonts/`, }).promise; } The bytes.slice() matters. Without it, the original array is detached after the first call, and any second operation on the same file fails with an empty buffer. Rendering itself is a matter of scale. PDF coordinates are points, 72 per inch, so 150 dpi means a scale of 150 / 72: const MAX_PIXELS = 16_000_000; // Safari refuses canvases much larger than this export async function pageToImage(pdf, pageNumber, { dpi = 150, type = "image/png" } = {}) { const page = await pdf.getPage(pageNumber); const unit = page.getViewport({ scale: 1 }); let scale = dpi / 72; const pixels = unit.width * unit.height * scale * scale; if (pixels > MAX_PIXELS) scale *= Math.sqrt(MAX_PIXELS / pixels); const viewport = page.getViewport({ scale }); const canvas = document.createElement("canvas"); canvas.width = Math.floor(viewport.width); canvas.height = Math.floor(viewport.height); await page.render({ canvas, viewport, // JPEG has no transparency: paint white behind pages without a background. background: type === "image/jpeg" ? "#ffffff" : undefined, }).promise; const blob = await new Promise((resolve) => canvas.toBlob(resolve, type, 0.9)); page.cleanup(); canvas.width = canvas.height = 0; // release the canvas memory right away return blob; } The pixel cap matters more than it looks. A large-format drawing at 300 dpi easily exceeds what Safari will allocate, and the render fails or comes back blank instead of throwing a useful error. Capping the area and scaling down keeps the output useful on every browser. Compression without a server Compression sounds like it needs a server. It doesn't, as long as you only touch images. Most of the weight in a typical oversized PDF is photos and scans, so our compressor leaves text, vector graphics and fonts alone and only touches embedded images: 1. Walk the document's objects with pdf-lib and collect the image streams. Images under 256 pixels on their longest side aren't worth the effort and stay as they are. 2. Decode them. JPEGs go straight to the canvas. Flate-compressed raw images are inflated with fflate, and PNG predictors are undone by hand. 3. Draw the pixels onto a canvas, scaled so the longest side stays below a limit, and export a JPEG at lower quality. 4. Keep the new version only if it's at least 10% smaller. Otherwise the original stays. Then set the stream's filter and dimensions and save with object streams. We offer three levels. The strongest keeps the longest image side at 1,200 pixels and JPEG quality 0.6, which is roughly 150 dpi on a full A4 page. The lightest keeps 2,600 pixels at 0.85. Some images are better left alone than rewritten badly: CMYK, indexed colors and 16-bit samples would change color on the way through a canvas, so we skip them. Images with a soft mask, the PDF way of storing transparency, get recompressed but keep their pixel size, so image and mask still line up. At the end the tool reports how many images it found and how many it actually recompressed. An honest "we saved 4%" beats a smaller file that looks wrong. HTML to PDF: the one thing a browser can only print A browser has exactly one way to turn HTML into a PDF: print it. So the HTML to PDF tool renders your markup in a sandboxed iframe and opens the print dialog, where you pick "Save as PDF". Page size and margins come from CSS: @page { size: A4; margin: 15mm; } @media print { .no-print { display: none; } } In Chrome and Edge the result comes from Chromium's layout engine, so it's faithful to modern CSS. What you can't do from a web page is skip the dialog, control the browser's own headers and footers reliably, or wait for remote fonts and scripts the way a headless browser can. That's the point where client-side stops and you need Chromium running on a server. Write it once, test it in Node One design choice saved us a lot of time: every operation is a plain function from bytes to bytes, and anything that needs a canvas gets one passed in. interface RasterEnvironment<Canvas, Output> { createCanvas(width: number, height: number): Canvas; encode(canvas: Canvas, type: "png" | "jpeg", quality?: number): Promise<Output>; } In the browser, the environment creates DOM canvases and calls toBlob. In the test suite, the same functions run in Node with a native canvas package. Merge, split, rotate, compress and PDF to image are all covered by unit tests, without spinning up a browser for every case. Where the browser runs out Client-side isn't free of trade-offs: - Memory is the user's. Files of several hundred megabytes depend on how much memory the tab gets. When a file fails to open, closing other tabs often helps. - Rendering is PDF.js, not Acrobat. Unusual font encodings and some blend modes can look slightly different from desktop viewers. XFA forms we turn off entirely. - Text you add needs fonts. pdf-lib's built-in standard fonts only cover Latin characters. Watermarks or page numbers in other scripts need an embedded font, and failing with a clear message beats drawing empty boxes. - No automation. A tool in a tab is great for one file and useless for ten thousand a night. That's what a server-side API is for. Takeaways - Read files with arrayBuffer(), hand back results with a Blob URL, and nothing ever needs to be uploaded. - pdf-lib covers structure, PDF.js covers pixels and text, and the canvas covers image encoding. - Self-host the PDF.js worker and its assets, pass it a copy of your bytes, and cap canvas sizes. - Keep operations as pure byte-in, byte-out functions so they run in Node for tests. If you've built something similar, which PDF edge cases bit you?