Trimming video in the browser with FFmpeg.wasm: 6 things I had to figure out
In my previous post I wrote about converting images to JPG with Canvas. This time it's video: a trim tool and a video to MP3 converter for CompressPhoto. Both run FFmpeg compiled to WebAssembly, right in the browser tab. The file never goes to a server.
Getting a demo working with ffmpeg.wasm takes ten minutes. Making it usable for real people took longer. Here's what I ran into.
1. Use the single-threaded core
ffmpeg.wasm has a multi-threaded core, and it's faster. But it needs SharedArrayBuffer. That means serving the page with two headers:
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
Those headers isolate the page. On my site they would break Google sign-in, which uses a popup, and third-party scripts like analytics. So I use the single-threaded @ffmpeg/core. It's slower, but it works on any page without special headers.
2. Host the 32 MB core yourself, and show real progress
The WASM file is about 32 MB. I don't load it from a CDN. A build script copies it from node_modules into public/ffmpeg/<version>/. Because the version is in the path, the file can be cached forever.
32 MB is a long wait on a phone, so users need a progress bar. Here's the catch: the server sends the file compressed. Content-Length is the compressed size, so loaded / total is wrong. I hard-code the uncompressed size and stream the download myself:
const CORE_WASM_BYTES = 32_232_419; // size of ffmpeg-core.wasm 0.12.10, uncompressed
async function fetchWasm(onProgress) {
const res = await fetch(`/ffmpeg/0.12.10/ffmpeg-core.wasm`);
const reader = res.body.getReader();
const chunks = [];
let loaded = 0;
for (;;) {
const { done, value } = await reader.read();
if (done) break;
chunks.push(value);
loaded += value.length;
onProgress(Math.min(0.99, loaded / CORE_WASM_BYTES));
}
return URL.createObjectURL(new Blob(chunks, { type: "application/wasm" }));
}
await ffmpeg.load({ coreURL, wasmURL: await fetchWasm(setProgress) });
A test checks that the core version in the code matches package.json. So an upgrade fails the build until I update the path and this size together.
3. Don't copy the video into memory: mount it
Most examples do this:
await ffmpeg.writeFile("input.mp4", await fetchFile(file));
That copies the whole video into WebAssembly memory. A 700 MB phone video plus the output can crash the tab.
ffmpeg.wasm can mount the File with WORKERFS instead. FFmpeg then reads it in chunks, straight from disk:
// WORKERFS exposes the file under its own name, so give it a safe one.
// new File([file], ...) doesn't copy the data: it's the same Blob.
const input = new File([file], "source.mp4", { type: file.type });
await ffmpeg.createDir("/input").catch(() => {});
await ffmpeg.mount("WORKERFS", { files: [input] }, "/input");
await ffmpeg.exec(["-i", "/input/source.mp4", /* ... */ "output.mp4"]);
const data = await ffmpeg.readFile("output.mp4");
await ffmpeg.unmount("/input");
Only the result lives in memory now. That's why the trim tool accepts files up to 1 GB, far more than I could allow for server-side tools.
The safe file name matters too. Real file names contain spaces, quotes and emoji. You don't want them anywhere near FFmpeg arguments.
4. Offer two trim modes: fast and precise
The fast mode copies the streams without re-encoding:
["-ss", start, "-i", input, "-t", duration,
"-map", "0:v:0?", "-map", "0:a:0?",
"-c", "copy", "-avoid_negative_ts", "make_zero",
"-movflags", "+faststart", "output.mp4"]
It finishes in seconds even on a phone. The catch: with stream copy, the cut can only happen on a keyframe. The clip may start a little earlier than the user asked.
The precise mode re-encodes to H.264 and AAC:
["-ss", start, "-i", input, "-t", duration,
"-vf", "scale=trunc(iw/2)*2:trunc(ih/2)*2",
"-c:v", "libx264", "-preset", "veryfast", "-crf", "20",
"-pix_fmt", "yuv420p",
"-c:a", "aac", "-b:a", "160k",
"-movflags", "+faststart", "output.mp4"]
Two details are worth noting. H.264 with yuv420p needs even width and height, so the scale filter rounds them down. Some phone videos have odd dimensions, and without the filter the encode just fails.
Also, in fast mode I keep the original container. Stream copy keeps the original codecs, and not every container accepts every codec. For example, AMR audio from an old 3GP file won't fit into MP4.
The ? in -map 0:a:0? makes the audio track optional. Without it, a silent screen recording fails with "matches no streams".
5. Progress, cancel and errors all come from the log
exec() doesn't report progress, but FFmpeg's log does. I read Duration: once and then every time= line:
ffmpeg.on("log", ({ message }) => {
const t = /time=\s*(\d+):(\d{2}):(\d{2}(?:\.\d+)?)/.exec(message);
if (t && total) onProgress((+t[1] * 3600 + +t[2] * 60 + +t[3]) / total);
});
For files the browser can't preview, like MKV or some HEVC videos, I get the duration by running ffmpeg -i input with no output. FFmpeg prints the stream info and exits with an error. That's expected, and the log is all I need.
Cancelling is blunt. exec() can't stop halfway, so on abort I terminate the worker. The next task creates a new one. The core is already in the HTTP cache, so restarting is quick.
Errors are classified from the log too. "moov atom not found" means a broken or incomplete file. "Cannot enlarge memory" means the tab ran out of memory. Each case gets its own message instead of "something went wrong".
6. Run one job at a time
The single-threaded FFmpeg instance can't run two commands at once. If a user starts a second file, it waits in a simple promise chain:
let chain = Promise.resolve();
function run(job) {
const task = chain.then(job);
chain = task.catch(() => {}); // the next job waits even if this one failed
return task;
}
Was it worth it?
For trimming and extracting audio, yes. The fast trim is instant, there's no server bill, and the privacy story is simple: the file stays on your device.
For heavy re-encoding it's a different story. Single-threaded WASM is several times slower than native FFmpeg, and on a phone a long precise trim takes a while. That's why the precise mode is opt-in.
You can try the result here: Trim video online and Video to MP3.
Have you shipped ffmpeg.wasm in production? I'd like to know how you handle memory on phones.
1 comment
Mentions
Trimming video in the browser with FFmpeg.wasm: 6 things I had to figure out In my previous post I wrote about converting images to JPG with Canvas. This time it's video: a trim tool and a video to MP3 converter for CompressPhoto. Both run FFmpeg compiled to WebAssembly, right in the browser tab. The file never goes to a server. Getting a demo working with ffmpeg.wasm takes ten minutes. Making it usable for real people took longer. Here's what I ran into. 1. Use the single-threaded core ffmpeg.wasm has a multi-threaded core, and it's faster. But it needs SharedArrayBuffer. That means serving the page with two headers: Cross-Origin-Opener-Policy: same-origin Cross-Origin-Embedder-Policy: require-corp Those headers isolate the page. On my site they would break Google sign-in, which uses a popup, and third-party scripts like analytics. So I use the single-threaded @ffmpeg/core. It's slower, but it works on any page without special headers. 2. Host the 32 MB core yourself, and show real progress The WASM file is about 32 MB. I don't load it from a CDN. A build script copies it from node_modules into public/ffmpeg/<version>/. Because the version is in the path, the file can be cached forever. 32 MB is a long wait on a phone, so users need a progress bar. Here's the catch: the server sends the file compressed. Content-Length is the compressed size, so loaded / total is wrong. I hard-code the uncompressed size and stream the download myself: const CORE_WASM_BYTES = 32_232_419; // size of ffmpeg-core.wasm 0.12.10, uncompressed async function fetchWasm(onProgress) { const res = await fetch(`/ffmpeg/0.12.10/ffmpeg-core.wasm`); const reader = res.body.getReader(); const chunks = []; let loaded = 0; for (;;) { const { done, value } = await reader.read(); if (done) break; chunks.push(value); loaded += value.length; onProgress(Math.min(0.99, loaded / CORE_WASM_BYTES)); } return URL.createObjectURL(new Blob(chunks, { type: "application/wasm" })); } await ffmpeg.load({ coreURL, wasmURL: await fetchWasm(setProgress) }); A test checks that the core version in the code matches package.json. So an upgrade fails the build until I update the path and this size together. 3. Don't copy the video into memory: mount it Most examples do this: await ffmpeg.writeFile("input.mp4", await fetchFile(file)); That copies the whole video into WebAssembly memory. A 700 MB phone video plus the output can crash the tab. ffmpeg.wasm can mount the File with WORKERFS instead. FFmpeg then reads it in chunks, straight from disk: // WORKERFS exposes the file under its own name, so give it a safe one. // new File([file], ...) doesn't copy the data: it's the same Blob. const input = new File([file], "source.mp4", { type: file.type }); await ffmpeg.createDir("/input").catch(() => {}); await ffmpeg.mount("WORKERFS", { files: [input] }, "/input"); await ffmpeg.exec(["-i", "/input/source.mp4", /* ... */ "output.mp4"]); const data = await ffmpeg.readFile("output.mp4"); await ffmpeg.unmount("/input"); Only the result lives in memory now. That's why the trim tool accepts files up to 1 GB, far more than I could allow for server-side tools. The safe file name matters too. Real file names contain spaces, quotes and emoji. You don't want them anywhere near FFmpeg arguments. 4. Offer two trim modes: fast and precise The fast mode copies the streams without re-encoding: ["-ss", start, "-i", input, "-t", duration, "-map", "0:v:0?", "-map", "0:a:0?", "-c", "copy", "-avoid_negative_ts", "make_zero", "-movflags", "+faststart", "output.mp4"] It finishes in seconds even on a phone. The catch: with stream copy, the cut can only happen on a keyframe. The clip may start a little earlier than the user asked. The precise mode re-encodes to H.264 and AAC: ["-ss", start, "-i", input, "-t", duration, "-vf", "scale=trunc(iw/2)*2:trunc(ih/2)*2", "-c:v", "libx264", "-preset", "veryfast", "-crf", "20", "-pix_fmt", "yuv420p", "-c:a", "aac", "-b:a", "160k", "-movflags", "+faststart", "output.mp4"] Two details are worth noting. H.264 with yuv420p needs even width and height, so the scale filter rounds them down. Some phone videos have odd dimensions, and without the filter the encode just fails. Also, in fast mode I keep the original container. Stream copy keeps the original codecs, and not every container accepts every codec. For example, AMR audio from an old 3GP file won't fit into MP4. The ? in -map 0:a:0? makes the audio track optional. Without it, a silent screen recording fails with "matches no streams". 5. Progress, cancel and errors all come from the log exec() doesn't report progress, but FFmpeg's log does. I read Duration: once and then every time= line: ffmpeg.on("log", ({ message }) => { const t = /time=\s*(\d+):(\d{2}):(\d{2}(?:\.\d+)?)/.exec(message); if (t && total) onProgress((+t[1] * 3600 + +t[2] * 60 + +t[3]) / total); }); For files the browser can't preview, like MKV or some HEVC videos, I get the duration by running ffmpeg -i input with no output. FFmpeg prints the stream info and exits with an error. That's expected, and the log is all I need. Cancelling is blunt. exec() can't stop halfway, so on abort I terminate the worker. The next task creates a new one. The core is already in the HTTP cache, so restarting is quick. Errors are classified from the log too. "moov atom not found" means a broken or incomplete file. "Cannot enlarge memory" means the tab ran out of memory. Each case gets its own message instead of "something went wrong". 6. Run one job at a time The single-threaded FFmpeg instance can't run two commands at once. If a user starts a second file, it waits in a simple promise chain: let chain = Promise.resolve(); function run(job) { const task = chain.then(job); chain = task.catch(() => {}); // the next job waits even if this one failed return task; } Was it worth it? For trimming and extracting audio, yes. The fast trim is instant, there's no server bill, and the privacy story is simple: the file stays on your device. For heavy re-encoding it's a different story. Single-threaded WASM is several times slower than native FFmpeg, and on a phone a long precise trim takes a while. That's why the precise mode is opt-in. You can try the result here: Trim video online and Video to MP3. Have you shipped ffmpeg.wasm in production? I'd like to know how you handle memory on phones.