My PR demo videos make themselves now, in my voice
How I built a free pipeline that runs on my MacBook: it films the real app, narrates in a clone of my voice and can put a lip-synced webcam tile of me in the corner. Plus the benchmarks, the bugs, and the 23-minute render that nearly killed the whole idea.
!A script draws on the Annotator website while a small webcam tile of me talks in the bottom-left corner.
Nothing in this clip was recorded live. A script moves the cursor, the voice is a clone of mine, and the tile in the corner is a two-minute webcam take of me with new lips. Full 76-second video.
TL;DR
- Every PR in my projects ships with a one-to-three-minute video of the real app, narrated in my voice. I don't record them anymore.
- I'm cheap, so it's all free and runs locally on one MacBook Pro: a voice clone (Voicebox), a lip-synced webcam tile (MuseTalk), and a screen recording driven through macOS accessibility APIs.
- The avatar used to take 23 minutes to render one minute of video. It now takes about one.
- Most of the time went into unglamorous bugs: pauses in the audio, a toolbar that hides itself after three seconds, and my Hebrew keyboard layout.
Why I wanted this
Most of my PRs are written with a coding agent, usually on autopilot. A demo video in the PR makes it much easier to understand what the author and their agent were going for. Bigger PRs stop being scary, and you can see the before and after for yourself instead of picturing it from the diff.
Before AI coding, when you didn't understand someone's change, you'd call them and ask them to show you. A demo video does exactly that, without the call.
Recording them myself didn't scale. With agents writing the code, sometimes I can't keep up with my own PRs. On top of that I had to prepare the environment, record, edit, add zooms and do it all again after every fix. Now the agent does that part.
The obvious answer was a hosted avatar product like HeyGen. It would look better than anything in this post. But I'm cheap, and I didn't want to pay for it. So I set four rules:
- Free. No subscriptions, no credits.
- Local. My face and voice don't go to anyone's servers.
- Mac only. My work machine is a MacBook Pro with an M4 Max and 36 GB of memory. There's no NVIDIA GPU, so anything that needs CUDA was out.
- The real app. No mock-ups and no clips cut from a marketing site. If the video shows a click, the app got clicked.
I didn't write most of the code. I worked with a coding agent (GitHub Copilot) that had computer use on my Mac. It installed the models, ran the benchmarks, drove the app and showed me what came out. My job was to watch and say what was wrong. A lot of this post is that list.
!Pipeline: a script goes to the voice clone and to the screen take; the voice drives the avatar and gives timing cues to the screen take; the screen take and avatar are composed into the final demo video.
One script in, one video out. The voice's word timings tell the screen recording when to act.
Part 1: The face
Hosted tools hit a wall fast
Hosted avatar products like HeyGen look great, and their free plans are small. When I checked, HeyGen's free plan gave you three one-minute videos a month with a watermark. They also need your face on their servers. That broke two of my rules on day one, so I looked for models that run on a laptop.
Three lip-sync models, one clip, one scoreboard
Lip-sync means you start from a real video of yourself and regenerate only the mouth to match new audio. Your face, your room and your head movement stay real. I picked three models that can run on Apple Silicon and gave each the same 17-second clip of me (cut from a tech talk I'd given) and the same voice track.
I didn't want to judge by eye alone, so the agent scored them:
- SyncNet confidence: a model trained to spot out-of-sync video rates how sure it is that lips and audio match. Higher is better.
- Offset: how many frames the lips run ahead of or behind the voice.
- Still looks like me: face-recognition similarity to the original. 1.0 means identical.
| Model | SyncNet confidence | Offset | Still looks like me | Time for the 17 s clip |
| --- | --- | --- | --- | --- |
| No lip-sync (baseline) | 0.38 | 11 frames | 1.00 | none |
| MuseTalk 1.5 | 2.66 | 1 frame | 0.92 | 96 s |
| FaceFusion (EDTalk) | 4.65 | 3 frames (120 ms) | 0.95 | 92 s, CPU only |
| LatentSync 1.5 | 6.55 | 0 | 0.97 | about 9 min |
!The same clip side by side: no lip-sync, MuseTalk 1.5, FaceFusion EDTalk and LatentSync 1.5, with mouth close-ups and scores under each.
Same clip, same voice, three models. (It's a GIF, so no sound.)
The numbers matched what I saw. This chart made it obvious:
!Four rows, one per model. Blue shows voice loudness, red shows how open the mouth is. With no lip-sync and with MuseTalk the red line is almost flat; FaceFusion twitches; LatentSync opens and closes along with the voice and closes on p, b and m sounds.
Red is how far my mouth opens, blue is how loud the voice is. Dotted lines mark words starting with p, b or m, where lips have to close.
LatentSync won. Its newer 1.6 version works at 512 px, but one attention step alone needed about 8.6 GB and my Mac started swapping, so I settled on 1.5 at 256 px with a few small patches to run on Apple's GPU.
Recording the source take
The model needs footage of you to put new lips on. The agent built me a small recording page with a hands-free teleprompter: a neutral pause, a greeting, a bit of explaining, 30 seconds of just listening, and a sign-off. Each part gives the avatar different raw material to draw from.
!The recording kit page: three shirts, about three minutes each, with a setup checklist.
I recorded two takes in two shirts. Then it turned out the microphone had recorded pure noise, because QuickTime had picked the wrong input. It didn't matter. The model replaces the mouth anyway, and for timing the agent read my lip movement straight from the video with face landmarks instead of from the audio.
Getting the head to move with the voice
Good lips weren't enough. My head would nod during a pause in the new voice and sit still in the middle of a sentence, which looks like a bad dub.
The fix was to nudge the source footage's speed a little (between 0.85× and 1.17×) and line up pauses in the new voice with moments where I'd paused on camera. One number tracks it: the correlation between voice loudness and mouth activity. It went from −0.08 without the nudge to 0.40 with it.
Feedback that shaped the look
- Circle to webcam tile. A round bubble looked like a sticker. A rounded 16:9 tile showing head, shoulders and a bit of room looks like someone on a call.
- Colour. My camera's white balance turned a black T-shirt green. The fix is baked into the render.
- No nodding. Nodding while nobody is talking reads as agreeing with no one. The avatar only uses the calmest stretch of the take.
- Smaller, twice. The tile went from 352 px to about 260 px wide on a 1920 px video.
Part 2: The voice
Voicebox is an open-source app that clones a voice from a short sample and runs it locally. My first clone came from a meeting recording and sounded flat, a bit robotic. My second came from four lively bits of a tech talk where I was presenting, and the difference was huge. That's the one I use.
What I learned about getting a cloned voice to sound like me on a good day:
- Style instructions don't work on a cloned voice. "Say this enthusiastically" did nothing. Livelier lines did: a couple of exclamation marks, shorter sentences.
- Generate the whole script in one go. Line by line, the energy jumped around: an excited first line, a sleepy third one.
- Generate several takes and measure them. The agent makes six takes and keeps the one that matches the script word for word (checked with a transcript), has the most movement in pitch and loudness, and drops in pitch on the last word so the ending sounds finished.
- Read every transcript. One take said "and then do anything" instead of "undo anything". That one sentence got swapped in from another take.
- Don't respell names. Spelling "Annotator" phonetically to help the model backfired: it came out sounding French. The plain spelling plus picking the best take worked better.
Three bugs hiding in the pauses
The screen needs room between sentences, so the voice gets short pauses. Those pauses caused three bugs in a row.
Bug 1: digital silence. The first version inserted silence between sentences. My feedback: "the hard pauses are too snappy". The sound dropped from my voice to absolute zero, and real rooms are never that quiet.
Bug 2: the stuck record. The next version filled the gaps by looping a slice of the quiet bit between words. "It sounds snappy, repeated, like a stuck disk." Short loops are easy to hear.
Bug 3: clipped words. Sentences were cut at the word edges a speech-to-text model (Whisper) gave us. In flowing speech those edges drift by about 0.3 seconds, so word endings got chopped.
!Two audio level plots around the end of a sentence. Before: the level falls off a cliff to digital silence. After: it settles to a steady low room tone, then rises into the next word.
The version that works cuts at the quietest point measured in the audio itself, fills gaps with smooth, non-repeating noise at about −66 dB, and never loops. An automated check now flags any drop of more than 30 dB within 15 ms anywhere in the track.
Part 3: The screen
Why it has to be the real app
An early version stitched together clips from the product website. I rejected it: it had to be a recording of the installed app. The website shows what the app should do. A recording shows that it does.
How the computer drives itself
- A separate screen. DeskPad creates a virtual 1920×1200 display. The recording happens there, so my real screens stay mine, and ffmpeg records only that display.
- Buttons by name. A small Swift tool posts mouse and keyboard events. It finds controls through the macOS accessibility API by their names ("Pen", "Arrow", "Laser pointer") instead of pixel positions, so a moved button doesn't break the take.
- A cursor that moves like a hand. Moves take longer for far or small targets (Fitts's law), overshoot slightly and follow a curve.
- Timing from the voice. The voice comes with word timings, so the plan can say "draw the box when I say box":
at 16.20 # a beat before the word "box"
tool Pen
path 0.8 -1528 512 -1450 506 -1360 505 ...
mark box # log when it really happened
Every action logs when it really happened. If any lands more than about half a second from its word, the agent records another take. The first version took about 15 takes.
!Six frames from one take, each captioned with the words being spoken: a messy box around the download button, a curvy arrow and a "Start here!" label, a laser loop around the Applications icon, numbered steps, the arrow turning blue, and the page scrolled back to the top.
Six moments from one take, with what the voice is saying at each.
What broke
| What I saw | Why | Fix |
| --- | --- | --- |
| Stray "4" and "5" counters later in the video | The toolbar hides itself after 3 seconds. A click meant for it hit the page while the counter tool was still on. | Check the button is still there right before clicking. Switch back to Select straight after the last counter. |
| Colour keys did nothing, the arrow stayed blue | My keyboard was on Hebrew, and the app's shortcuts read the typed character. | The recorder won't start unless the layout is English. |
| Every click missed | I'd rearranged my monitors overnight. | A setup script finds the virtual display and shifts every coordinate. |
| The laser pointer looked robotic | Constant speed between points makes sharp corners. | About 120 points along an eased curve, with equal time per point. |
| Things changed like magic | Fast moments use keyboard shortcuts, which you can't see. | Show the keys on screen as keycaps. |
| Undo looked like a glitch | Undoing a move made the arrow snap back. | Undo a throwaway scribble instead. |
| The label landed 2 seconds late | A trip to the toolbar costs about 2 seconds. | Use the keyboard shortcut. |
!Four keycaps: control + 2 "close", ⌘ + Z "undo" (pressed), L "laser" and B "blue".
Keycaps appear at the right edge when a shortcut is pressed, so nothing changes by magic.
A few smaller touches came from watching drafts. A card with the URL slides up when the voice says it. Captions come from the voice's word timings. The video ends on the website's promo reel, synced through Chrome's DevTools Protocol, so the last shot lands on "Draw over any app."
Part 4: The 23-minute problem
LatentSync's quality came at a price. It runs 20 diffusion steps per frame, and one minute of video took 23 minutes to render. I said it plainly at the time: "Even 23 minutes for 1 minute is too much. Sometimes we render 10 minutes and I don't want to wait 5 hours."
So I went back to MuseTalk. It had the weakest lips in the benchmark, but it makes each frame in one network pass instead of 20 steps. Profiling it showed most of the time wasn't in the model at all: turning its output back into pixels took about 70% of GPU time. Three changes:
1. Swap the full image decoder for TAESD, a tiny decoder built for the same kind of output.
2. Prepare the face once per take (landmarks, encoded frames, blend masks) and cache it.
3. Write frames straight into the video encoder in a single pass, with no files in between.
!Bar chart of minutes of compute to render one minute of avatar video: LatentSync 1.5 23.5 minutes, MuseTalk 1.5 with the stock decoder 2.6 minutes, MuseTalk 1.5 with the tiny decoder and cache 1 minute.
| Setup | Minutes to render 1 minute of video |
| --- | --- |
| LatentSync 1.5, 20 diffusion steps | 23.5 |
| MuseTalk 1.5, stock decoder | 2.6 |
| MuseTalk 1.5, tiny decoder and cache | 1.0 |
A 59-second video now renders in 61 seconds.
!Left: LatentSync, 23 minutes to render this minute. Right: fast MuseTalk, 61 seconds. Below each, the same video at the real tile size.
Old renderer on the left, new on the right, the same ten seconds at full size and at tile size. (No sound in the GIF.)
At full size, the fast version's lips are softer. At the size of the tile, nobody noticed. The slow model is still there behind a --quality best flag.
In the benchmark I judged the models full screen. The product is a 260-pixel tile. If I'd judged at the size people actually see, I'd have picked the fast model a week earlier.
Part 5: How I use it on PRs
The pipeline exists for pull requests. These are the rules I settled on, most of them from me watching a video and saying "no, not like that":
- Every PR gets a demo in its description, without anyone asking. Stacked PRs each get their own, showing only their own changes.
- Open on the app. No title slide, no agenda and no "Hey". End on a plain "Thanks".
- Show, then explain. The real app in the browser, the cursor working, my voice explaining as it goes, like showing a colleague.
- One short zoom-out, only when it's needed. For things you can't see in the UI: where new data is saved, who can read it, a new API route or background job. Taken from the code, not from memory.
- Before and Now side by side for anything visible. The agent runs the base branch from a second worktree on another port, with the same demo data, the same clicks and the same pace.
- Plain, casual English. Every script goes through a no-AI-slop pass before the voice is generated.
- Demo data only. A made-up demo user, never real client data, and nothing from the recording gets committed.
- Short and light. About a minute per main change and two to four minutes in total, at −16 LUFS loudness and under 90 MB so GitHub plays it inline.
The PR description stays short. My words to the agent were roughly "only the intent, what's necessary for a human to read, and the visuals":
Two or three lines: what changed and why.
Figma: <link>
**Demo**
<video>
| | Before | After |
| -------------- | --------- | --------- |
| Template editor | <image> | <image> |
| PDF export | <image> | <image> |
**Notes for reviewers**
- the one or two things worth a closer look
It isn't free in tokens. A full demo cost the agent 28 to 99 million tokens (about 59 million on average) and 44 to 128 minutes. An updated screenshot costs about 2 million tokens and five minutes. So after a review fix, the default is a new screenshot in the Before | After table. The agent re-records one beat only when a fix changes what the video shows, and the whole video only when I ask. Recording, voice and rendering also load the CPU and GPU, so only one demo runs on my machine at a time.
The avatar is off by default for PR demos. My voice stays on. A face on screen means something when you're showing stakeholders, not when a reviewer is checking a change. Leaving it off also saves the processing power when it isn't needed. The same skill makes complete demo videos too, and that's where the avatar comes in: sprint demos and public videos like the Annotator one above, where a human face is worth it.
What I'd do differently
- Judge at the size people watch. It would have saved the whole LatentSync detour.
- Record the source take with a working mic. Clean audio from the same session would have made a better voice clone than an old meeting recording.
- Assume the UI will move. Toolbars hide, keyboards switch layouts and monitors get rearranged. Check the state before every step instead of trusting coordinates.
I still have an open question. The avatar loops back and forth over the same calm stretch of footage, which is fine for two minutes, but I don't have a good answer yet for natural head movement in a 10-minute video. If you've solved that on a laptop, I'd love to hear how.
Try it
Everything here is free and runs on a Mac with Apple Silicon:
- Voice: Voicebox (Qwen3-TTS 1.7B).
- Lip-sync: MuseTalk 1.5 with the TAESD decoder. LatentSync if you have the time.
- Recording: DeskPad for a virtual display, ffmpeg to record it, and macOS accessibility APIs to drive the app.
- The app in the demo: Annotator, a free screen-drawing app for Mac and Windows.
Here's the full Annotator video from the top of the post:
{% embed youtu.be %}
Mentions
My PR demo videos make themselves now, in my voice How I built a free pipeline that runs on my MacBook: it films the real app, narrates in a clone of my voice and can put a lip-synced webcam tile of me in the corner. Plus the benchmarks, the bugs, and the 23-minute render that nearly killed the whole idea. !A script draws on the Annotator website while a small webcam tile of me talks in the bottom-left corner. Nothing in this clip was recorded live. A script moves the cursor, the voice is a clone of mine, and the tile in the corner is a two-minute webcam take of me with new lips. Full 76-second video. TL;DR - Every PR in my projects ships with a one-to-three-minute video of the real app, narrated in my voice. I don't record them anymore. - I'm cheap, so it's all free and runs locally on one MacBook Pro: a voice clone (Voicebox), a lip-synced webcam tile (MuseTalk), and a screen recording driven through macOS accessibility APIs. - The avatar used to take 23 minutes to render one minute of video. It now takes about one. - Most of the time went into unglamorous bugs: pauses in the audio, a toolbar that hides itself after three seconds, and my Hebrew keyboard layout. Why I wanted this Most of my PRs are written with a coding agent, usually on autopilot. A demo video in the PR makes it much easier to understand what the author and their agent were going for. Bigger PRs stop being scary, and you can see the before and after for yourself instead of picturing it from the diff. Before AI coding, when you didn't understand someone's change, you'd call them and ask them to show you. A demo video does exactly that, without the call. Recording them myself didn't scale. With agents writing the code, sometimes I can't keep up with my own PRs. On top of that I had to prepare the environment, record, edit, add zooms and do it all again after every fix. Now the agent does that part. The obvious answer was a hosted avatar product like HeyGen. It would look better than anything in this post. But I'm cheap, and I didn't want to pay for it. So I set four rules: - Free. No subscriptions, no credits. - Local. My face and voice don't go to anyone's servers. - Mac only. My work machine is a MacBook Pro with an M4 Max and 36 GB of memory. There's no NVIDIA GPU, so anything that needs CUDA was out. - The real app. No mock-ups and no clips cut from a marketing site. If the video shows a click, the app got clicked. I didn't write most of the code. I worked with a coding agent (GitHub Copilot) that had computer use on my Mac. It installed the models, ran the benchmarks, drove the app and showed me what came out. My job was to watch and say what was wrong. A lot of this post is that list. !Pipeline: a script goes to the voice clone and to the screen take; the voice drives the avatar and gives timing cues to the screen take; the screen take and avatar are composed into the final demo video. One script in, one video out. The voice's word timings tell the screen recording when to act. Part 1: The face Hosted tools hit a wall fast Hosted avatar products like HeyGen look great, and their free plans are small. When I checked, HeyGen's free plan gave you three one-minute videos a month with a watermark. They also need your face on their servers. That broke two of my rules on day one, so I looked for models that run on a laptop. Three lip-sync models, one clip, one scoreboard Lip-sync means you start from a real video of yourself and regenerate only the mouth to match new audio. Your face, your room and your head movement stay real. I picked three models that can run on Apple Silicon and gave each the same 17-second clip of me (cut from a tech talk I'd given) and the same voice track. I didn't want to judge by eye alone, so the agent scored them: - SyncNet confidence: a model trained to spot out-of-sync video rates how sure it is that lips and audio match. Higher is better. - Offset: how many frames the lips run ahead of or behind the voice. - Still looks like me: face-recognition similarity to the original. 1.0 means identical. | Model | SyncNet confidence | Offset | Still looks like me | Time for the 17 s clip | | --- | --- | --- | --- | --- | | No lip-sync (baseline) | 0.38 | 11 frames | 1.00 | none | | MuseTalk 1.5 | 2.66 | 1 frame | 0.92 | 96 s | | FaceFusion (EDTalk) | 4.65 | 3 frames (120 ms) | 0.95 | 92 s, CPU only | | LatentSync 1.5 | 6.55 | 0 | 0.97 | about 9 min | !The same clip side by side: no lip-sync, MuseTalk 1.5, FaceFusion EDTalk and LatentSync 1.5, with mouth close-ups and scores under each. Same clip, same voice, three models. (It's a GIF, so no sound.) The numbers matched what I saw. This chart made it obvious: !Four rows, one per model. Blue shows voice loudness, red shows how open the mouth is. With no lip-sync and with MuseTalk the red line is almost flat; FaceFusion twitches; LatentSync opens and closes along with the voice and closes on p, b and m sounds. Red is how far my mouth opens, blue is how loud the voice is. Dotted lines mark words starting with p, b or m, where lips have to close. LatentSync won. Its newer 1.6 version works at 512 px, but one attention step alone needed about 8.6 GB and my Mac started swapping, so I settled on 1.5 at 256 px with a few small patches to run on Apple's GPU. Recording the source take The model needs footage of you to put new lips on. The agent built me a small recording page with a hands-free teleprompter: a neutral pause, a greeting, a bit of explaining, 30 seconds of just listening, and a sign-off. Each part gives the avatar different raw material to draw from. !The recording kit page: three shirts, about three minutes each, with a setup checklist. I recorded two takes in two shirts. Then it turned out the microphone had recorded pure noise, because QuickTime had picked the wrong input. It didn't matter. The model replaces the mouth anyway, and for timing the agent read my lip movement straight from the video with face landmarks instead of from the audio. Getting the head to move with the voice Good lips weren't enough. My head would nod during a pause in the new voice and sit still in the middle of a sentence, which looks like a bad dub. The fix was to nudge the source footage's speed a little (between 0.85× and 1.17×) and line up pauses in the new voice with moments where I'd paused on camera. One number tracks it: the correlation between voice loudness and mouth activity. It went from −0.08 without the nudge to 0.40 with it. Feedback that shaped the look - Circle to webcam tile. A round bubble looked like a sticker. A rounded 16:9 tile showing head, shoulders and a bit of room looks like someone on a call. - Colour. My camera's white balance turned a black T-shirt green. The fix is baked into the render. - No nodding. Nodding while nobody is talking reads as agreeing with no one. The avatar only uses the calmest stretch of the take. - Smaller, twice. The tile went from 352 px to about 260 px wide on a 1920 px video. Part 2: The voice Voicebox is an open-source app that clones a voice from a short sample and runs it locally. My first clone came from a meeting recording and sounded flat, a bit robotic. My second came from four lively bits of a tech talk where I was presenting, and the difference was huge. That's the one I use. What I learned about getting a cloned voice to sound like me on a good day: - Style instructions don't work on a cloned voice. "Say this enthusiastically" did nothing. Livelier lines did: a couple of exclamation marks, shorter sentences. - Generate the whole script in one go. Line by line, the energy jumped around: an excited first line, a sleepy third one. - Generate several takes and measure them. The agent makes six takes and keeps the one that matches the script word for word (checked with a transcript), has the most movement in pitch and loudness, and drops in pitch on the last word so the ending sounds finished. - Read every transcript. One take said "and then do anything" instead of "undo anything". That one sentence got swapped in from another take. - Don't respell names. Spelling "Annotator" phonetically to help the model backfired: it came out sounding French. The plain spelling plus picking the best take worked better. Three bugs hiding in the pauses The screen needs room between sentences, so the voice gets short pauses. Those pauses caused three bugs in a row. Bug 1: digital silence. The first version inserted silence between sentences. My feedback: "the hard pauses are too snappy". The sound dropped from my voice to absolute zero, and real rooms are never that quiet. Bug 2: the stuck record. The next version filled the gaps by looping a slice of the quiet bit between words. "It sounds snappy, repeated, like a stuck disk." Short loops are easy to hear. Bug 3: clipped words. Sentences were cut at the word edges a speech-to-text model (Whisper) gave us. In flowing speech those edges drift by about 0.3 seconds, so word endings got chopped. !Two audio level plots around the end of a sentence. Before: the level falls off a cliff to digital silence. After: it settles to a steady low room tone, then rises into the next word. The version that works cuts at the quietest point measured in the audio itself, fills gaps with smooth, non-repeating noise at about −66 dB, and never loops. An automated check now flags any drop of more than 30 dB within 15 ms anywhere in the track. Part 3: The screen Why it has to be the real app An early version stitched together clips from the product website. I rejected it: it had to be a recording of the installed app. The website shows what the app should do. A recording shows that it does. How the computer drives itself - A separate screen. DeskPad creates a virtual 1920×1200 display. The recording happens there, so my real screens stay mine, and ffmpeg records only that display. - Buttons by name. A small Swift tool posts mouse and keyboard events. It finds controls through the macOS accessibility API by their names ("Pen", "Arrow", "Laser pointer") instead of pixel positions, so a moved button doesn't break the take. - A cursor that moves like a hand. Moves take longer for far or small targets (Fitts's law), overshoot slightly and follow a curve. - Timing from the voice. The voice comes with word timings, so the plan can say "draw the box when I say box": at 16.20 # a beat before the word "box" tool Pen path 0.8 -1528 512 -1450 506 -1360 505 ... mark box # log when it really happened Every action logs when it really happened. If any lands more than about half a second from its word, the agent records another take. The first version took about 15 takes. !Six frames from one take, each captioned with the words being spoken: a messy box around the download button, a curvy arrow and a "Start here!" label, a laser loop around the Applications icon, numbered steps, the arrow turning blue, and the page scrolled back to the top. Six moments from one take, with what the voice is saying at each. What broke | What I saw | Why | Fix | | --- | --- | --- | | Stray "4" and "5" counters later in the video | The toolbar hides itself after 3 seconds. A click meant for it hit the page while the counter tool was still on. | Check the button is still there right before clicking. Switch back to Select straight after the last counter. | | Colour keys did nothing, the arrow stayed blue | My keyboard was on Hebrew, and the app's shortcuts read the typed character. | The recorder won't start unless the layout is English. | | Every click missed | I'd rearranged my monitors overnight. | A setup script finds the virtual display and shifts every coordinate. | | The laser pointer looked robotic | Constant speed between points makes sharp corners. | About 120 points along an eased curve, with equal time per point. | | Things changed like magic | Fast moments use keyboard shortcuts, which you can't see. | Show the keys on screen as keycaps. | | Undo looked like a glitch | Undoing a move made the arrow snap back. | Undo a throwaway scribble instead. | | The label landed 2 seconds late | A trip to the toolbar costs about 2 seconds. | Use the keyboard shortcut. | !Four keycaps: control + 2 "close", ⌘ + Z "undo" (pressed), L "laser" and B "blue". Keycaps appear at the right edge when a shortcut is pressed, so nothing changes by magic. A few smaller touches came from watching drafts. A card with the URL slides up when the voice says it. Captions come from the voice's word timings. The video ends on the website's promo reel, synced through Chrome's DevTools Protocol, so the last shot lands on "Draw over any app." Part 4: The 23-minute problem LatentSync's quality came at a price. It runs 20 diffusion steps per frame, and one minute of video took 23 minutes to render. I said it plainly at the time: "Even 23 minutes for 1 minute is too much. Sometimes we render 10 minutes and I don't want to wait 5 hours." So I went back to MuseTalk. It had the weakest lips in the benchmark, but it makes each frame in one network pass instead of 20 steps. Profiling it showed most of the time wasn't in the model at all: turning its output back into pixels took about 70% of GPU time. Three changes: 1. Swap the full image decoder for TAESD, a tiny decoder built for the same kind of output. 2. Prepare the face once per take (landmarks, encoded frames, blend masks) and cache it. 3. Write frames straight into the video encoder in a single pass, with no files in between. !Bar chart of minutes of compute to render one minute of avatar video: LatentSync 1.5 23.5 minutes, MuseTalk 1.5 with the stock decoder 2.6 minutes, MuseTalk 1.5 with the tiny decoder and cache 1 minute. | Setup | Minutes to render 1 minute of video | | --- | --- | | LatentSync 1.5, 20 diffusion steps | 23.5 | | MuseTalk 1.5, stock decoder | 2.6 | | MuseTalk 1.5, tiny decoder and cache | 1.0 | A 59-second video now renders in 61 seconds. !Left: LatentSync, 23 minutes to render this minute. Right: fast MuseTalk, 61 seconds. Below each, the same video at the real tile size. Old renderer on the left, new on the right, the same ten seconds at full size and at tile size. (No sound in the GIF.) At full size, the fast version's lips are softer. At the size of the tile, nobody noticed. The slow model is still there behind a --quality best flag. In the benchmark I judged the models full screen. The product is a 260-pixel tile. If I'd judged at the size people actually see, I'd have picked the fast model a week earlier. Part 5: How I use it on PRs The pipeline exists for pull requests. These are the rules I settled on, most of them from me watching a video and saying "no, not like that": - Every PR gets a demo in its description, without anyone asking. Stacked PRs each get their own, showing only their own changes. - Open on the app. No title slide, no agenda and no "Hey". End on a plain "Thanks". - Show, then explain. The real app in the browser, the cursor working, my voice explaining as it goes, like showing a colleague. - One short zoom-out, only when it's needed. For things you can't see in the UI: where new data is saved, who can read it, a new API route or background job. Taken from the code, not from memory. - Before and Now side by side for anything visible. The agent runs the base branch from a second worktree on another port, with the same demo data, the same clicks and the same pace. - Plain, casual English. Every script goes through a no-AI-slop pass before the voice is generated. - Demo data only. A made-up demo user, never real client data, and nothing from the recording gets committed. - Short and light. About a minute per main change and two to four minutes in total, at −16 LUFS loudness and under 90 MB so GitHub plays it inline. The PR description stays short. My words to the agent were roughly "only the intent, what's necessary for a human to read, and the visuals": Two or three lines: what changed and why. Figma: <link> **Demo** <video> | | Before | After | | -------------- | --------- | --------- | | Template editor | <image> | <image> | | PDF export | <image> | <image> | **Notes for reviewers** - the one or two things worth a closer look It isn't free in tokens. A full demo cost the agent 28 to 99 million tokens (about 59 million on average) and 44 to 128 minutes. An updated screenshot costs about 2 million tokens and five minutes. So after a review fix, the default is a new screenshot in the Before | After table. The agent re-records one beat only when a fix changes what the video shows, and the whole video only when I ask. Recording, voice and rendering also load the CPU and GPU, so only one demo runs on my machine at a time. The avatar is off by default for PR demos. My voice stays on. A face on screen means something when you're showing stakeholders, not when a reviewer is checking a change. Leaving it off also saves the processing power when it isn't needed. The same skill makes complete demo videos too, and that's where the avatar comes in: sprint demos and public videos like the Annotator one above, where a human face is worth it. What I'd do differently - Judge at the size people watch. It would have saved the whole LatentSync detour. - Record the source take with a working mic. Clean audio from the same session would have made a better voice clone than an old meeting recording. - Assume the UI will move. Toolbars hide, keyboards switch layouts and monitors get rearranged. Check the state before every step instead of trusting coordinates. I still have an open question. The avatar loops back and forth over the same calm stretch of footage, which is fine for two minutes, but I don't have a good answer yet for natural head movement in a 10-minute video. If you've solved that on a laptop, I'd love to hear how. Try it Everything here is free and runs on a Mac with Apple Silicon: - Voice: Voicebox (Qwen3-TTS 1.7B). - Lip-sync: MuseTalk 1.5 with the TAESD decoder. LatentSync if you have the time. - Recording: DeskPad for a virtual display, ffmpeg to record it, and macOS accessibility APIs to drive the app. - The app in the demo: Annotator, a free screen-drawing app for Mac and Windows. Here's the full Annotator video from the top of the post: {% embed youtu.be %}
Губы в видео двигает нейросеть. Открытый MuseTalk с GitHub Telegram: t.me Наш рейтинг ИИ моделей: comrad404.com MuseTalk синхронизирует движения губ в видео с отдельной аудиодорожкой. Перевод и создание голоса — отдельные задачи. Показываем опубликованные авторские примеры, включая короткий фрагмент исходного звука. Локальный запуск модели в этом ролике не проводился. Использование лица и голоса требует соответствующих прав и согласия. Проект и источники: TMElyralab/MuseTalk Помощь студентам: kursovaya-online.com #shorts #ии #нейросети #comrad404 #github [en-US] AI-powered lip-syncing. Open-source MuseTalk from GitHub Telegram: t.me Our AI model ranking: comrad404.com MuseTalk synchronizes lip movements in videos with a separate audio track. Translation and voice generation are separate tasks. We show published examples by the author, including a short fragment of the original audio. The model was not run locally in this video. Use of facial features and voice requires appropriate rights and consent. Project and sources: TMElyralab/MuseTalk Student support: kursovaya-online.com #shorts #ai #neuralnetworks #comrad404 #github
TMElyralab/MuseTalk 音频驱动的视频口型同步模型。 主要修改嘴部区域；官方报告 V100 上可达 30fps 以上。 TMElyralab/MuseTalk