Local AI Video Generation: ComfyUI and LTX-Video 2B on an RTX 3050 6GB
> Phase 00 — Building a local AI-powered video generation pipeline.
1. Introduction
AI-powered video generation has evolved significantly, but running generative models locally remains challenging, especially on computers with limited hardware resources.
Many demonstrations rely on high-performance GPUs with substantial VRAM or cloud-based computing platforms.
This raises an interesting question:
Is it possible to generate AI videos using a laptop GPU with only 6GB of VRAM?
To investigate this, I started JC IA Video, a project focused on developing a local text-to-video generation pipeline.
The goal is not simply to generate videos, but to understand the underlying infrastructure, identify technical limitations, and progressively build an automated solution.
In this initial stage, called Phase 00, the objective was to validate local execution of ComfyUI with the LTX-Video 2B model and produce usable MP4 files.
The result was a functional proof of concept, including the resolution of a real issue involving animated WebP conversion.
2. Phase 00 Objectives
The experiment was organized around five primary objectives:
1. Configure a Python environment with NVIDIA GPU support.
2. Run ComfyUI locally using memory optimization techniques.
3. Load LTX-Video 2B and generate videos from text prompts.
4. Convert generated outputs into H.264 MP4 files.
5. Validate the resulting media files and document the findings.
An important requirement was to keep inference entirely local, without relying on external inference services.
3. Hardware and Software Environment
The experiment was conducted on an Alienware 16 Aurora laptop with the following configuration:
| Component | Configuration |
|---|---|
| Device | Alienware 16 Aurora |
| Operating system | Ubuntu Linux |
| GPU | NVIDIA GeForce RTX 3050 Laptop |
| VRAM | 6 GB |
| System RAM | 16 GB |
| Python | 3.12.3 |
| ComfyUI | 0.44.0 |
| PyTorch | 2.15.0+cu132 |
| Generative model | LTX-Video 2B v0.9.5 |
| Video processing | FFmpeg 6.1.1 |
The limited amount of VRAM was one of the main constraints.
For this reason, the initial configuration prioritized low resolutions, short videos, and memory-efficient execution.
Evidence 01 — GPU Detection
!Figure 1 — NVIDIA RTX 3050 Laptop GPU detected by the operating system, with 6GB of VRAM.
Figure 1 — NVIDIA RTX 3050 Laptop GPU detected by the operating system, with 6GB of VRAM.
The screenshot should display the GPU model, driver version, and available memory.
4. Solution Architecture
The initial pipeline was built using tools with clearly separated responsibilities.
Text Prompt
|
v
ComfyUI
|
v
LTX-Video 2B
|
v
CUDA Inference
|
v
Generated Frames
|
v
Animated WebP
|
v
Pillow
Frame Extraction
|
v
FFmpeg
H.264 Encoding
|
v
Final MP4
ComfyUI provides the interface for executing AI generation workflows.
LTX-Video generates the visual frames based on the supplied conditioning inputs.
Pillow processes frames from animated WebP files, while FFmpeg handles the final MP4 encoding.
This separation of responsibilities will become particularly important in future phases, when the workflow is controlled by a dedicated Python application.
5. Configuring ComfyUI
ComfyUI was installed inside a Python virtual environment to isolate its dependencies.
The application was launched using:
cd ~/projects/jc-ia-video/ComfyUI
source .venv/bin/activate
python main.py \
--listen 127.0.0.1 \
--port 8188 \
--lowvram
The --lowvram parameter was used to adapt execution to the GPU's memory constraints.
The application was available at:
http://127.0.0.1:8188
The environment successfully initialized with CUDA support and memory management mechanisms, including DynamicVRAM and model weight offloading between system memory and GPU memory.
These mechanisms do not eliminate hardware limitations, but they help make certain workloads feasible on GPUs with limited VRAM.
Evidence 02 — ComfyUI Execution and Model Loading
The logs recorded dynamic model loading and three completed executions.
Selected log entries:
[INFO] To see the GUI go to: http://127.0.0.1:8188
[INFO] Requested to load LTXV
[INFO] Model LTXV prepared for dynamic VRAM loading.
[INFO] Requested to load VideoVAE
[INFO] Prompt executed in 16.72 seconds
[INFO] Prompt executed in 6.87 seconds
[INFO] Prompt executed in 12.01 seconds
These messages confirm that ComfyUI was accessible, the inference components were prepared, and all three prompts completed execution.
!Figure 2 — ComfyUI logs showing dynamic loading of LTXV and VideoVAE models and three completed prompt executions.
Figure 2 — ComfyUI logs showing dynamic loading of LTXV and VideoVAE models and three completed prompt executions.
The full logs also recorded text encoder preparation and execution of the 12 sampling steps.
6. Models Used
Two main components were used for video generation.
LTX-Video 2B v0.9.5
The generative model responsible for producing visual content.
Model file:
models/checkpoints/
└── ltx-video-2b-v0.9.5.safetensors
T5 XXL FP8
The text encoder used to process text conditioning inputs.
Model file:
models/text_encoders/
└── t5xxl_fp8_e4m3fn_scaled.safetensors
The LTX-Video model was downloaded from Hugging Face.
The text encoder was made available to ComfyUI through a symbolic link pointing to a file already stored in the local Hugging Face cache.
This avoided creating another copy of approximately 4.9GB.
One important operational consideration is that deleting the original cached file would break the symbolic link.
Model weights are excluded from the project's Git repository.
7. Building the Generation Workflow
After configuring the environment, I loaded a text-to-video workflow compatible with LTX-Video.
The workflow connects components responsible for:
- Checkpoint loading.
- Text encoder loading.
- Positive and negative prompt processing.
- Video latent configuration.
- Sampling schedule configuration.
- Frame generation.
- Output decoding.
- Animated WebP export.
Evidence 03 — Workflow and Generation Parameters
Conservative generation parameters were selected to reduce resource consumption.
| Parameter | Value |
|---|---|
| Landscape resolution | 512 × 288 |
| Portrait resolution | 288 × 512 |
| Frames | 25 |
| Frame rate | 8 FPS |
| Sampling steps | 12 |
| CFG | 3.0 |
| Sampler | res_multistep |
| Batch size | 1 |
With 25 frames at 8 FPS, each video has a nominal duration of:
25 / 8 = 3.125 seconds.
The portrait resolution of 288 × 512 preserves the 9:16 aspect ratio commonly used by TikTok, Instagram Reels, and YouTube Shorts.
However, this resolution is significantly lower than typical production-quality video resolutions.
The purpose of this phase was to validate the generation pipeline, not to achieve final production quality.
!LTX-Video 2B text-to-video workflow in ComfyUI, including generation parameters.
Figure 3 — LTX-Video 2B text-to-video workflow in ComfyUI, including generation parameters.
This screenshot combines the visual workflow architecture and the relevant configuration values.
8. First Generation Results
The workflow successfully generated three animated WebP files.
| File | Resolution | Frames | FPS |
|---|---|---|---|
| ComfyUI_00001_.webp | 512 × 288 | 25 | 8 |
| ComfyUI_00002_.webp | 512 × 288 | 25 | 8 |
| ComfyUI_00003_.webp | 288 × 512 | 25 | 8 |
During generation, GPU utilization was observed reaching 100% at one point.
This indicates intensive GPU processing, although it does not establish that the GPU remained fully utilized throughout the entire inference process.
Execution Times
ComfyUI recorded the following execution times:
| Execution | Recorded Time |
|---|---:|
| First | 16.72 seconds |
| Second | 6.87 seconds |
| Third | 12.01 seconds |
The first execution may include additional model loading and initialization overhead.
These values represent execution times reported by ComfyUI. They should not be interpreted as controlled benchmark results.
Repeated tests under standardized conditions will be required to evaluate performance, consistency, and resource consumption.
Demo — Portrait Video (9:16)
The third video was generated at 288 × 512 pixels, with 25 frames and a frame rate of 8 FPS.
The result was published on YouTube Shorts as a visual demonstration of the proof of concept.
Watch the AI-generated video on YouTube Shorts
Demo 1 — Portrait video generated locally using ComfyUI and LTX-Video 2B on an RTX 3050 Laptop GPU with 6GB of VRAM.
This demonstration provides a visual reference for the model's output, complementing the configuration and execution evidence.
9. A Real Issue: FFmpeg and Animated WebP
After generating the videos, I encountered an issue while converting the output files to MP4.
The first attempt used FFmpeg directly:
ffmpeg -i input.webp \
-c:v libx264 \
-pix_fmt yuv420p \
output.mp4
However, the FFmpeg version available in the environment could not correctly decode the animated WebP files produced by ComfyUI.
The observed messages included:
skipping unsupported chunk: ANIM
skipping unsupported chunk: ANMF
image data not found
The conversion process failed to produce valid MP4 videos.
Subsequent validation also identified invalid media containers, including the error:
moov atom not found
This demonstrated an important lesson: the existence of a file with an .mp4 extension does not guarantee that it contains a valid, playable video.
The container structure and media metadata must be validated.
The original terminal screenshots were not preserved. Therefore, the error messages recorded during the experiment are included as textual evidence.
10. The Solution: Pillow + FFmpeg
To work around the decoding limitation, I separated the conversion process into two stages.
First, Pillow extracts individual frames from the animated WebP file.
Then, FFmpeg encodes those frames into an H.264 MP4 video.
The updated pipeline became:
Animated WebP
|
v
Pillow
|
v
Temporary PNG Frames
|
v
FFmpeg / libx264
|
v
H.264 MP4
I created a Python script to automate this conversion.
The initial implementation uses Image.open() to read the animated WebP frames, TemporaryDirectory() to store temporary PNG files, and subprocess.run() to execute FFmpeg.
A simplified example of the frame extraction process:
from PIL import Image
with Image.open("input.webp") as image:
for index in range(image.n_frames):
image.seek(index)
frame = image.convert("RGB")
frame.save(f"{index:05d}.png")
The extracted frames can then be encoded using:
ffmpeg \
-framerate 8 \
-i "%05d.png" \
-c:v libx264 \
-pix_fmt yuv420p \
-movflags +faststart \
output.mp4
The yuv420p pixel format improves compatibility with media players and online platforms.
The +faststart option relocates MP4 metadata to support progressive playback more efficiently.
The initial converter assumes a constant frame rate. Handling variable frame durations remains a potential improvement for a future version.
11. Validating the MP4 Files
After implementing the conversion script, all three WebP files were successfully processed.
The recorded output was:
OK: ComfyUI_00001_.mp4 | 25 frames | 8.00 FPS
OK: ComfyUI_00002_.mp4 | 25 frames | 8.00 FPS
OK: ComfyUI_00003_.mp4 | 25 frames | 8.00 FPS
I then used ffprobe to inspect the generated media metadata.
The results were:
| File | Resolution | FPS | Duration | File Size |
|---|---|---|---|---|
| ComfyUI_00001_.mp4 | 512×288 | 8 | 3.125 s | 43,349 bytes |
| ComfyUI_00002_.mp4 | 512×288 | 8 | 3.125 s | 46,037 bytes |
| ComfyUI_00003_.mp4 | 288×512 | 8 | 3.125 s | 157,615 bytes |
Validation confirmed that all three MP4 files contained resolution, frame rate, and duration metadata consistent with the generation settings.
The third file confirmed successful export in a portrait 9:16 format.
The results are preserved as textual evidence and a technical table, without requiring additional terminal screenshots.
12. Results and Limitations
Phase 00 successfully validated the fundamental components of a local AI video generation pipeline.
Confirmed Results
- ComfyUI running locally with CUDA support.
- LTX-Video 2B loaded and used for generation.
- Three animated WebP outputs generated.
- Landscape and portrait generation tested.
- Successful conversion to H.264 MP4.
- Output metadata verified using ffprobe.
- Three execution times recorded by ComfyUI.
- A portrait video published as a visual demonstration.
Identified Limitations
- Low output resolution.
- Video duration limited to approximately three seconds in the initial tests.
- High GPU resource utilization.
- No controlled performance benchmark yet.
- Video conversion still handled by a separate script.
- Generation workflow still operated manually.
- Visual quality not yet evaluated using systematic criteria.
These limitations help define the scope of future development.
The results do not imply that every video configuration, resolution, or model will run successfully on a 6GB GPU.
They demonstrate that one specific configuration was successfully validated on this hardware.
13. Next Phase: Python Automation
With the proof of concept completed, the next objective is to transform the manual workflow into an automated pipeline.
Phase 01 will focus on using the ComfyUI API to:
1. Receive text prompts.
2. Load previously validated workflows.
3. Submit generation jobs.
4. Monitor execution status.
5. Identify and retrieve generated outputs.
6. Automatically convert results into MP4.
7. Record execution metadata, errors, and processing times.
The proposed architecture is:
Python CLI
|
v
Workflow Manager
|
v
ComfyUI API
|
v
LTX-Video 2B
|
v
Output Manager
|
v
Media Converter
|
v
MP4 + Execution Metadata
This approach will allow the project to evolve without modifying ComfyUI's internal source code.
14. Conclusion
This experiment demonstrated that short AI-generated videos can be produced locally using an RTX 3050 Laptop GPU with 6GB of VRAM, provided that the workload is configured within the available hardware constraints.
It also highlighted that building a functional AI video generation pipeline involves much more than running a generative model.
It requires understanding the inference environment, managing hardware resources, interpreting execution errors, processing intermediate media formats, and validating the resulting artifacts.
The animated WebP conversion issue was particularly valuable: a seemingly simple failure demonstrated the importance of separating responsibilities between tools and verifying outputs rather than assuming success.
Phase 00 established the technical foundation of JC IA Video.
The next step is to transform this proof of concept into a reproducible and automated pipeline.
15. Project Repository and Availability
JC IA Video is being developed incrementally, with each phase documenting technical decisions, implementation details, limitations, and validation results.
The project currently maintains its source code, workflows, scripts, and technical documentation in a local Git repository.
The GitHub repository will be made publicly available in the future, once the project reaches an appropriate level of organization, reproducibility, and documentation.
Future articles in this series will cover the implementation of the automation pipeline, resource optimization, and additional video processing capabilities.
---
References
- ComfyUI — Official GitHub Repository
- LTX-Video — Official GitHub Repository
- LTX-Video — Hugging Face
- PyTorch — Official Documentation
- Pillow — Official Documentation
- FFmpeg — Official Documentation
- FFprobe — Official Documentation
---
This is the first article in the JC IA Video — Local AI Video Generation series, documenting the incremental development of a local AI video generation pipeline using Python, ComfyUI, and open-source tools.
Real-looking people in a product video, for $0, on a MacBook: I tested 5 methods
> TL;DR
> - I made the same 30-second story five ways to find the best free, local way to put real-looking people in a product video: free stock, AI images (SDXL), AI video (LTX-Video), Blender 3D and a lip-synced AI narrator.
> - Everything ran on one MacBook Pro (M1 Max, 64 GB), cost $0, and used about 28 GB of models.
> - The winner is AI images with SDXL: realistic, consistent characters at ~2 min per shot on the Mac GPU.
> - AI video (LTX-Video) is the only method where people truly move, but it took 11–26 min per 5-second clip, and crowds and big motion melt.
> - Surprise: the video model generates a clip in ~30 s. The other 95% of the time is decoding plus memory swap.
> - Everything is open: live side-by-side dashboard · repo with inputs, outputs, scripts and logs · full report
!Same story, five ways. Rows: free stock, AI images, AI video, Blender 3D, talking narrator
Same 30-second story, five ways. Rows top to bottom: free stock · AI images (SDXL) · AI video (LTX-Video) · Blender 3D · talking narrator (SadTalker). Columns are the five shots.
▶ Watch all five videos in sync: side-by-side dashboard · each video embedded on the original post
Who this is for
You make product demos, explainers or launch videos with code, an editor or a tool like Remotion or HyperFrames. Your app screens look great, but there are no humans: no customer, no user, no story. You want people in the video, you don't want to pay for footage or an AI video subscription, and you have a decent Mac.
By the end you'll know:
- which of the five free methods to use for which shot;
- what each one really costs in time on Apple Silicon, and why;
- the exact settings, prompts and patches that made them work, including the errors you'll hit and their fixes.
The problem
My explainer videos had a recurring hole. The product screens were sharp, but every "real world" moment (a candidate arriving, a phone buzzing, an interview) was a silhouette or an icon. Viewers connect with people. The usual answers are paid stock footage, a paid AI video tool, or filming. I wanted to know how far free and local gets you in 2026.
How I tested (methodology)
One story, five methods; only the people change. The story: Aarav gets a QR call letter for a walk-in interview → scans in at reception → waits in a lounge until his phone buzzes → interviews → his record is handed off to the company's HR system. That's 5 shots, 30.4 seconds in total.
Fairness controls. Every version shares:
- the same storyboard and shot timings (6.5 / 5.3 / 6.5 / 5.7 / 6.4 s);
- the same narration (Kokoro TTS, voice am_michael, generated locally);
- the same app overlay cards and the same music bed;
- the same assembly template, rendered with HyperFrames (HTML + GSAP → MP4) at 1080p.
Hardware. MacBook Pro, Apple M1 Max: 10-core CPU (8 performance + 2 efficiency), 32-core GPU, 64 GB of unified memory, macOS 26.5. There's no NVIDIA GPU, so "GPU" means the Apple GPU through PyTorch's mps backend or Blender's Metal renderer. I left my normal apps open (an IDE, a Linux VM, an editor), which turned out to matter.
What I measured.
- Wall-clock time per step, from logs.
- Which device each step ran on.
- Memory and swap.
- Seven quality criteria scored 1–5: realism, character consistency, motion, story fit, speed, reliability and licence clarity.
Prerequisites
| You need | Why |
|---|---|
| Apple Silicon Mac, ideally 32 GB+ of RAM | The AI video model needs ~25 GB of working memory in fp32 |
| ~30 GB of free disk | Model weights |
| Python 3.10 and 3.11, uv, ffmpeg | Two separate environments; SadTalker needs old pinned libraries |
| Blender 4.5 | Method 4. Use the blender.org DMG; the Homebrew cask asks for sudo |
| Node 18+ | npx hyperframes for the final assembly (any editor works) |
The models and their download sizes:
| Model | Used for | Size | Licence |
|---|---|---|---|
| RealVisXL V4 (SDXL, fp16) + fp16-fix VAE | AI images | 7.3 GB | Open RAIL++-M |
| LTX-Video 0.9.5 transformer + VAE | AI video | 6.3 GB | Open RAIL-M |
| T5-XXL encoder, bf16 copy | LTX text encoder | 9.5 GB (instead of 19 GB) | Apache-2.0 |
| SadTalker + GFPGAN weights | Narrator | 1.7 GB | Apache-2.0 |
| Quaternius animated characters | Blender | small | CC0 |
Method 1: free stock photos
How it works. Search Wikimedia Commons (its API needs no key) and Openverse for each shot. Filter licences to CC0, CC BY and CC BY-SA (never NC or ND), crop to 16:9, and add a slow camera push.
curl -s "https://commons.wikimedia.org/w/api.php?action=query&list=search&srnamespace=6&srsearch=filetype:video%20job%20interview&format=json"
Result. No usable free video exists for offices, receptions or interviews: the hits are under 720p or are news footage. So all five shots became photos. Each shot shows a different person, most aren't Indian, the "lounge" is a bus terminal, and in shot 1 we only see the back of a head.
Time: ~22 min of searching and reviewing. GPU: none.
Verdict: real, and legally clean with attribution, but it can't tell your story with your character. Pexels has much better free video, but needs an API key.
Method 2: AI images with SDXL (the winner)
How it works. Write one fixed character description and reuse it in every shot prompt. Fix the seeds, generate 2 options per shot, and pick the best.
AARAV = "Aarav, a 26 year old Indian man, short neat black hair, light stubble, navy blazer over a white shirt"
STYLE = "photorealistic, candid documentary photo, 35mm lens, natural light, shallow depth of field, ..."
vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipe = StableDiffusionXLPipeline.from_pretrained("SG161222/RealVisXL_V4.0", vae=vae,
torch_dtype=torch.float16, variant="fp16").to("mps")
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True)
img = pipe(f"{AARAV}, sitting on a sofa ... looking down at his smartphone, {STYLE}",
negative_prompt=NEG, width=1344, height=768, num_inference_steps=30,
guidance_scale=5.5, generator=torch.Generator("cpu").manual_seed(1000)).images[0]
The fp16-fix VAE matters: the stock SDXL VAE in fp16 on Apple GPUs tends to decode black or NaN images.
!All 10 SDXL candidates, two per shot
All 10 candidates, two per shot. Choosing took seconds.
Result. These look like candid office photos, and Aarav is recognisably the same person in most shots. Weaknesses:
- In the busy interview scene his identity drifted (glasses and a beard appeared).
- Prompts are followed loosely: no receptionist actually scans his phone.
- The images are stills, so motion comes only from the camera.
Time: 108–131 s per 1344×768 image (average 119 s), so 16 images took ~28 min. GPU: yes (MPS, fp16).
Verdict: ⭐ the best realism per minute.
Method 3: AI video with LTX-Video (alive, but fragile)
How it works. Animate the method 2 stills with LTX-Video 0.9.5 image-to-video, prompting only the motion. Two memory tricks are essential on a Mac:
pipe = LTXImageToVideoPipeline.from_pretrained("Lightricks/LTX-Video-0.9.5",
text_encoder=None, torch_dtype=torch.float32)
# 1) encode every prompt once on the CPU, then drop the 19 GB text encoder before generating
pipe.text_encoder = T5EncoderModel.from_pretrained("city96/t5-v1_1-xxl-encoder-bf16", torch_dtype=torch.float32)
emb = {s: pipe.encode_prompt(MOTION[s], NEG, do_classifier_free_guidance=True, device="cpu") for s in shots}
pipe.text_encoder = None; gc.collect()
pipe.to("mps"); pipe.vae.enable_tiling() # 2) tiled decoding
pe, pm, ne, nm = (t.to("mps") for t in emb["s1"])
frames = pipe(image=still, prompt_embeds=pe, prompt_attention_mask=pm,
negative_prompt_embeds=ne, negative_prompt_attention_mask=nm,
width=768, height=448, num_frames=121, num_inference_steps=30, guidance_scale=3.0,
generator=torch.Generator("mps").manual_seed(42)).frames[0]
Result. For one person with small motion it's genuinely convincing: Aarav smiles at his phone, and in the interview he talks and gestures while the panel listens.
!Interview clip: natural talking and gesturing
The interview clip at 0 s, 2.5 s and 5 s: natural talking and gesturing.
Crowds and big motion break it. The busy reception melts into a blur after ~3 s, and walking out of the building warps his body.
!Reception clip: fine at 0s, morphing at 2.5s, melted at 5s
The reception clip at 0 s, 2.5 s and 5 s: the crowd melts.
I rescued those two clips by keeping the clean first ~3 s and playing them forward then backward (ffmpeg ... trim,split,reverse,concat). The raw clips are in the repo.
Time:
- Quick test (49 frames, 20 steps): 278 s.
- Full clips (121 frames ≈ 5 s, 30 steps): 811, 681, 1,568, 1,381 and 804 s, ~87 min for all five.
GPU: yes (MPS, fp32; see the pitfalls).
Verdict: use it for 2–3 hero moments, not a whole film.
Method 4: Blender 3D with free rigged characters
How it works.
- Download CC0 rigged and animated characters (Quaternius, via Poly Pizza; no login).
- Recolour them to match the cast and build sets from primitives.
- Script every shot in Blender Python and render headless with EEVEE on the GPU:
~/Applications/Blender.app/Contents/MacOS/Blender -b -P shots.py -- s1 full 1920 1080 24
Result. Every shot does exactly what the storyboard asks. He holds the phone to the scanner, the phone buzzes, he walks out, and nobody is stuck in a T-pose. But it looks like a video game:
- faces have no expressions and there's no lip-sync;
- the same three base models are reused for everyone;
- the sets are untextured.
Realistic free 3D humans are hard to get: Mixamo needs an Adobe login, and MetaHuman and Character Creator aren't free.
Time: ~35 min in total, of which 14 min was rendering 878 frames at 1080p. GPU: yes (Metal, EEVEE).
Verdict: total control and consistency, but no realism.
Method 5: a talking AI narrator with SadTalker
How it works.
1. Generate a fictional presenter face with SDXL (method 2).
2. Lip-sync it to the full narration with SadTalker.
3. Clean up the face with GFPGAN.
4. Show it as a corner bubble.
python inference.py --driven_audio narration.wav --source_image presenter.png \
--still --preprocess crop --enhancer gfpgan --size 256
Result. The lip-sync is frame-accurate across the full 30 s, but only the head moves, and at 512×512 it only works small. Match the face to the voice: my first presenter was a woman against a male voice, and it looked wrong.
Time:
- On the CPU: 24 s per step, projected at ~2.5 h.
- On the GPU after a patch: 11.7 min, about 13× faster.
- The GFPGAN clean-up then took 39 min on the CPU, because GFPGAN has no Apple GPU path.
GPU: partly.
Verdict: a nice guide or presenter; it can't carry the story.
Results
| | 1 · Stock | 2 · AI images | 3 · AI video | 4 · Blender | 5 · Narrator |
|---|---|---|---|---|---|
| Realism | 5 | 4 | 4 | 2 | 3 |
| Same character throughout | 1 | 4 | 4 | 5 | 5 |
| People actually move | 1 | 1 | 4 | 3 | 2 |
| Matches the storyboard | 2 | 4 | 3 | 4 | n/a |
| Speed | 4 | 4 | 1 | 3 | 2 |
| Reliability | 4 | 5 | 2 | 4 | 3 |
| Licence clarity | 3 | 4 | 3 | 5 | 4 |
| Average | 2.9 | 3.7 | 3.0 | 3.7 | 3.2 |
| Time for 30 s | ~22 min | ~28 min | ~87 min | ~35 min | ~52 min |
| Ran on | CPU | GPU | GPU | GPU | GPU + CPU |
Blender ties SDXL on the average only because consistency and licence clarity weigh as much as realism. If "real-looking people" is the goal, SDXL wins clearly.
Where the time actually goes on a Mac
| Step | Device | Measured |
|---|---|---|
| SDXL image, 1344×768, 30 steps | GPU | 108–131 s |
| LTX-Video: generating a 5 s clip | GPU | 27–29 s (~1 step/s) |
| LTX-Video: decoding + swap for that clip | GPU + swap | 650–1,540 s |
| Blender, 878 frames at 1080p | GPU (Metal) | 14 min |
| SadTalker face renderer, 380 steps | CPU → GPU | 24.1 → 1.85 s per step |
| GFPGAN clean-up, 760 frames | CPU | ~3.1 s per frame |
| Assembling one 30 s video | CPU + headless Chrome | 63–93 s |
The biggest surprise: the AI video model isn't slow at generating. 95% of each clip's time went to decoding the frames while macOS swapped up to 32 GB to disk. The T5 encoder alone is 19 GB in fp32, and my IDE and VM were open. Unloading T5 after encoding cut the swap to 17.5 GB. Closing other apps is the single biggest speed-up.
Pitfalls and fixes
| Symptom | Cause | Fix |
|---|---|---|
| Input type (float) and bias type (c10::BFloat16) should be the same | M1 has no native bf16 | Load LTX-Video in fp32; it costs ~2× the memory |
| Expected a 'mps' device type for generator but found 'cpu' | A CPU random generator fed to an MPS pipeline | torch.Generator("mps") |
| SadTalker: Conv3D is not supported on MPS | Its pinned PyTorch 2.1 | Upgrade to PyTorch 2.5.1 and apply this 3-file patch |
| SadTalker: invalid type: 'torch.mps.FloatTensor' | CUDA-style .type() calls | Covered by the same patch |
| No module named 'pkg_resources' | New setuptools | uv pip install "setuptools<70" |
| basicsr import error on functional_tensor | New torchvision | Change the import to torchvision.transforms.functional |
| SDXL produces black or NaN images in fp16 | The stock VAE overflows | Use madebyollin/sdxl-vae-fp16-fix |
| A 30 GB download takes hours | One slow stream | Download models in parallel with hf download, and use the bf16 T5 copy (9.5 GB, not 19 GB) |
| brew install --cask blender asks for a password | The cask needs sudo | Copy Blender.app from the official DMG into ~/Applications |
Which method should you use?
- You need a recurring, realistic main character → SDXL stills with a fixed character description and seeds, plus camera motion. Generate 2–3 options per shot.
- One hero moment needs real motion (a smile, a nod, someone talking) → LTX-Video on that SDXL still. One person, small motion, keep the clean seconds.
- The shot has a crowd or a person walking → keep it a still. Today's local video models melt here.
- You need exact staging (someone holds a phone to a scanner at a precise moment) → Blender, if a game look is acceptable.
- You want a presenter face → a SadTalker bubble with a voice-matched face.
- You need real people and don't care who → stock, ideally Pexels with a free API key.
Limitations
- One machine and one run per clip. No repeated runs, so no variance numbers. An M3 or M4, or a Mac with less load, will be faster.
- The scores are my judgement, not a user study. The repo has every output so you can judge for yourself.
- I picked the best of 2 seeds for the stills and trimmed two AI video clips. That's realistic practice, but it is curation.
- On an NVIDIA GPU (CUDA, bf16) the AI video steps would be many times faster; this is a Mac-specific result.
Reproduce it
git clone https://github.com/kartikjindal/local-character-video-5-methods-benchmark && cd local-character-video-5-methods-benchmark
uv venv .venv-ai --python 3.11 && uv pip install --python .venv-ai/bin/python -r requirements-ai.txt
./download-models.sh # ~23 GB, in parallel
.venv-ai/bin/python gen_stills.py --seeds 2 # method 2
.venv-ai/bin/python gen_video.py --frames 121 --steps 30 # method 3
python3 -m http.server 8777 # dashboard: compare all five
The SadTalker and Blender steps are in the README.
Conclusion
Free and local gets you surprisingly far. SDXL stills give realistic, consistent characters for ~2 minutes of GPU time per shot, and that should be the backbone. Local AI video is real but narrow: magic for one person making small movements, unusable for crowds, and slow on a Mac mainly because of memory, not compute. Everything here cost $0 and about 28 GB of disk.
What would you try next? I'm curious whether a newer local video model, or a Mac with more memory headroom, changes the AI video verdict. If you've run LTX-Video, Wan or HunyuanVideo on Apple Silicon, share your timings in the comments.
---
Resources
- ▶ Live side-by-side dashboard (all five videos in sync)
- 📦 Repo: inputs, outputs, scripts, logs
- 📄 Full report with every measured number
- Tools: RealVisXL V4 · LTX-Video · SadTalker · GFPGAN · Blender · Quaternius · Kokoro TTS · HyperFrames · ffmpeg
All AI faces in this post are fictional.
Mentions
Model: ltx.io Code: Lightricks/LTX-2 Footage: LTX
Local AI Video Generation: ComfyUI and LTX-Video 2B on an RTX 3050 6GB > Phase 00 — Building a local AI-powered video generation pipeline. 1. Introduction AI-powered video generation has evolved significantly, but running generative models locally remains challenging, especially on computers with limited hardware resources. Many demonstrations rely on high-performance GPUs with substantial VRAM or cloud-based computing platforms. This raises an interesting question: Is it possible to generate AI videos using a laptop GPU with only 6GB of VRAM? To investigate this, I started JC IA Video, a project focused on developing a local text-to-video generation pipeline. The goal is not simply to generate videos, but to understand the underlying infrastructure, identify technical limitations, and progressively build an automated solution. In this initial stage, called Phase 00, the objective was to validate local execution of ComfyUI with the LTX-Video 2B model and produce usable MP4 files. The result was a functional proof of concept, including the resolution of a real issue involving animated WebP conversion. 2. Phase 00 Objectives The experiment was organized around five primary objectives: 1. Configure a Python environment with NVIDIA GPU support. 2. Run ComfyUI locally using memory optimization techniques. 3. Load LTX-Video 2B and generate videos from text prompts. 4. Convert generated outputs into H.264 MP4 files. 5. Validate the resulting media files and document the findings. An important requirement was to keep inference entirely local, without relying on external inference services. 3. Hardware and Software Environment The experiment was conducted on an Alienware 16 Aurora laptop with the following configuration: | Component | Configuration | |---|---| | Device | Alienware 16 Aurora | | Operating system | Ubuntu Linux | | GPU | NVIDIA GeForce RTX 3050 Laptop | | VRAM | 6 GB | | System RAM | 16 GB | | Python | 3.12.3 | | ComfyUI | 0.44.0 | | PyTorch | 2.15.0+cu132 | | Generative model | LTX-Video 2B v0.9.5 | | Video processing | FFmpeg 6.1.1 | The limited amount of VRAM was one of the main constraints. For this reason, the initial configuration prioritized low resolutions, short videos, and memory-efficient execution. Evidence 01 — GPU Detection !Figure 1 — NVIDIA RTX 3050 Laptop GPU detected by the operating system, with 6GB of VRAM. Figure 1 — NVIDIA RTX 3050 Laptop GPU detected by the operating system, with 6GB of VRAM. The screenshot should display the GPU model, driver version, and available memory. 4. Solution Architecture The initial pipeline was built using tools with clearly separated responsibilities. Text Prompt | v ComfyUI | v LTX-Video 2B | v CUDA Inference | v Generated Frames | v Animated WebP | v Pillow Frame Extraction | v FFmpeg H.264 Encoding | v Final MP4 ComfyUI provides the interface for executing AI generation workflows. LTX-Video generates the visual frames based on the supplied conditioning inputs. Pillow processes frames from animated WebP files, while FFmpeg handles the final MP4 encoding. This separation of responsibilities will become particularly important in future phases, when the workflow is controlled by a dedicated Python application. 5. Configuring ComfyUI ComfyUI was installed inside a Python virtual environment to isolate its dependencies. The application was launched using: cd ~/projects/jc-ia-video/ComfyUI source .venv/bin/activate python main.py \ --listen 127.0.0.1 \ --port 8188 \ --lowvram The --lowvram parameter was used to adapt execution to the GPU's memory constraints. The application was available at: http://127.0.0.1:8188 The environment successfully initialized with CUDA support and memory management mechanisms, including DynamicVRAM and model weight offloading between system memory and GPU memory. These mechanisms do not eliminate hardware limitations, but they help make certain workloads feasible on GPUs with limited VRAM. Evidence 02 — ComfyUI Execution and Model Loading The logs recorded dynamic model loading and three completed executions. Selected log entries: [INFO] To see the GUI go to: http://127.0.0.1:8188 [INFO] Requested to load LTXV [INFO] Model LTXV prepared for dynamic VRAM loading. [INFO] Requested to load VideoVAE [INFO] Prompt executed in 16.72 seconds [INFO] Prompt executed in 6.87 seconds [INFO] Prompt executed in 12.01 seconds These messages confirm that ComfyUI was accessible, the inference components were prepared, and all three prompts completed execution. !Figure 2 — ComfyUI logs showing dynamic loading of LTXV and VideoVAE models and three completed prompt executions. Figure 2 — ComfyUI logs showing dynamic loading of LTXV and VideoVAE models and three completed prompt executions. The full logs also recorded text encoder preparation and execution of the 12 sampling steps. 6. Models Used Two main components were used for video generation. LTX-Video 2B v0.9.5 The generative model responsible for producing visual content. Model file: models/checkpoints/ └── ltx-video-2b-v0.9.5.safetensors T5 XXL FP8 The text encoder used to process text conditioning inputs. Model file: models/text_encoders/ └── t5xxl_fp8_e4m3fn_scaled.safetensors The LTX-Video model was downloaded from Hugging Face. The text encoder was made available to ComfyUI through a symbolic link pointing to a file already stored in the local Hugging Face cache. This avoided creating another copy of approximately 4.9GB. One important operational consideration is that deleting the original cached file would break the symbolic link. Model weights are excluded from the project's Git repository. 7. Building the Generation Workflow After configuring the environment, I loaded a text-to-video workflow compatible with LTX-Video. The workflow connects components responsible for: - Checkpoint loading. - Text encoder loading. - Positive and negative prompt processing. - Video latent configuration. - Sampling schedule configuration. - Frame generation. - Output decoding. - Animated WebP export. Evidence 03 — Workflow and Generation Parameters Conservative generation parameters were selected to reduce resource consumption. | Parameter | Value | |---|---| | Landscape resolution | 512 × 288 | | Portrait resolution | 288 × 512 | | Frames | 25 | | Frame rate | 8 FPS | | Sampling steps | 12 | | CFG | 3.0 | | Sampler | res_multistep | | Batch size | 1 | With 25 frames at 8 FPS, each video has a nominal duration of: 25 / 8 = 3.125 seconds. The portrait resolution of 288 × 512 preserves the 9:16 aspect ratio commonly used by TikTok, Instagram Reels, and YouTube Shorts. However, this resolution is significantly lower than typical production-quality video resolutions. The purpose of this phase was to validate the generation pipeline, not to achieve final production quality. !LTX-Video 2B text-to-video workflow in ComfyUI, including generation parameters. Figure 3 — LTX-Video 2B text-to-video workflow in ComfyUI, including generation parameters. This screenshot combines the visual workflow architecture and the relevant configuration values. 8. First Generation Results The workflow successfully generated three animated WebP files. | File | Resolution | Frames | FPS | |---|---|---|---| | ComfyUI_00001_.webp | 512 × 288 | 25 | 8 | | ComfyUI_00002_.webp | 512 × 288 | 25 | 8 | | ComfyUI_00003_.webp | 288 × 512 | 25 | 8 | During generation, GPU utilization was observed reaching 100% at one point. This indicates intensive GPU processing, although it does not establish that the GPU remained fully utilized throughout the entire inference process. Execution Times ComfyUI recorded the following execution times: | Execution | Recorded Time | |---|---:| | First | 16.72 seconds | | Second | 6.87 seconds | | Third | 12.01 seconds | The first execution may include additional model loading and initialization overhead. These values represent execution times reported by ComfyUI. They should not be interpreted as controlled benchmark results. Repeated tests under standardized conditions will be required to evaluate performance, consistency, and resource consumption. Demo — Portrait Video (9:16) The third video was generated at 288 × 512 pixels, with 25 frames and a frame rate of 8 FPS. The result was published on YouTube Shorts as a visual demonstration of the proof of concept. Watch the AI-generated video on YouTube Shorts  Demo 1 — Portrait video generated locally using ComfyUI and LTX-Video 2B on an RTX 3050 Laptop GPU with 6GB of VRAM. This demonstration provides a visual reference for the model's output, complementing the configuration and execution evidence. 9. A Real Issue: FFmpeg and Animated WebP After generating the videos, I encountered an issue while converting the output files to MP4. The first attempt used FFmpeg directly: ffmpeg -i input.webp \ -c:v libx264 \ -pix_fmt yuv420p \ output.mp4 However, the FFmpeg version available in the environment could not correctly decode the animated WebP files produced by ComfyUI. The observed messages included: skipping unsupported chunk: ANIM skipping unsupported chunk: ANMF image data not found The conversion process failed to produce valid MP4 videos. Subsequent validation also identified invalid media containers, including the error: moov atom not found This demonstrated an important lesson: the existence of a file with an .mp4 extension does not guarantee that it contains a valid, playable video. The container structure and media metadata must be validated. The original terminal screenshots were not preserved. Therefore, the error messages recorded during the experiment are included as textual evidence. 10. The Solution: Pillow + FFmpeg To work around the decoding limitation, I separated the conversion process into two stages. First, Pillow extracts individual frames from the animated WebP file. Then, FFmpeg encodes those frames into an H.264 MP4 video. The updated pipeline became: Animated WebP | v Pillow | v Temporary PNG Frames | v FFmpeg / libx264 | v H.264 MP4 I created a Python script to automate this conversion. The initial implementation uses Image.open() to read the animated WebP frames, TemporaryDirectory() to store temporary PNG files, and subprocess.run() to execute FFmpeg. A simplified example of the frame extraction process: from PIL import Image with Image.open("input.webp") as image: for index in range(image.n_frames): image.seek(index) frame = image.convert("RGB") frame.save(f"{index:05d}.png") The extracted frames can then be encoded using: ffmpeg \ -framerate 8 \ -i "%05d.png" \ -c:v libx264 \ -pix_fmt yuv420p \ -movflags +faststart \ output.mp4 The yuv420p pixel format improves compatibility with media players and online platforms. The +faststart option relocates MP4 metadata to support progressive playback more efficiently. The initial converter assumes a constant frame rate. Handling variable frame durations remains a potential improvement for a future version. 11. Validating the MP4 Files After implementing the conversion script, all three WebP files were successfully processed. The recorded output was: OK: ComfyUI_00001_.mp4 | 25 frames | 8.00 FPS OK: ComfyUI_00002_.mp4 | 25 frames | 8.00 FPS OK: ComfyUI_00003_.mp4 | 25 frames | 8.00 FPS I then used ffprobe to inspect the generated media metadata. The results were: | File | Resolution | FPS | Duration | File Size | |---|---|---|---|---| | ComfyUI_00001_.mp4 | 512×288 | 8 | 3.125 s | 43,349 bytes | | ComfyUI_00002_.mp4 | 512×288 | 8 | 3.125 s | 46,037 bytes | | ComfyUI_00003_.mp4 | 288×512 | 8 | 3.125 s | 157,615 bytes | Validation confirmed that all three MP4 files contained resolution, frame rate, and duration metadata consistent with the generation settings. The third file confirmed successful export in a portrait 9:16 format. The results are preserved as textual evidence and a technical table, without requiring additional terminal screenshots. 12. Results and Limitations Phase 00 successfully validated the fundamental components of a local AI video generation pipeline. Confirmed Results - ComfyUI running locally with CUDA support. - LTX-Video 2B loaded and used for generation. - Three animated WebP outputs generated. - Landscape and portrait generation tested. - Successful conversion to H.264 MP4. - Output metadata verified using ffprobe. - Three execution times recorded by ComfyUI. - A portrait video published as a visual demonstration. Identified Limitations - Low output resolution. - Video duration limited to approximately three seconds in the initial tests. - High GPU resource utilization. - No controlled performance benchmark yet. - Video conversion still handled by a separate script. - Generation workflow still operated manually. - Visual quality not yet evaluated using systematic criteria. These limitations help define the scope of future development. The results do not imply that every video configuration, resolution, or model will run successfully on a 6GB GPU. They demonstrate that one specific configuration was successfully validated on this hardware. 13. Next Phase: Python Automation With the proof of concept completed, the next objective is to transform the manual workflow into an automated pipeline. Phase 01 will focus on using the ComfyUI API to: 1. Receive text prompts. 2. Load previously validated workflows. 3. Submit generation jobs. 4. Monitor execution status. 5. Identify and retrieve generated outputs. 6. Automatically convert results into MP4. 7. Record execution metadata, errors, and processing times. The proposed architecture is: Python CLI | v Workflow Manager | v ComfyUI API | v LTX-Video 2B | v Output Manager | v Media Converter | v MP4 + Execution Metadata This approach will allow the project to evolve without modifying ComfyUI's internal source code. 14. Conclusion This experiment demonstrated that short AI-generated videos can be produced locally using an RTX 3050 Laptop GPU with 6GB of VRAM, provided that the workload is configured within the available hardware constraints. It also highlighted that building a functional AI video generation pipeline involves much more than running a generative model. It requires understanding the inference environment, managing hardware resources, interpreting execution errors, processing intermediate media formats, and validating the resulting artifacts. The animated WebP conversion issue was particularly valuable: a seemingly simple failure demonstrated the importance of separating responsibilities between tools and verifying outputs rather than assuming success. Phase 00 established the technical foundation of JC IA Video. The next step is to transform this proof of concept into a reproducible and automated pipeline. 15. Project Repository and Availability JC IA Video is being developed incrementally, with each phase documenting technical decisions, implementation details, limitations, and validation results. The project currently maintains its source code, workflows, scripts, and technical documentation in a local Git repository. The GitHub repository will be made publicly available in the future, once the project reaches an appropriate level of organization, reproducibility, and documentation. Future articles in this series will cover the implementation of the automation pipeline, resource optimization, and additional video processing capabilities. --- References - ComfyUI — Official GitHub Repository - LTX-Video — Official GitHub Repository - LTX-Video — Hugging Face - PyTorch — Official Documentation - Pillow — Official Documentation - FFmpeg — Official Documentation - FFprobe — Official Documentation --- This is the first article in the JC IA Video — Local AI Video Generation series, documenting the incremental development of a local AI video generation pipeline using Python, ComfyUI, and open-source tools.
@ltx_io 출처는 LTX Desktop 공식 릴리스 노트와 X 발표예요. github.com/Lightricks/LTX-Desktop/releases/tag/v1… x.com
Real-looking people in a product video, for $0, on a MacBook: I tested 5 methods > TL;DR > - I made the same 30-second story five ways to find the best free, local way to put real-looking people in a product video: free stock, AI images (SDXL), AI video (LTX-Video), Blender 3D and a lip-synced AI narrator. > - Everything ran on one MacBook Pro (M1 Max, 64 GB), cost $0, and used about 28 GB of models. > - The winner is AI images with SDXL: realistic, consistent characters at ~2 min per shot on the Mac GPU. > - AI video (LTX-Video) is the only method where people truly move, but it took 11–26 min per 5-second clip, and crowds and big motion melt. > - Surprise: the video model generates a clip in ~30 s. The other 95% of the time is decoding plus memory swap. > - Everything is open: live side-by-side dashboard · repo with inputs, outputs, scripts and logs · full report !Same story, five ways. Rows: free stock, AI images, AI video, Blender 3D, talking narrator Same 30-second story, five ways. Rows top to bottom: free stock · AI images (SDXL) · AI video (LTX-Video) · Blender 3D · talking narrator (SadTalker). Columns are the five shots. ▶ Watch all five videos in sync: side-by-side dashboard · each video embedded on the original post Who this is for You make product demos, explainers or launch videos with code, an editor or a tool like Remotion or HyperFrames. Your app screens look great, but there are no humans: no customer, no user, no story. You want people in the video, you don't want to pay for footage or an AI video subscription, and you have a decent Mac. By the end you'll know: - which of the five free methods to use for which shot; - what each one really costs in time on Apple Silicon, and why; - the exact settings, prompts and patches that made them work, including the errors you'll hit and their fixes. The problem My explainer videos had a recurring hole. The product screens were sharp, but every "real world" moment (a candidate arriving, a phone buzzing, an interview) was a silhouette or an icon. Viewers connect with people. The usual answers are paid stock footage, a paid AI video tool, or filming. I wanted to know how far free and local gets you in 2026. How I tested (methodology) One story, five methods; only the people change. The story: Aarav gets a QR call letter for a walk-in interview → scans in at reception → waits in a lounge until his phone buzzes → interviews → his record is handed off to the company's HR system. That's 5 shots, 30.4 seconds in total. Fairness controls. Every version shares: - the same storyboard and shot timings (6.5 / 5.3 / 6.5 / 5.7 / 6.4 s); - the same narration (Kokoro TTS, voice am_michael, generated locally); - the same app overlay cards and the same music bed; - the same assembly template, rendered with HyperFrames (HTML + GSAP → MP4) at 1080p. Hardware. MacBook Pro, Apple M1 Max: 10-core CPU (8 performance + 2 efficiency), 32-core GPU, 64 GB of unified memory, macOS 26.5. There's no NVIDIA GPU, so "GPU" means the Apple GPU through PyTorch's mps backend or Blender's Metal renderer. I left my normal apps open (an IDE, a Linux VM, an editor), which turned out to matter. What I measured. - Wall-clock time per step, from logs. - Which device each step ran on. - Memory and swap. - Seven quality criteria scored 1–5: realism, character consistency, motion, story fit, speed, reliability and licence clarity. Prerequisites | You need | Why | |---|---| | Apple Silicon Mac, ideally 32 GB+ of RAM | The AI video model needs ~25 GB of working memory in fp32 | | ~30 GB of free disk | Model weights | | Python 3.10 and 3.11, uv, ffmpeg | Two separate environments; SadTalker needs old pinned libraries | | Blender 4.5 | Method 4. Use the blender.org DMG; the Homebrew cask asks for sudo | | Node 18+ | npx hyperframes for the final assembly (any editor works) | The models and their download sizes: | Model | Used for | Size | Licence | |---|---|---|---| | RealVisXL V4 (SDXL, fp16) + fp16-fix VAE | AI images | 7.3 GB | Open RAIL++-M | | LTX-Video 0.9.5 transformer + VAE | AI video | 6.3 GB | Open RAIL-M | | T5-XXL encoder, bf16 copy | LTX text encoder | 9.5 GB (instead of 19 GB) | Apache-2.0 | | SadTalker + GFPGAN weights | Narrator | 1.7 GB | Apache-2.0 | | Quaternius animated characters | Blender | small | CC0 | Method 1: free stock photos How it works. Search Wikimedia Commons (its API needs no key) and Openverse for each shot. Filter licences to CC0, CC BY and CC BY-SA (never NC or ND), crop to 16:9, and add a slow camera push. curl -s "https://commons.wikimedia.org/w/api.php?action=query&list=search&srnamespace=6&srsearch=filetype:video%20job%20interview&format=json" Result. No usable free video exists for offices, receptions or interviews: the hits are under 720p or are news footage. So all five shots became photos. Each shot shows a different person, most aren't Indian, the "lounge" is a bus terminal, and in shot 1 we only see the back of a head. Time: ~22 min of searching and reviewing. GPU: none. Verdict: real, and legally clean with attribution, but it can't tell your story with your character. Pexels has much better free video, but needs an API key. Method 2: AI images with SDXL (the winner) How it works. Write one fixed character description and reuse it in every shot prompt. Fix the seeds, generate 2 options per shot, and pick the best. AARAV = "Aarav, a 26 year old Indian man, short neat black hair, light stubble, navy blazer over a white shirt" STYLE = "photorealistic, candid documentary photo, 35mm lens, natural light, shallow depth of field, ..." vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) pipe = StableDiffusionXLPipeline.from_pretrained("SG161222/RealVisXL_V4.0", vae=vae, torch_dtype=torch.float16, variant="fp16").to("mps") pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True) img = pipe(f"{AARAV}, sitting on a sofa ... looking down at his smartphone, {STYLE}", negative_prompt=NEG, width=1344, height=768, num_inference_steps=30, guidance_scale=5.5, generator=torch.Generator("cpu").manual_seed(1000)).images[0] The fp16-fix VAE matters: the stock SDXL VAE in fp16 on Apple GPUs tends to decode black or NaN images. !All 10 SDXL candidates, two per shot All 10 candidates, two per shot. Choosing took seconds. Result. These look like candid office photos, and Aarav is recognisably the same person in most shots. Weaknesses: - In the busy interview scene his identity drifted (glasses and a beard appeared). - Prompts are followed loosely: no receptionist actually scans his phone. - The images are stills, so motion comes only from the camera. Time: 108–131 s per 1344×768 image (average 119 s), so 16 images took ~28 min. GPU: yes (MPS, fp16). Verdict: ⭐ the best realism per minute. Method 3: AI video with LTX-Video (alive, but fragile) How it works. Animate the method 2 stills with LTX-Video 0.9.5 image-to-video, prompting only the motion. Two memory tricks are essential on a Mac: pipe = LTXImageToVideoPipeline.from_pretrained("Lightricks/LTX-Video-0.9.5", text_encoder=None, torch_dtype=torch.float32) # 1) encode every prompt once on the CPU, then drop the 19 GB text encoder before generating pipe.text_encoder = T5EncoderModel.from_pretrained("city96/t5-v1_1-xxl-encoder-bf16", torch_dtype=torch.float32) emb = {s: pipe.encode_prompt(MOTION[s], NEG, do_classifier_free_guidance=True, device="cpu") for s in shots} pipe.text_encoder = None; gc.collect() pipe.to("mps"); pipe.vae.enable_tiling() # 2) tiled decoding pe, pm, ne, nm = (t.to("mps") for t in emb["s1"]) frames = pipe(image=still, prompt_embeds=pe, prompt_attention_mask=pm, negative_prompt_embeds=ne, negative_prompt_attention_mask=nm, width=768, height=448, num_frames=121, num_inference_steps=30, guidance_scale=3.0, generator=torch.Generator("mps").manual_seed(42)).frames[0] Result. For one person with small motion it's genuinely convincing: Aarav smiles at his phone, and in the interview he talks and gestures while the panel listens. !Interview clip: natural talking and gesturing The interview clip at 0 s, 2.5 s and 5 s: natural talking and gesturing. Crowds and big motion break it. The busy reception melts into a blur after ~3 s, and walking out of the building warps his body. !Reception clip: fine at 0s, morphing at 2.5s, melted at 5s The reception clip at 0 s, 2.5 s and 5 s: the crowd melts. I rescued those two clips by keeping the clean first ~3 s and playing them forward then backward (ffmpeg ... trim,split,reverse,concat). The raw clips are in the repo. Time: - Quick test (49 frames, 20 steps): 278 s. - Full clips (121 frames ≈ 5 s, 30 steps): 811, 681, 1,568, 1,381 and 804 s, ~87 min for all five. GPU: yes (MPS, fp32; see the pitfalls). Verdict: use it for 2–3 hero moments, not a whole film. Method 4: Blender 3D with free rigged characters How it works. - Download CC0 rigged and animated characters (Quaternius, via Poly Pizza; no login). - Recolour them to match the cast and build sets from primitives. - Script every shot in Blender Python and render headless with EEVEE on the GPU: ~/Applications/Blender.app/Contents/MacOS/Blender -b -P shots.py -- s1 full 1920 1080 24 Result. Every shot does exactly what the storyboard asks. He holds the phone to the scanner, the phone buzzes, he walks out, and nobody is stuck in a T-pose. But it looks like a video game: - faces have no expressions and there's no lip-sync; - the same three base models are reused for everyone; - the sets are untextured. Realistic free 3D humans are hard to get: Mixamo needs an Adobe login, and MetaHuman and Character Creator aren't free. Time: ~35 min in total, of which 14 min was rendering 878 frames at 1080p. GPU: yes (Metal, EEVEE). Verdict: total control and consistency, but no realism. Method 5: a talking AI narrator with SadTalker How it works. 1. Generate a fictional presenter face with SDXL (method 2). 2. Lip-sync it to the full narration with SadTalker. 3. Clean up the face with GFPGAN. 4. Show it as a corner bubble. python inference.py --driven_audio narration.wav --source_image presenter.png \ --still --preprocess crop --enhancer gfpgan --size 256 Result. The lip-sync is frame-accurate across the full 30 s, but only the head moves, and at 512×512 it only works small. Match the face to the voice: my first presenter was a woman against a male voice, and it looked wrong. Time: - On the CPU: 24 s per step, projected at ~2.5 h. - On the GPU after a patch: 11.7 min, about 13× faster. - The GFPGAN clean-up then took 39 min on the CPU, because GFPGAN has no Apple GPU path. GPU: partly. Verdict: a nice guide or presenter; it can't carry the story. Results | | 1 · Stock | 2 · AI images | 3 · AI video | 4 · Blender | 5 · Narrator | |---|---|---|---|---|---| | Realism | 5 | 4 | 4 | 2 | 3 | | Same character throughout | 1 | 4 | 4 | 5 | 5 | | People actually move | 1 | 1 | 4 | 3 | 2 | | Matches the storyboard | 2 | 4 | 3 | 4 | n/a | | Speed | 4 | 4 | 1 | 3 | 2 | | Reliability | 4 | 5 | 2 | 4 | 3 | | Licence clarity | 3 | 4 | 3 | 5 | 4 | | Average | 2.9 | 3.7 | 3.0 | 3.7 | 3.2 | | Time for 30 s | ~22 min | ~28 min | ~87 min | ~35 min | ~52 min | | Ran on | CPU | GPU | GPU | GPU | GPU + CPU | Blender ties SDXL on the average only because consistency and licence clarity weigh as much as realism. If "real-looking people" is the goal, SDXL wins clearly. Where the time actually goes on a Mac | Step | Device | Measured | |---|---|---| | SDXL image, 1344×768, 30 steps | GPU | 108–131 s | | LTX-Video: generating a 5 s clip | GPU | 27–29 s (~1 step/s) | | LTX-Video: decoding + swap for that clip | GPU + swap | 650–1,540 s | | Blender, 878 frames at 1080p | GPU (Metal) | 14 min | | SadTalker face renderer, 380 steps | CPU → GPU | 24.1 → 1.85 s per step | | GFPGAN clean-up, 760 frames | CPU | ~3.1 s per frame | | Assembling one 30 s video | CPU + headless Chrome | 63–93 s | The biggest surprise: the AI video model isn't slow at generating. 95% of each clip's time went to decoding the frames while macOS swapped up to 32 GB to disk. The T5 encoder alone is 19 GB in fp32, and my IDE and VM were open. Unloading T5 after encoding cut the swap to 17.5 GB. Closing other apps is the single biggest speed-up. Pitfalls and fixes | Symptom | Cause | Fix | |---|---|---| | Input type (float) and bias type (c10::BFloat16) should be the same | M1 has no native bf16 | Load LTX-Video in fp32; it costs ~2× the memory | | Expected a 'mps' device type for generator but found 'cpu' | A CPU random generator fed to an MPS pipeline | torch.Generator("mps") | | SadTalker: Conv3D is not supported on MPS | Its pinned PyTorch 2.1 | Upgrade to PyTorch 2.5.1 and apply this 3-file patch | | SadTalker: invalid type: 'torch.mps.FloatTensor' | CUDA-style .type() calls | Covered by the same patch | | No module named 'pkg_resources' | New setuptools | uv pip install "setuptools<70" | | basicsr import error on functional_tensor | New torchvision | Change the import to torchvision.transforms.functional | | SDXL produces black or NaN images in fp16 | The stock VAE overflows | Use madebyollin/sdxl-vae-fp16-fix | | A 30 GB download takes hours | One slow stream | Download models in parallel with hf download, and use the bf16 T5 copy (9.5 GB, not 19 GB) | | brew install --cask blender asks for a password | The cask needs sudo | Copy Blender.app from the official DMG into ~/Applications | Which method should you use? - You need a recurring, realistic main character → SDXL stills with a fixed character description and seeds, plus camera motion. Generate 2–3 options per shot. - One hero moment needs real motion (a smile, a nod, someone talking) → LTX-Video on that SDXL still. One person, small motion, keep the clean seconds. - The shot has a crowd or a person walking → keep it a still. Today's local video models melt here. - You need exact staging (someone holds a phone to a scanner at a precise moment) → Blender, if a game look is acceptable. - You want a presenter face → a SadTalker bubble with a voice-matched face. - You need real people and don't care who → stock, ideally Pexels with a free API key. Limitations - One machine and one run per clip. No repeated runs, so no variance numbers. An M3 or M4, or a Mac with less load, will be faster. - The scores are my judgement, not a user study. The repo has every output so you can judge for yourself. - I picked the best of 2 seeds for the stills and trimmed two AI video clips. That's realistic practice, but it is curation. - On an NVIDIA GPU (CUDA, bf16) the AI video steps would be many times faster; this is a Mac-specific result. Reproduce it git clone https://github.com/kartikjindal/local-character-video-5-methods-benchmark && cd local-character-video-5-methods-benchmark uv venv .venv-ai --python 3.11 && uv pip install --python .venv-ai/bin/python -r requirements-ai.txt ./download-models.sh # ~23 GB, in parallel .venv-ai/bin/python gen_stills.py --seeds 2 # method 2 .venv-ai/bin/python gen_video.py --frames 121 --steps 30 # method 3 python3 -m http.server 8777 # dashboard: compare all five The SadTalker and Blender steps are in the README. Conclusion Free and local gets you surprisingly far. SDXL stills give realistic, consistent characters for ~2 minutes of GPU time per shot, and that should be the backbone. Local AI video is real but narrow: magic for one person making small movements, unusable for crowds, and slow on a Mac mainly because of memory, not compute. Everything here cost $0 and about 28 GB of disk. What would you try next? I'm curious whether a newer local video model, or a Mac with more memory headroom, changes the AI video verdict. If you've run LTX-Video, Wan or HunyuanVideo on Apple Silicon, share your timings in the comments. --- Resources - ▶ Live side-by-side dashboard (all five videos in sync) - 📦 Repo: inputs, outputs, scripts, logs - 📄 Full report with every measured number - Tools: RealVisXL V4 · LTX-Video · SadTalker · GFPGAN · Blender · Quaternius · Kokoro TTS · HyperFrames · ffmpeg All AI faces in this post are fictional.
GPU-ல் எரியாம video பண்ண முடியுது.. LTX-Video distilled model 8 step-ல் HD. 2B model real-time, consumer GPU-ல். Text/image-to-video, open source. நீங்கள் எதைப் பற்றி பண்ணுவீங்க ? Lightricks/LTX-Video Instagram-லும் பின்தொடருங்கள்: instagram.com
LTX Desktop is an open-source desktop app for AI video generation with LTX models. It runs locally on supported Windows and Linux NVIDIA GPUs and on Apple Silicon Macs. Generation runs on your own hardware. Lightricks/ltx-desktop t.co
5 GitHub Repos for AI Video Creators (and What Each One Makes) All 5 links 👇 (checked Oct 22, 2026) 1. LTX-2: photo + prompt → short moving clip Lightricks/LTX-2 Try it: official online demo huggingface.co (a free Hugging Face account gets a small daily allowance). Running it locally needs a powerful NVIDIA GPU. LTX-2 Community License: free under US$10M annual revenue; not a standard open-source licence. 2. OpenMontage: describe a video, and your AI coding assistant finds footage and cuts the edit calesthio/OpenMontage Try it: clone it, run "make setup", open the folder in Claude Code, Cursor or Codex and describe your video. Free archive footage works without API keys; paid AI providers are optional. Our run needed some manual fixes. AGPL-3.0. 3. HyperFrames: animation written as a web page, rendered to MP4 (this video was built with it) heygen-com/hyperframes Try it: install Node.js 22+ and FFmpeg, then run "npx hyperframes init my-video". Apache-2.0. 4. Remotion: one video template in React, many versions from your data remotion-dev/remotion Try it: run "npx create-video@latest". Free for individuals, non-profits and companies of up to 3 people; larger companies need a paid licence. 5. Manim: charts and diagrams that draw themselves, from Python code ManimCommunity/manim Try it in your browser: try.manim.community . MIT licence. The chart in the video uses sample data. Follow AI With MTalha for more AI tools. What we did: we made every example in this Short ourselves on Oct 22, 2026. The LTX-2 clips came from its official online demo and are shown without sound. The other four ran on a Mac. Footage and media • Balloon photo: "Hot Air Balloon over Lake" by Bonnie Moreland (CC0), commons.wikimedia.org • Archive shots (OpenMontage example and two Remotion cards): NASA, public domain, images.nasa.gov • Remotion card 2: our own 3D render. The episode titles on the cards are mock examples. • Narration is AI-generated with my own voice clone. #GitHub #AIVideo #VideoCreators #MotionGraphics #OpenSource
LTX Desktop 1.3.1 can turn a video subject into a transparent cutout. No green screen needed. That opens up some fun ways to mix clips. github.com/Lightricks/LTX-Desktop/releases/tag/v1…
LTX Desktop 1.3.0 is out! A big update to your local video workflow: the app has a redesigned Home page and sidebar, quick search, easier setup, and a new Assistant that can turn a chat into queued generations. Create videos locally with Text-to-Video, Image-to-Video, and Audio-to-Video. Retake or Extend clips, try IC-LoRAs like Ingredients, Day to Night, Deblur, and Water Simulation, then queue jobs in the background and reorder or cancel them. You can also use Remote from your phone or another browser on the same network. Generations and assets persist across restarts. The update adds fast-preview resolutions, more duration, frame-rate, and aspect-ratio options, prompt enhancement controls, and seed settings. It also includes installation and memory fixes across Mac, Windows, and Linux. Download:LTX Desktop ·Full changelog
Maxwell made LTX-2.5 generate 4K video nearly twice as fast, with the same output. Seems to be available on LTX-2 v1.5.0 based on the Changelog github.com/Lightricks/LTX-2/blob/main/CHANGELOG.md > Performance improvements, in collaboration with doubleAI (doubleai.com)