TencentARC

From ImportError to Video: Bringing Looking Ahead to Windows Looking Ahead's 1.3B image-to-video example stopped on my Windows machine before inference started: ModuleNotFoundError: No module named 'fcntl' The machine had an RTX 5090. FlashAttention 4 was installed. Neither was enough to get past the import. My fix, TencentARC/Looking-Ahead PR #2, was merged on October 23, 2026 (UTC). With the patch, the tested 1.3B example produced 80 frames at 768×432 on Windows 11, using PyTorch's scaled dot-product attention fallback. The change spans six files. The interesting part is how three separate assumptions had to be removed before the fallback actually worked end to end. Why I wanted to run Looking Ahead Looking Ahead is a video-generation research project from researchers at Nanjing University, Tencent ARC Lab, and Zhejiang University. Its method, Future Chunk Prediction (FCP), addresses a shortcut in autoregressive video training: neighboring video chunks can be similar enough that a model relies on local continuity without learning much about how a scene evolves. FCP adds auxiliary training branches that predict more distant future chunks from the current chunk's intermediate features, without seeing the intervening chunks. Those branches are removed at inference. The authors report improvements in physical consistency; the project page explains the method and shows their comparisons. The released code includes 1.3B and 5B models. Its documented test environment was Linux with an NVIDIA H200. My contribution makes the tested Windows inference path accessible through the upstream code. It does not change FCP, retrain the model, or establish Windows support for every configuration. That is why this merge matters to me: the compatibility fix now lives in the research repository, where the next person trying the model can receive it. 1. Installed does not mean importable The attention modules used is_flash_attn_4_available() before importing flash_attn.cute. In the tested environment, that availability check reported the installed package. Importing it still failed: its quack dependency imported fcntl, a Unix-specific module unavailable on Windows. The merged code attempts the import and treats an ImportError as an unavailable optional backend: FLASH_ATTN_4_AVAILABLE = is_flash_attn_4_available() if FLASH_ATTN_4_AVAILABLE: try: import flash_attn.cute except ImportError as exc: FLASH_ATTN_4_AVAILABLE = False warn_flash_attention_fallback(type(exc).__name__, str(exc)) Both attention modules receive this handling. A shared helper logs the selected backend and reports each distinct import failure once. The warning explicitly says that performance and numerical results may differ. That matters because a run that silently switches backend is harder to reproduce or compare. The helper receives the exception type and message as strings. It does not cache the exception object and its traceback just to deduplicate a warning. This check establishes importability, not a guarantee that every subsequent kernel call will work on every device. It also catches ImportError specifically; it does not convert arbitrary runtime failures into silent fallbacks. 2. Cross-attention bypassed the fallback Getting past the import exposed the next problem. The code already had an attention() function that selected between FlashAttention 4 and PyTorch SDPA. But cross-attention called the FA4-only flash_attention() directly. Removing FA4 from the environment therefore avoided the import error while leaving an assertion waiting later in the run. The relevant change is small: - x = flash_attention(q, k, v, k_lens=context_lens) + x = attention(q, k, v, k_lens=context_lens) Now cross-attention uses the same backend-selection boundary. There is a semantic condition behind that edit. In the call path covered by the PR, context_lens is None, so the fallback computes full attention. The SDPA branch does not implement arbitrary variable-length padding masks. Routing a masked attention call through it would need additional work. When FA4 is available, the dispatcher forwards the same arguments to the existing FA4 implementation. The general lesson is to trace the callers. An optional backend is only optional if the execution paths that need it actually pass through the fallback. 3. Model loading had its own Unix dependency The remaining fcntl dependency was outside attention. The project stages base-model files and checkpoints into a local cache, using fcntl for file locking. The default staging path brought the same platform assumption back into the run. The patch makes the import optional in both loading utilities. If fcntl is unavailable, staging is skipped and the model or checkpoint is loaded from its original path. This preserves the underlying operation—reading the weights—without pretending that the Unix locking mechanism exists. It also avoids adding a new cross-platform lock implementation for a cache optimization. On systems where fcntl exists, the existing staging path remains available. On systems where FA4 imports successfully, that attention path remains available too. What the Windows run established The PR records the following setup: - Windows 11 and RTX 5090. - Python 3.12 and PyTorch 2.10.0+cu128. - Other package versions from the project's uv.lock. - FCP 1.3B EMA checkpoint, sailboat example, seed 42, and a five-second duration request. Before the patch, the run stopped at import. With the patch and SDPA, it produced 80 frames at 768×432, with 209 seconds of sampling, 231 seconds for the whole run, and 16.4 GiB peak allocated GPU memory. One detail belongs beside those results: all reported runs used load_diffusion_pretrained=false. The PR explains that the EMA checkpoint strictly replaces every backbone weight, so the base DiT initialization does not determine the final output. This was not a claim that every model file or loading step could be omitted. These numbers describe one tested example and environment. They are not an end-to-end validation of training, every checkpoint, or every GPU. Preserving FA4 and comparing SDPA are different checks The fallback needed to work, but the original fast path also needed a regression check. For a controlled experiment, I made FA4 importable using triton-windows and a test-only no-op fcntl module. That shim is not in the PR and is not an installation recommendation—it does not provide real file locking. With FA4 available, the original branch and the patched branch produced bit-identical decoded frames. That comparison checks whether the patch changed the existing FA4 path. Comparing SDPA against FA4 answers a different question. On the same example, the first conditioning frame was identical, while frame PSNR was 33.5 dB on average and 31.6 dB at minimum. The generated results were not bit-identical. FA4 sampling took 154 seconds, versus 209 seconds for SDPA. The fallback made the example runnable with a measurable speed tradeoff. It did not reproduce FA4's exact numerical output. These are comparisons recorded in the PR's validation notes, not new experiments performed for this article. The 5B follow-up found the next boundary I also checked the 5B model with the paper-fan example and documented the result in a follow-up comment. Sampling completed: 30 latent frames in 231 seconds, with 24.3 GiB peak allocated GPU memory. The immediately following VAE decode did not fit in the same process on the tested 32 GB GPU / 64 GB RAM machine. The generator and both KV caches remained on the GPU; Windows commit charge passed 104 GiB, and I stopped the run. Decoding those saved latents separately took 8.2 GiB and 11 seconds. That result points to a phase-lifetime and memory-management problem, beyond the portability fix in this PR. So the precise outcome is: the 1.3B example completed end to end; the 5B example completed sampling, with decode checked in a separate process. The merge does not include a solution for the combined 5B memory peak. What I would carry into the next portability fix This patch connected three things: optional-backend detection, the callers that select that backend, and the platform-specific optimization in model loading. For another research implementation, I would follow the same checks: attempt the optional import, trace every caller, verify that the fallback preserves the required attention semantics, log the selected path, and test the complete workload. A successful import is only the beginning. The merged changes are small enough to read alongside the model's setup and inference instructions. FCP and the released models are the research authors' work; my contribution is getting this tested inference path through Windows without replacing the existing FA4 path. Have you encountered an “optional” dependency that turned out to be mandatory through a different call site? Drafted with AI assistance from the merged diff, project documentation, and recorded PR validation results. Merge commit: 15f90becc4d37f4f8c74cb0c868b6d855a7c605d.

Player-1: Learning to Play from Synthetic Experience Mingyu Ouyang, Yiran Wang, Siyuan Hu, Xingyilang Yin, Huiqiang Sun, Guangzhi Wang, Junfu Pu, Cheng Cheng, Honghao Chen, Ying Shan, … arxiv.org [𝚌𝚜.𝙲𝚅 𝚌𝚜.𝙰𝙸 𝚌𝚜.𝙷𝙲] 💬Code: TencentARC/Player-1 t.co

@Teknium That is what I did. Using all open source programs like TencentARC/Pixal3D and others. I am up to GTA V levels of gameplay.

Introducing Player-1: empowering general VLMs and computer-use agents to master complex gameplay by learning from synthetic experience. @OuyyyangMingyu Hugging Face Daily: huggingface.co GitHub: TencentARC/Player-1 Project Page: player-1-project.github.io · Challenge: Games have become a vital proving ground for AI models to reason, plan, and operate. However, there is no mature open-source framework to boost gameplay capabilities of general VLMs or computer-use agents. Moreover, interactive Game RL remains nearly uncharted. . What: The first framework to unlock Game RL via synthetic environments, releasing a full suite of Player-1 model family across 2B, 4B, 9B, and 35B model sizes. ·How: Bridging a unified keyboard-mouse game action space with supervised warm-up and PPO training across 16 controllable 2D/3D synthetic environments with randomized visuals, physics, and verifiable rewards. . Result: Substantially surpasses existing baselines in task success and out-of-distribution generalization. Without game-specific training, Player-1 generalizes directly to diverse unseen games and AAA titles, e.g., Watch Dogs, Forza Horizon 6, and GTA V (see more demos later!).

Player-1: Learning to Play from Synthetic Experience Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain. We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience. We extend the computer-use action space with relative pointer motion, sustained inputs, and composite actions. For supervised initialization, we fine-tune computer-use models on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning. We construct Player-1-Gym, sixteen synthetic 2D and 3D environments designed around control demands shared across games. Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment reinforcement learning with PPO. The resulting Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points, respectively. Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld. Further analyses show that broader synthetic environment coverage improves transfer, while the use of composite actions that combine multiple keyboard and mouse inputs increases during RL. We demonstrate that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite. Comments: Project Page: player-1-project.github.io and code: TencentARC/Player-1

Open-source Multics: Generative Multi-Player World Model with Hierarchical Communication ✨ Cross-player consistency, player-specific controllability & interference robustness 📈 Train on 4 → 6/8/10 players karine-huang.github.io huggingface.co TencentARC/Multics t.co

GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay Modern video games provide a measurable testbed for AI models, combining abilities of visual understanding, instruction decomposition, goal planning, and precise action control over multiple temporal horizons. Existing datasets and benchmarks, however, either cover a narrow range of games, lack language instructions, or rely on high-variance online rollouts. To address these challenges, we introduce GameHorizon, a unified data and evaluation suite that measures gameplay capabilities at different horizons for diverse model families. GameHorizon Suite consists of three components. First, GameHorizon-Annotator is a scalable and automated annotation pipeline for multi-horizon instructions. Second, utilizing the pipeline, we construct GameHorizon-Data, the first large-scale AAA gameplay dataset with temporally aligned videos, player actions, and multi-horizon instructions. It comprises 5,000 hours of recordings from 21 games, collected by 100 human expert players. Third, we build GameHorizon-Bench with reproducible offline and stepwise online testing. The offline track enables reproducible evaluation using thousands of standardized questions organized into three primary tasks and a series of diagnostic variants, while the online track tests whether offline scores reflect actual gameplay capabilities and localizes failures to specific steps within long-horizon gameplay. Based on our GameHorizon Suite, we evaluate 47 models through more than one million model invocations, revealing a meaningful hierarchy of task difficulty and pronounced differences in model capabilities. Our work can provide a standardized yardstick for evaluating gameplay capabilities across horizons and model families. We will release our dataset, annotator, and benchmark to facilitate future research. Comments: We will release our dataset, annotator, and benchmark to facilitate future research. Github Repo: TencentARC/GameHorizon & Project Page: gamehorizon-suite.github.io

Paper introduction WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory Project page: drexubery.github.io Paper: arxiv.org Code: TencentARC/WorldCrafter [1/N] t.co