🔗 GitHub: AIGeeksGroup/DragMesh
📚 プロジェクトページ: aigeeksgroup.github.io
📚 論文(arXiv): arxiv.org
出典:mp.weixin.qq.com
✍️ by ツキタマ(出典を突き合わせて確認するキュレーター視点)
14 views
InteractAgent: Agentic Human Motion Interaction with Memory-Augmented Reflection
Long-horizon human-scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors. Text-driven motion models are largely scene-agnostic and lack closed-loop correction, while geometry-conditioned approaches depend on paired scene-motion training or explicit 3D preprocessing that limits generalization to novel layouts and open-vocabulary goals. We present InteractAgent, an agentic multimodal LLM system for long-horizon human-scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor. InteractAgent integrates Online Observation for step-wise recovery decisions (continue/redo/replan) and Memory-Augmented Reflection for multi-round plan refinement with adaptive stopping. On UniHSI, InteractAgent improves task success and reduces penetration compared with training-free baselines. Code: AIGeeksGroup/InteractAgent.
Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation
Haoyu Zhang, Siming Li, Zeyu Zhang, Yi Zhang, Tat-Seng Chua, Hao Tang
arxiv.org [𝚌𝚜.𝚁𝙾 𝚌𝚜.𝙲𝚅]
💬Code: AIGeeksGroup/Code2Worlds-2, t.co
104 views
Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation
Recent coding large language models (LLMs) can synthesize dynamic 4D worlds from natural language, but the resulting content remains a visual artifact. It lacks physical semantics such as the collision geometry, mass, friction, inertia, and joint structure, required to enable it to simulate the physical world, and its dynamics are scripted to look plausible rather than designed to execute. What is needed is an executable world: one that a physics engine can load and step, resolving contact physically. We present Code2Worlds-2, an extension of Code2Worlds that turns 4D world generation into an embodied world simulation. We keep the original dual-stream generation pipeline intact and append two modules. First, a SimReady Scene Compiler that turns a generated visual scene into a simulator-compatible representation: it assigns collision meshes via convex decomposition, estimates physical attributes with a vision-language model, infers a lightweight support graph, instantiates the engine's contact, friction, and dynamics constraints, and applies a gravity settling pass, before exporting the scene as MuJoCo MJCF. Second, a Simulator-in-the-Loop 4D Refiner that implements an automated agent harness-a compile, simulate, critique and refine loop-that converts visual dynamics into executable rigid and articulated dynamics and evolves the physical settings rather than only the code, replacing the purely visual motion critic with a simulator critic that inspects penetration depth, collision pairs, stability, contact forces, joint limits, and robot reachability. We further propose SimCode4D, a state-based benchmark with five task templates for evaluating embodied simulation. Experiments on SimCode4D and across natural, indoor, industrial, urban, and service scenes show that Code2Worlds-2 produces physically plausible, simulator-compatible, and robot-interactable scenes.
Comments: Code: AIGeeksGroup/Code2Worlds-2, Website: aigeeksgroup.github.io
GENERATE ACCURATE DEPTH MAPS FROM IMAGES! 👁️
Aigeeksgroup AnydepthAnyDepth generates depth maps from single images using the Simple Depth Transformer decoder.
AIGeeksGroup/AnyDepth t.co
1 view
Code2Games: Enabling Coding Agents for Gaming World Generation
Wei Wu, Ziyang Xu, Zeyu Zhang, Yang Zhao, Hao Tang
arxiv.org [𝚌𝚜.𝙲𝚅]
💬Code: AIGeeksGroup/Code2Games, t.co
60 views
PWM: Personalized World Models with Online Reinforcement Learning
Zhexin Lou, Guancheng Lu, Zeyu Zhang, Yi Zhang, Yang Zhao, Hao Tang
arxiv.org [𝚌𝚜.𝙲𝚅]
💬Code: AIGeeksGroup/PWM, t.co
61 views
RFPO: Rectified Flow Policy Optimization for Embodied Control
Flow-based policies provide an expressive framework for continuous robot control, but their iterative ODE integration incurs substantial inference cost. Naively reducing the integration budget can severely degrade control, since policies optimized under full-step execution are not explicitly constrained to remain reliable under coarse numerical integration. We refer to this mismatch as the few-step discretization gap. To address this problem, we introduce RFPO, a flow-policy optimization framework for reliable few-step execution. Reward-aware online Reflow rectifies student-induced transport paths during on-policy learning, making the resulting policy more robust to coarse integration. A frozen Gaussian PPO controller supplies complementary action-space supervision at full and intermediate integration budgets, while the deployed policy remains a single flow student executed with one Euler step. Across Unitree Go2, Boston Dynamics Spot, Unitree H1, and Unitree G1, RFPO consistently preserves full-step control performance under one-step execution, with one-step returns remaining within 2.4% of their corresponding 64-step values across both zero and random initialization. On Unitree Go2, one-step execution retains 98.5% of the 64-step reward while reducing onboard mean inference latency from 4.39 ms to 0.08 ms, yielding a 54.9x speedup. Real-robot experiments further validate stable one-step locomotion. Code: AIGeeksGroup/RFPO. Website: aigeeksgroup.github.io.
Mentions
🔗 GitHub: AIGeeksGroup/DragMesh 📚 プロジェクトページ: aigeeksgroup.github.io 📚 論文(arXiv): arxiv.org 出典:mp.weixin.qq.com ✍️ by ツキタマ(出典を突き合わせて確認するキュレーター視点)
InteractAgent: Agentic Human Motion Interaction with Memory-Augmented Reflection Long-horizon human-scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors. Text-driven motion models are largely scene-agnostic and lack closed-loop correction, while geometry-conditioned approaches depend on paired scene-motion training or explicit 3D preprocessing that limits generalization to novel layouts and open-vocabulary goals. We present InteractAgent, an agentic multimodal LLM system for long-horizon human-scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor. InteractAgent integrates Online Observation for step-wise recovery decisions (continue/redo/replan) and Memory-Augmented Reflection for multi-round plan refinement with adaptive stopping. On UniHSI, InteractAgent improves task success and reduces penetration compared with training-free baselines. Code: AIGeeksGroup/InteractAgent.
Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation Haoyu Zhang, Siming Li, Zeyu Zhang, Yi Zhang, Tat-Seng Chua, Hao Tang arxiv.org [𝚌𝚜.𝚁𝙾 𝚌𝚜.𝙲𝚅] 💬Code: AIGeeksGroup/Code2Worlds-2, t.co
Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation Recent coding large language models (LLMs) can synthesize dynamic 4D worlds from natural language, but the resulting content remains a visual artifact. It lacks physical semantics such as the collision geometry, mass, friction, inertia, and joint structure, required to enable it to simulate the physical world, and its dynamics are scripted to look plausible rather than designed to execute. What is needed is an executable world: one that a physics engine can load and step, resolving contact physically. We present Code2Worlds-2, an extension of Code2Worlds that turns 4D world generation into an embodied world simulation. We keep the original dual-stream generation pipeline intact and append two modules. First, a SimReady Scene Compiler that turns a generated visual scene into a simulator-compatible representation: it assigns collision meshes via convex decomposition, estimates physical attributes with a vision-language model, infers a lightweight support graph, instantiates the engine's contact, friction, and dynamics constraints, and applies a gravity settling pass, before exporting the scene as MuJoCo MJCF. Second, a Simulator-in-the-Loop 4D Refiner that implements an automated agent harness-a compile, simulate, critique and refine loop-that converts visual dynamics into executable rigid and articulated dynamics and evolves the physical settings rather than only the code, replacing the purely visual motion critic with a simulator critic that inspects penetration depth, collision pairs, stability, contact forces, joint limits, and robot reachability. We further propose SimCode4D, a state-based benchmark with five task templates for evaluating embodied simulation. Experiments on SimCode4D and across natural, indoor, industrial, urban, and service scenes show that Code2Worlds-2 produces physically plausible, simulator-compatible, and robot-interactable scenes. Comments: Code: AIGeeksGroup/Code2Worlds-2, Website: aigeeksgroup.github.io
GENERATE ACCURATE DEPTH MAPS FROM IMAGES! 👁️ Aigeeksgroup AnydepthAnyDepth generates depth maps from single images using the Simple Depth Transformer decoder. AIGeeksGroup/AnyDepth t.co
Code2Games: Enabling Coding Agents for Gaming World Generation Wei Wu, Ziyang Xu, Zeyu Zhang, Yang Zhao, Hao Tang arxiv.org [𝚌𝚜.𝙲𝚅] 💬Code: AIGeeksGroup/Code2Games, t.co
PWM: Personalized World Models with Online Reinforcement Learning Zhexin Lou, Guancheng Lu, Zeyu Zhang, Yi Zhang, Yang Zhao, Hao Tang arxiv.org [𝚌𝚜.𝙲𝚅] 💬Code: AIGeeksGroup/PWM, t.co
RFPO: Rectified Flow Policy Optimization for Embodied Control Flow-based policies provide an expressive framework for continuous robot control, but their iterative ODE integration incurs substantial inference cost. Naively reducing the integration budget can severely degrade control, since policies optimized under full-step execution are not explicitly constrained to remain reliable under coarse numerical integration. We refer to this mismatch as the few-step discretization gap. To address this problem, we introduce RFPO, a flow-policy optimization framework for reliable few-step execution. Reward-aware online Reflow rectifies student-induced transport paths during on-policy learning, making the resulting policy more robust to coarse integration. A frozen Gaussian PPO controller supplies complementary action-space supervision at full and intermediate integration budgets, while the deployed policy remains a single flow student executed with one Euler step. Across Unitree Go2, Boston Dynamics Spot, Unitree H1, and Unitree G1, RFPO consistently preserves full-step control performance under one-step execution, with one-step returns remaining within 2.4% of their corresponding 64-step values across both zero and random initialization. On Unitree Go2, one-step execution retains 98.5% of the 64-step reward while reducing onboard mean inference latency from 4.39 ms to 0.08 ms, yielding a 54.9x speedup. Real-robot experiments further validate stable one-step locomotion. Code: AIGeeksGroup/RFPO. Website: aigeeksgroup.github.io.