mlsanityData to 14 Oct 2026

VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System

arXiv 2607.27380 ↗First posted 29 Jul 2026cs.CV cs.AIcited by 8

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 63 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

43 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026MetaPoint: Unlocking Precise Spatial Control in Agentic Visual Generation3 Jun

Other references · 42

  1. 2024GPT-4o System Cardcited by 4,588
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  7. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  8. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
  9. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  10. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  11. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  12. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 168
30 more
  1. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  2. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 108
  3. 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 90
  4. 2025Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Modelscited by 60
  5. 2025PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Dropcited by 50
  6. 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 49
  7. 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generationcited by 41
  8. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 35
  9. 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoningcited by 27
  10. 2025PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generationcited by 23
  11. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 23
  12. 2026PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Modelscited by 17
  13. 2025Think Before You Diffuse: Infusing Physical Rules into Video Diffusioncited by 16
  14. 2025PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selectioncited by 15
  15. 2026Proact-VL: A Proactive VideoLLM for Real-Time AI Companionscited by 15
  16. 2025DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generationcited by 13
  17. 2026SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Modelscited by 13
  18. 2026How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editingcited by 12
  19. 2026GENIUS: Generative Fluid Intelligence Evaluation Suitecited by 12
  20. 2025PhysChoreo: Physics-Controllable Video Generation with Part-Aware Semantic Groundingcited by 9
  21. 2026PEARL: Personalized Streaming Video Understanding Modelcited by 9
  22. 2026Video Models Reason Early: Exploiting Plan Commitment for Maze Solvingcited by 9
  23. 2026Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteriacited by 9
  24. 2026Video Models Can Reason with Verifiable Rewardscited by 9
  25. 2026WebVR: Benchmarking Multimodal LLMs for WebPage Recreation from Videos via Human-Aligned Visual Rubricscited by 8
  26. 2026CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generationcited by 6
  27. 2026GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environmentscited by 5
  28. 2026Diffusion Model as a Generalist Segmentation Learnercited by 4
  29. 2026Temporal Backtracking Search for Test-time Generative Video Reasoningcited by 4
  30. 2026AD-MIR: Bridging the Gap from Perception to Persuasion in Advertising Video Understanding via Structured Reasoningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Simulating the Visual World with Artificial Intelligence: A Roadmap
  2. 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
  3. 2026NEWTON: Agentic Planning for Physically Grounded Video Generation
  4. 2026A Mechanistic View on Video Generation as World Models: State and Dynamics
  5. 2024Kubrick: Multimodal Agent Collaborations for Synthetic Video Generation
  6. 2025Plan-X: Instruct Video Generation via Semantic Planning
  7. 2026LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows
  8. 2025MoReGen: Multi-Agent Motion-Reasoning Engine for Code-based Text-to-Video Synthesis