# VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System

arXiv: https://arxiv.org/abs/2607.27380 · HTML: https://mlsanity.com/paper/2607.27380/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 43 of its 63 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 30 of 43)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation](https://mlsanity.com/paper/2410.05363/) (2024)
- [Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model](https://mlsanity.com/paper/2502.10248/) (2025)
- [VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation](https://mlsanity.com/paper/2503.06800/) (2025)
- [VideoDPO: Omni-Preference Alignment for Video Diffusion Generation](https://mlsanity.com/paper/2412.14167/) (2024)
- [WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generation](https://mlsanity.com/paper/2503.08153/) (2025)
- [Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models](https://mlsanity.com/paper/2501.08453/) (2025)
- [PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Drop](https://mlsanity.com/paper/2503.09595/) (2025)
- [Reconstruction Alignment Improves Unified Multimodal Models](https://mlsanity.com/paper/2509.07295/) (2025)
- [PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation](https://mlsanity.com/paper/2509.20358/) (2025)
- [MIND: Benchmarking Memory Consistency and Action Control in World Models](https://mlsanity.com/paper/2602.08025/) (2026)
- [OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning](https://mlsanity.com/paper/2603.24458/) (2026)
- [PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation](https://mlsanity.com/paper/2512.24551/) (2025)
- [MultiWorld: Scalable Multi-Agent Multi-View Video World Models](https://mlsanity.com/paper/2604.18564/) (2026)
- [PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models](https://mlsanity.com/paper/2601.11087/) (2026)
- [Think Before You Diffuse: Infusing Physical Rules into Video Diffusion](https://mlsanity.com/paper/2505.21653/) (2025)
- [PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection](https://mlsanity.com/paper/2511.03997/) (2025)
- [Proact-VL: A Proactive VideoLLM for Real-Time AI Companions](https://mlsanity.com/paper/2603.03447/) (2026)
- [DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generation](https://mlsanity.com/paper/2512.05112/) (2025)
- [SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models](https://mlsanity.com/paper/2605.23345/) (2026)
- [How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing](https://mlsanity.com/paper/2602.01851/) (2026)

## Built on by (most cited first, 8 of 8)

- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)
- [PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model](https://mlsanity.com/paper/2608.06794/) (2026)
- [Editable Visual Design](https://mlsanity.com/paper/2609.04034/) (2026)
- [SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration](https://mlsanity.com/paper/2609.06651/) (2026)
- [VideoGen-Agent: Reinforcing Video Generation Agents](https://mlsanity.com/paper/2609.24997/) (2026)
- [Learning to Reason with Persistent Object States for Video Instance Segmentation](https://mlsanity.com/paper/2609.35539/) (2026)
- [Physis-Lang: Self-Evolving Language as a Physical Representation for Video World Model](https://mlsanity.com/paper/2609.40358/) (2026)
- [Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness](https://mlsanity.com/paper/2610.08621/) (2026)

## Abstract (arXiv)

Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.