# VideoAgent: Self-Improving Video Generation

arXiv: https://arxiv.org/abs/2410.10076 · HTML: https://mlsanity.com/paper/2410.10076/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 70 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Builds on (most cited first, 30 of 60)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [Self-Rewarding Language Models](https://mlsanity.com/paper/2401.10020/) (2024)
- [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [Learning Universal Policies via Text-Guided Video Generation](https://mlsanity.com/paper/2302.00111/) (2023)
- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023)

## Built on by (most cited first, 25 of 25)

- [Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation](https://mlsanity.com/paper/2412.15109/) (2024)
- [WoW: Towards a World omniscient World model Through Embodied Interaction](https://mlsanity.com/paper/2509.22642/) (2025)
- [Exploring the Evolution of Physics Cognition in Video Generation: A Survey](https://mlsanity.com/paper/2503.21765/) (2025)
- [VISTA: A Test-Time Self-Improving Video Generation Agent](https://mlsanity.com/paper/2510.15831/) (2025)
- [Generative Physical AI in Vision: A Survey](https://mlsanity.com/paper/2501.10928/) (2025)
- [T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generation](https://mlsanity.com/paper/2507.18107/) (2025)
- [UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalist](https://mlsanity.com/paper/2511.08521/) (2025)
- [Survey of Video Diffusion Models: Foundations, Implementations, and Applications](https://mlsanity.com/paper/2504.16081/) (2025)
- [Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances](https://mlsanity.com/paper/2508.10316/) (2025)
- [VQQA: An Agentic Approach for Video Evaluation and Quality Improvement](https://mlsanity.com/paper/2603.12310/) (2026)
- [MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agent](https://mlsanity.com/paper/2502.03207/) (2025)
- [Self-Improving Loops for Visual Robotic Planning](https://mlsanity.com/paper/2506.06658/) (2025)
- [HANDI: Hand-Centric Text-and-Image Conditioned Video Generation](https://mlsanity.com/paper/2412.04189/) (2024)
- [A Challenge to Build Neuro-Symbolic Video Agents](https://mlsanity.com/paper/2505.13851/) (2025)
- [Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms](https://mlsanity.com/paper/2603.28489/) (2026)
- [Artificial Intelligence for Biomedical Video Generation](https://mlsanity.com/paper/2411.07619/) (2024)
- [Seeing Space and Motion: Enhancing Latent Actions with Geometric and Dynamic Awareness for Vision-Language-Action Models](https://mlsanity.com/paper/2509.26251/) (2025)
- [FlowLoss: Dynamic Flow-Conditioned Loss Strategy for Video Diffusion Models](https://mlsanity.com/paper/2504.14535/) (2025)
- [Stealing Creator's Workflow: A Creator-Inspired Agentic Framework with Iterative Feedback Loop for Improved Scientific Short-form Generation](https://mlsanity.com/paper/2504.18805/) (2025)
- [Paper2Web: Let's Make Your Paper Alive!](https://mlsanity.com/paper/2510.15842/) (2025)
- [MotiMotion: Motion-Controlled Video Generation with Visual Reasoning](https://mlsanity.com/paper/2605.22818/) (2026)
- [CineForge: Self-Improving Agents for Long-Horizon Video Generation](https://mlsanity.com/paper/2608.29621/) (2026)
- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)
- [Draft-and-Target Sampling for Video Generation Policy](https://mlsanity.com/paper/2603.13438/) (2026)
- [Video Generation Models: A Survey of Post-Training and Alignment](https://mlsanity.com/paper/2610.00812/) (2026)

## Abstract (arXiv)

Video generation has been used to generate visual plans for controlling robotic systems. Given an image observation and a language instruction, previous work has generated video plans which are then converted to robot controls to be executed. However, a major bottleneck in leveraging video generation for control lies in the quality of the generated videos, which often suffer from hallucinatory content and unrealistic physics, resulting in low task success when control actions are extracted from the generated videos. While scaling up dataset and model size provides a partial solution, integrating external feedback is both natural and essential for grounding video generation in the real world. With this observation, we propose VideoAgent for self-improving generated video plans based on external feedback. Instead of directly executing the generated video plan, VideoAgent first refines the generated video plans using a novel procedure which we call self-conditioning consistency, allowing inference-time compute to be turned into better generated video plans. As the refined video plan is being executed, VideoAgent can collect additional data from the environment to further improve video plan generation. Experiments in simulated robotic manipulation from MetaWorld and iTHOR show that VideoAgent drastically reduces hallucination, thereby boosting success rate of downstream manipulation tasks. We further illustrate that VideoAgent can effectively refine real-robot videos, providing an early indicator that robots can be an effective tool in grounding video generation in the physical world. Video demos and code can be found at https://video-as-agent.github.io.