Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling
On the map: World models → Video-generation world models and simulators
also in 2 more
Abstract (arXiv)
Recent visual generation models have made major progress in photorealism, typography, instruction following, and interactive editing, yet they still struggle with spatial reasoning, persistent state, long-horizon consistency, and causal understanding. We argue that the field should move beyond appearance synthesis toward intelligent visual generation: plausible visuals grounded in structure, dynamics, domain knowledge, and causal relations. To frame this shift, we introduce a five-level taxonomy: Atomic Generation, Conditional Generation, In-Context Generation, Agentic Generation, and World-Modeling Generation, progressing from passive renderers to interactive, agentic, world-aware generators. We analyze key technical drivers, including flow matching, unified understanding-and-generation models, improved visual representations, post-training, reward modeling, data curation, synthetic data distillation, and sampling acceleration. We further show that current evaluations often overestimate progress by emphasizing perceptual quality while missing structural, temporal, and causal failures. By combining benchmark review, in-the-wild stress tests, and expert-constrained case studies, this roadmap offers a capability-centered lens for understanding, evaluating, and advancing the next generation of intelligent visual generation systems.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 202 of its 371 references that are in the map. First posted on arXiv 30 Apr 2026.
Builds on
202 of 371 references.
Closest ancestors · 5
Same direction, newest first
- 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2022Scalable Diffusion Models with Transformers19 Dec
- 2022Flow Matching for Generative Modeling6 Oct
- 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
Other references · 197
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2017Improved Training of Wasserstein GANscited by 2,223
38 more
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
- 2016Least Squares Generative Adversarial Networkscited by 1,000
- 2025Qwen-Image Technical Reportcited by 973
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2023Training Diffusion Models with Reinforcement Learningcited by 939
- 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2018Deep Retinex Decomposition for Low-Light Enhancementcited by 720
- 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
- 2017Wasserstein GANcited by 635
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2025Mean Flows for One-step Generative Modelingcited by 524
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
- 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
- 2024One Step Diffusion via Shortcut Modelscited by 373
- 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
- 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
- 2025Diffusion Transformers with Representation Autoencoderscited by 275
50 most cited of 197 All 197 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictorscited by 6
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shapingcited by 3
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editingcited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing4 Sep 2026
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
- 2026Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shaping11 May 2026
- 2026WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors11 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Simulating the Visual World with Artificial Intelligence: A Roadmap
- 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency
- 2026Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Models
- 2025Dreamland: Controllable World Creation with Simulator and Generative Models
- 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Models
- 2026GenClaw: Code-Driven Agentic Image Generation
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?