mlsanityData to 7 Oct 2026

GAIA-1: A Generative World Model for Autonomous Driving

arXiv 2309.17080 ↗First posted 29 Sep 2023cs.CV cs.AI cs.ROcited by 618

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

Autonomous driving promises transformative improvements to transportation, but building systems capable of safely navigating the unstructured complexity of real-world scenarios remains challenging. A critical problem lies in effectively predicting the various potential outcomes that may emerge in response to the vehicle's actions as the world evolves. To address this challenge, we introduce GAIA-1 ('Generative AI for Autonomy'), a generative world model that leverages video, text, and action inputs to generate realistic driving scenarios while offering fine-grained control over ego-vehicle behavior and scene features. Our approach casts world modeling as an unsupervised sequence modeling problem by mapping the inputs to discrete tokens, and predicting the next token in the sequence. Emerging properties from our model include learning high-level structures and scene dynamics, contextual awareness, generalization, and understanding of geometry. The power of GAIA-1's learned representation that captures expectations of future events, combined with its ability to generate realistic samples, provides new possibilities for innovation in the field of autonomy, enabling enhanced and accelerated training of autonomous driving technology.

Key facts. Cited by 618 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 84 of its 94 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

84 of 94 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep

Other references · 83

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  10. 2020Denoising Diffusion Implicit Modelscited by 9,655
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
38 more
  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  3. 2022Scalable Diffusion Models with Transformerscited by 5,931
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  12. 2022Video Diffusion Modelscited by 2,280
  13. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  14. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  15. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  16. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  17. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  18. 2023Mastering Diverse Domains through World Modelscited by 1,193
  19. 2022Planning-oriented Autonomous Drivingcited by 1,068
  20. 2020Mastering Atari with Discrete World Modelscited by 990
  21. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  22. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  23. 2022A Generalist Agentcited by 802
  24. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  25. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  26. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  27. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  28. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  29. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  30. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  31. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  32. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  33. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  34. 2019Model-Based Reinforcement Learning for Ataricited by 495
  35. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  36. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  37. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  38. 2016Generating Videos with Scene Dynamicscited by 376

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 131; 2025: 207; 2026: 264.

618 papers.

Sort:

By area:

  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  3. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  4. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  5. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  6. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  7. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
  8. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  9. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  10. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  11. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  12. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
38 more
  1. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  2. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  3. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  4. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
  5. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  6. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  7. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  8. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  9. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
  10. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  11. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 149
  12. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 145
  13. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  14. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  15. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  16. 2025Unified Vision-Language-Action Modelcited by 128
  17. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
  18. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  19. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
  20. 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 124
  21. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  22. 2025Training Agents Inside of Scalable World Modelscited by 122
  23. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  24. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  25. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  26. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  27. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
  28. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 102
  29. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  30. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  31. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  32. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  33. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
  34. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  35. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  36. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 91
  37. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  38. 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 90

50 most cited of 618 All 618 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Generative AI for Autonomous Driving: Frontiers and Opportunities
  2. 2024GPD-1: Generative Pre-training for Driving
  3. 2024GenAD: Generative End-to-End Autonomous Driving
  4. 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges
  5. 2026UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learning
  6. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving
  7. 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
  8. 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving