mlsanityData to 14 Oct 2026

Generative AI for Autonomous Driving: Frontiers and Opportunities

arXiv 2505.08854 ↗First posted 13 May 2025cs.CV cs.AI cs.ROcited by 7

On the map: End-to-end autonomous driving → Generative simulation for driving

Abstract (arXiv)

Generative Artificial Intelligence (GenAI) constitutes a transformative technological wave that reconfigures industries through its unparalleled capabilities for content creation, reasoning, planning, and multimodal understanding. This revolutionary force offers the most promising path yet toward solving one of engineering's grandest challenges: achieving reliable, fully autonomous driving, particularly the pursuit of Level 5 autonomy. This survey delivers a comprehensive and critical synthesis of the emerging role of GenAI across the autonomous driving stack. We begin by distilling the principles and trade-offs of modern generative modeling, encompassing VAEs, GANs, Diffusion Models, and Large Language Models (LLMs). We then map their frontier applications in image, LiDAR, trajectory, occupancy, video generation as well as LLM-guided reasoning and decision making. We categorize practical applications, such as synthetic data workflows, end-to-end driving strategies, high-fidelity digital twin systems, smart transportation networks, and cross-domain transfer to embodied AI. We identify key obstacles and possibilities such as comprehensive generalization across rare cases, evaluation and safety checks, budget-limited implementation, regulatory compliance, ethical concerns, and environmental effects, while proposing research plans across theoretical assurances, trust metrics, transport integration, and socio-technical influence. By unifying these threads, the survey provides a forward-looking reference for researchers, engineers, and policymakers navigating the convergence of generative AI and advanced autonomous mobility. An actively maintained repository of cited works is available at https://github.com/taco-group/GenAI4AD.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 235 of its 912 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

235 of 912 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments28 Mar
  2. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving26 Mar
  3. 2025CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Driving9 Mar
  4. 2025A Survey of World Models for Autonomous Driving20 Jan
  5. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT27 Dec
  6. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
  7. 2024Driving View Synthesis on Free-form Trajectories with Generative Prior2 Dec
  8. 2024HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Driving2 Dec
  9. 2024Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey5 Nov
  10. 2024DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generation9 Sep
  11. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Driving1 Aug
  12. 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation3 Jun
5 more
  1. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving30 May
  2. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  3. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  4. 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
  5. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep

Other references · 218

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  11. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
38 more
  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  7. 2024Qwen2 Technical Reportcited by 2,120
  8. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  10. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  11. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  12. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  13. 2018Demystifying MMD GANscited by 1,293
  14. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  15. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  16. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  17. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  18. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  19. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  20. 2017Wasserstein GANcited by 639
  21. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  22. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565
  23. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  24. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  25. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
  26. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  27. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  28. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  29. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  30. 2023Learning Interactive Real-World Simulatorscited by 458
  31. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  32. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  33. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  34. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  35. 2023Llemma: An Open Language Model For Mathematicscited by 388
  36. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  37. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  38. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324

50 most cited of 218 All 218 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 1.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Generative AI for Autonomous Driving: A Review
  2. 2023GAIA-1: A Generative World Model for Autonomous Driving
  3. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
  4. 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges
  5. 2024Exploring Generative AI for Sim2Real in Driving Data Synthesis
  6. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions
  7. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving
  8. 2024GPD-1: Generative Pre-training for Driving