mlsanityData to 7 Oct 2026

GenAD: Generalized Predictive Model for Autonomous Driving

arXiv 2403.09630 ↗First posted 14 Mar 2024cs.CVcited by 150

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

In this paper, we introduce the first large-scale video prediction model in the autonomous driving discipline. To eliminate the restriction of high-cost data collection and empower the generalization ability of our model, we acquire massive data from the web and pair it with diverse and high-quality text descriptions. The resultant dataset accumulates over 2000 hours of driving videos, spanning areas all over the world with diverse weather conditions and traffic scenarios. Inheriting the merits from recent latent diffusion models, our model, dubbed GenAD, handles the challenging dynamics in driving scenes with novel temporal reasoning blocks. We showcase that it can generalize to various unseen driving datasets in a zero-shot manner, surpassing general or driving-specific video prediction counterparts. Furthermore, GenAD can be adapted into an action-conditioned prediction model or a motion planner, holding great potential for real-world driving applications.

Key facts. Cited by 150 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 119 of its 135 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

119 of 135 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Driving29 Dec
  2. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  3. 2023ADriver-I: A General World Model for Autonomous Driving22 Nov
  4. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
  5. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep
  6. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting25 Feb
  7. 2022Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model8 Oct
  8. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun

Other references · 111

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2022Training language models to follow instructions with human feedbackcited by 15,712
  9. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  10. 2020Denoising Diffusion Implicit Modelscited by 9,655
  11. 2023Segment Anythingcited by 8,158
  12. 2023Visual Instruction Tuningcited by 7,792
38 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  13. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  14. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  15. 2022Video Diffusion Modelscited by 2,280
  16. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  17. 2015Variational Inference with Normalizing Flowscited by 2,161
  18. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  19. 2017The Kinetics Human Action Video Datasetcited by 1,894
  20. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  21. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  22. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  23. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  24. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  25. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  26. 2018Self-Attention with Relative Position Representationscited by 1,112
  27. 2022Planning-oriented Autonomous Drivingcited by 1,068
  28. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  29. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  30. 2022On Distillation of Guided Diffusion Modelscited by 721
  31. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  32. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  33. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  34. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  35. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  36. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  37. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  38. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488

50 most cited of 111 All 111 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 30; 2025: 71; 2026: 49.

150 papers.

Sort:

By area:

  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  2. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  3. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  4. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  5. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  6. 2025Pseudo-Simulation for Autonomous Drivingcited by 141
  7. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  8. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  9. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  10. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  11. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  12. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
38 more
  1. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  2. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 95
  3. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
  4. 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Drivingcited by 84
  5. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  6. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 74
  7. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 67
  8. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  9. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
  10. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  11. 20253D and 4D World Modeling: A Surveycited by 64
  12. 2025A Survey of World Models for Autonomous Drivingcited by 60
  13. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
  14. 2024FreeVS: Generative View Synthesis on Free Driving Trajectorycited by 52
  15. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  16. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving Worldcited by 43
  17. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 42
  18. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 41
  19. 2024EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generationcited by 38
  20. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  21. 2024DreamDrive: Generative 4D Scene Modeling from Street View Imagescited by 35
  22. 2024SimGen: Simulator-conditioned Driving Scene Generationcited by 34
  23. 2024DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenescited by 34
  24. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modelingcited by 34
  25. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  26. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 30
  27. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29
  28. 2025NuPlanQA: A Large-Scale Dataset and Benchmark for Multi-View Driving Scene Understanding in Multi-Modal Large Language Modelscited by 29
  29. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 29
  30. 2024SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Trafficcited by 27
  31. 2024UniMLVG: Unified Framework for Multi-view Long Video Generation with Comprehensive Control Capabilities for Autonomous Drivingcited by 27
  32. 2025MaskGWM: A Generalizable Driving World Model with Video Mask Reconstructioncited by 27
  33. 2025BEVDriver: Leveraging BEV Maps in LLMs for Robust Closed-Loop Drivingcited by 26
  34. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  35. 2024DrivingDojo Dataset: Advancing Interactive and Knowledge-Enriched Driving World Modelcited by 25
  36. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 24
  37. 2025A Survey of Interactive Generative Videocited by 23
  38. 2025PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generationcited by 22

50 most cited of 150 All 150 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024GenDDS: Generating Diverse Driving Video Scenarios with Prompt-to-Video Generative Model
  2. 2024GenAD: Generative End-to-End Autonomous Driving
  3. 2026DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Driving
  4. 2024GPD-1: Generative Pre-training for Driving
  5. 2025World Model-Based End-to-End Scene Generation for Accident Anticipation in Autonomous Driving
  6. 2026Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
  7. 2025SynAD: Enhancing Real-World End-to-End Autonomous Driving Models through Synthetic Data Integration
  8. 2024DualAD: Disentangling the Dynamic and Static World for End-to-End Driving