mlsanityData to 14 Oct 2026

Exploring the Potential of World Models for Anomaly Detection in Autonomous Driving

arXiv 2308.05701 ↗First posted 10 Aug 2023cs.AI cs.ROcited by 7

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

In recent years there have been remarkable advancements in autonomous driving. While autonomous vehicles demonstrate high performance in closed-set conditions, they encounter difficulties when confronted with unexpected situations. At the same time, world models emerged in the field of model-based reinforcement learning as a way to enable agents to predict the future depending on potential actions. This led to outstanding results in sparse reward and complex control tasks. This work provides an overview of how world models can be leveraged to perform anomaly detection in the domain of autonomous driving. We provide a characterization of world models and relate individual components to previous works in anomaly detection to facilitate further research in the field.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 40 of its 51 references that are in the map. First posted on arXiv 10 Aug 2023.

Builds on

40 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Neural World Models for Computer Vision15 Jun
  2. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 38

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2018World Modelscited by 1,513
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  6. 2023Mastering Diverse Domains through World Modelscited by 1,253
  7. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  8. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  9. 2020Mastering Atari with Discrete World Modelscited by 1,007
  10. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  11. 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
  12. 2020Learning Memory-guided Normality for Anomaly Detectioncited by 278
26 more
  1. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  2. 2016Learning Temporal Regularity in Video Sequencescited by 254
  3. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  4. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  5. 2018Latent Space Autoregression for Novelty Detectioncited by 149
  6. 2021Pixel-wise Anomaly Detection in Complex Driving Scenescited by 109
  7. 2019Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videoscited by 104
  8. 2022Anomaly Detection in Autonomous Driving: A Surveycited by 87
  9. 2022Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wildcited by 70
  10. 2023On Advantages of Mask-level Recognition for Outlier-aware Segmentationcited by 42
  11. 2019Towards Corner Case Detection for Autonomous Drivingcited by 39
  12. 2021Description of Corner Cases in Automated Driving: Goals and Challengescited by 35
  13. 2021An Application-Driven Conceptualization of Corner Cases for Perception in Highly Automated Drivingcited by 33
  14. 2021Corner Cases for Visual Perception in Automated Driving: Some Guidance on Detection Approachescited by 27
  15. 20206-Layer Model for a Structured Description and Categorization of Urban Traffic and Environmentcited by 26
  16. 2019Anomaly Detection with Adversarial Dual Autoencoderscited by 16
  17. 2022One Ontology to Rule Them All: Corner Case Scenarios for Autonomous Drivingcited by 16
  18. 2023Perception Datasets for Anomaly Detection in Autonomous Driving: A Surveycited by 16
  19. 2022On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learningcited by 13
  20. 2022Ad-datasets: a meta-collection of data sets for autonomous drivingcited by 11
  21. 2019Latent space conditioning for improved classification and anomaly detectioncited by 8
  22. 2023Structural Attention-Based Recurrent Variational Autoencoder for Highway Vehicle Anomaly Detectioncited by 6
  23. 2023From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Drivingcited by 6
  24. 2019Implicit Discriminator in Variational Autoencodercited by 5
  25. 2021Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Modelscited by 5
  26. 2023Impact, Attention, Influence: Early Assessment of Autonomous Driving Datasetscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3; 2025: 2.

7 papers.

  1. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Drivingcited by 93
  2. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  3. 2024Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectivescited by 23
  4. 2023Applications of Large Scale Foundation Models for Autonomous Drivingcited by 14
  5. 2025Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Promptingcited by 8
  6. 2024UMAD: Unsupervised Mask-Level Anomaly Detection for Autonomous Drivingcited by 3
  7. 2025World Models for Anomaly Detection during Model-Based Reinforcement Learning Inferencecited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Anomalous Decision Discovery using Inverse Reinforcement Learning
  2. 2024World Models for Autonomous Driving: An Initial Survey
  3. 2026World Engine: Towards the Era of Post-Training for Autonomous Driving
  4. 2025Beyond Simulation: Benchmarking World Models for Planning and Causality in Autonomous Driving
  5. 2025A Survey of World Models for Autonomous Driving
  6. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey
  7. 2023ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events
  8. 2026What Probing Reveals about Autonomous Driving: Linking Internal Prediction Errors to Ego Planning