Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Survey
On the map: End-to-end autonomous driving → Reasoning VLA and VLM driving policies
Abstract (arXiv)
This comprehensive survey examines the integration of knowledge-based approaches in autonomous driving systems, specifically focusing on trajectory prediction and planning. We extensively analyze various methodologies for incorporating domain knowledge, traffic rules, and commonsense reasoning into autonomous driving systems. The survey categorizes and analyzes approaches based on their knowledge representation and integration methods, ranging from purely symbolic to hybrid neuro-symbolic architectures. We examine recent developments in logic programming, foundation models for knowledge representation, reinforcement learning frameworks, and other emerging technologies incorporating domain knowledge. This work systematically reviews recent approaches, identifying key challenges, opportunities, and future research directions in knowledge-enhanced autonomous driving systems. Our analysis reveals emerging trends in the field, including the increasing importance of interpretable AI, the role of formal verification in safety-critical systems, and the potential of hybrid approaches that combine traditional knowledge representation with modern machine learning techniques.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning VLA and VLM driving policies, one of 9 directions in End-to-end autonomous driving. Builds on 35 of its 71 references that are in the map. First posted on arXiv 13 Feb 2025.
Builds on
35 of 71 references.
Closest ancestors · 2
Same direction, newest first
- 2024EMMA: End-to-End Multimodal Model for Autonomous Driving30 Oct
- 2023Towards Knowledge-driven Autonomous Driving7 Dec
Other references · 33
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
- 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
- 2019Neural Probabilistic Logic Programming in DeepProbLogcited by 389
- 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 204
- 2020Logic Tensor Networkscited by 197
- 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
- 2023A Language Agent for Autonomous Drivingcited by 161
- 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Modelscited by 158
- 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 91
- 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 79
21 more
- 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
- 2025Robust Autonomy Emerges from Self-Playcited by 64
- 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoningcited by 59
- 2024Driving Everywhere with Large Language Model Policy Adaptationcited by 53
- 2024CoBL-Diffusion: Diffusion-Based Conditional Robot Planning in Dynamic Environments Using Control Barrier and Lyapunov Functionscited by 30
- 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
- 2021Injecting Knowledge in Data-driven Vehicle Trajectory Predictorscited by 18
- 2022Driving in Real Life with Inverse Reinforcement Learningcited by 18
- 2021AUTO-DISCERN: Autonomous Driving Using Common Sense Reasoningcited by 14
- 2020Formalizing Traffic Rules for Machine Interpretabilitycited by 12
- 2022Using Ontologies for the Formalization and Recognition of Criticality for Automated Drivingcited by 12
- 2024SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphscited by 10
- 2024TR2MTL: LLM based framework for Metric Temporal Logic Formalization of Traffic Rulescited by 9
- 2022Cautious Planning with Incremental Symbolic Perception: Designing Verified Reactive Driving Maneuverscited by 5
- 2022Road Traffic Law Adaptive Decision-making for Self-Driving Vehiclescited by 3
- 2022Model Predictive Robustness of Signal Temporal Logic Predicatescited by 3
- 2022Utilizing Background Knowledge for Robust Reasoning over Traffic Situationscited by 2
- 2014Traffic Network Control from Temporal Logic Specificationscited by 1
- 2023Assessing Drivers' Situation Awareness in Semi-Autonomous Vehicles: ASP based Characterisations of Driving Dynamics for Modelling Scene Interpretation and Projectioncited by 1
- 2024A Survey on Hybrid Motion Planning Methods for Automated Driving Systemscited by 1
- 2024Let Hybrid A* Path Planner Obey Traffic Rules: A Deep Reinforcement Learning-Based Planning Frameworkcited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.
- 2025Trajectory Prediction Meets Large Language Models: A Surveycited by 13
Similar, not cited
Close in wording, no citation either way
- 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions
- 2024WiseAD: Knowledge Augmented End-to-End Autonomous Driving with Vision-Language Model
- 2026Learning-Based Behavior Planning for Automated Driving: Real-World Integration and Deployment
- 2025A Framework for Learning Scoring Rules in Autonomous Driving Planning Systems
- 2025Human-like Semantic Navigation for Autonomous Driving using Knowledge Representation and Large Language Models
- 2026A Neuro-Symbolic Framework Combining Inductive and Deductive Reasoning for Autonomous Driving Planning
- 2025Chain-of-Thought for Autonomous Driving: A Comprehensive Survey and Future Prospects
- 2025ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving