# Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Survey

arXiv: https://arxiv.org/abs/2502.10477 · HTML: https://mlsanity.com/paper/2502.10477/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning VLA and VLM driving policies, one of 9 directions in End-to-end autonomous driving.
- Builds on 35 of its 71 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Placed in

- End-to-end autonomous driving / [Reasoning VLA and VLM driving policies](https://mlsanity.com/driving/reasoning-vla-and-vlm-driving-policies/)

## Builds on (most cited first, 30 of 35)

- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [EMMA: End-to-End Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2410.23262/) (2024)
- [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023)
- [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020)
- [LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving](https://mlsanity.com/paper/2310.03026/) (2023)
- [A Language Agent for Autonomous Driving](https://mlsanity.com/paper/2311.10813/) (2023)
- [Drive Like a Human: Rethinking Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2307.07162/) (2023)
- [RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model](https://mlsanity.com/paper/2402.10828/) (2024)
- [Virtual to Real Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1704.03952/) (2017)
- [SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policies](https://mlsanity.com/paper/2109.13602/) (2021)
- [Robust Autonomy Emerges from Self-Play](https://mlsanity.com/paper/2502.03349/) (2025)
- [LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning](https://mlsanity.com/paper/2401.00125/) (2023)
- [Driving Everywhere with Large Language Model Policy Adaptation](https://mlsanity.com/paper/2402.05932/) (2024)
- [CoBL-Diffusion: Diffusion-Based Conditional Robot Planning in Dynamic Environments Using Control Barrier and Lyapunov Functions](https://mlsanity.com/paper/2406.05309/) (2024)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [Towards Knowledge-driven Autonomous Driving](https://mlsanity.com/paper/2312.04316/) (2023)
- [Injecting Knowledge in Data-driven Vehicle Trajectory Predictors](https://mlsanity.com/paper/2103.04854/) (2021)
- [Driving in Real Life with Inverse Reinforcement Learning](https://mlsanity.com/paper/2206.03004/) (2022)
- [AUTO-DISCERN: Autonomous Driving Using Common Sense Reasoning](https://mlsanity.com/paper/2110.13606/) (2021)
- [Formalizing Traffic Rules for Machine Interpretability](https://mlsanity.com/paper/2007.00330/) (2020)
- [Using Ontologies for the Formalization and Recognition of Criticality for Automated Driving](https://mlsanity.com/paper/2205.01532/) (2022)
- [SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphs](https://mlsanity.com/paper/2404.19379/) (2024)
- [TR2MTL: LLM based framework for Metric Temporal Logic Formalization of Traffic Rules](https://mlsanity.com/paper/2406.05709/) (2024)
- [Cautious Planning with Incremental Symbolic Perception: Designing Verified Reactive Driving Maneuvers](https://mlsanity.com/paper/2209.09818/) (2022)
- [Road Traffic Law Adaptive Decision-making for Self-Driving Vehicles](https://mlsanity.com/paper/2204.11411/) (2022)
- [Model Predictive Robustness of Signal Temporal Logic Predicates](https://mlsanity.com/paper/2209.07881/) (2022)

## Built on by (most cited first, 1 of 1)

- [Trajectory Prediction Meets Large Language Models: A Survey](https://mlsanity.com/paper/2506.03408/) (2025)

## Abstract (arXiv)

This comprehensive survey examines the integration of knowledge-based approaches in autonomous driving systems, specifically focusing on trajectory prediction and planning. We extensively analyze various methodologies for incorporating domain knowledge, traffic rules, and commonsense reasoning into autonomous driving systems. The survey categorizes and analyzes approaches based on their knowledge representation and integration methods, ranging from purely symbolic to hybrid neuro-symbolic architectures. We examine recent developments in logic programming, foundation models for knowledge representation, reinforcement learning frameworks, and other emerging technologies incorporating domain knowledge. This work systematically reviews recent approaches, identifying key challenges, opportunities, and future research directions in knowledge-enhanced autonomous driving systems. Our analysis reveals emerging trends in the field, including the increasing importance of interpretable AI, the role of formal verification in safety-critical systems, and the potential of hybrid approaches that combine traditional knowledge representation with modern machine learning techniques.