LLM-MLFFN: Multi-Level Autonomous Driving Behavior Feature Fusion via Large Language Model
On the map: End-to-end autonomous driving → Large language models for driving
Abstract (arXiv)
Accurate classification of autonomous vehicle (AV) driving behaviors is critical for safety validation, performance diagnosis, and traffic integration analysis. However, existing approaches primarily rely on numerical time-series modeling and often lack semantic abstraction, limiting interpretability and robustness in complex traffic environments. This paper presents LLM-MLFFN, a novel large language model (LLM)-enhanced multi-level feature fusion network designed to address the complexities of multi-dimensional driving data. The proposed LLM-MLFFN framework integrates priors from largescale pre-trained models and employs a multi-level approach to enhance classification accuracy. LLM-MLFFN comprises three core components: (1) a multi-level feature extraction module that extracts statistical, behavioral, and dynamic features to capture the quantitative aspects of driving behaviors; (2) a semantic description module that leverages LLMs to transform raw data into high-level semantic features; and (3) a dual-channel multi-level feature fusion network that combines numerical and semantic features using weighted attention mechanisms to improve robustness and prediction accuracy. Evaluation on the Waymo open trajectory dataset demonstrates the superior performance of the proposed LLM-MLFFN, achieving a classification accuracy of over 94%, surpassing existing machine learning models. Ablation studies further validate the critical contributions of multi-level fusion, feature extraction strategies, and LLM-derived semantic reasoning. These results suggest that integrating structured feature modeling with language-driven semantic abstraction provides a principled and interpretable pathway for robust autonomous driving behavior classification.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 34 of its 48 references that are in the map. First posted on arXiv 3 Mar 2026.
Builds on
34 of 48 references.
Closest ancestors · 3
Same direction, newest first
- 2025SVBRD-LLM: Self-Verifying Behavioral Rule Discovery for Autonomous Vehicle Identification18 Nov
- 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Models12 Dec
- 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models28 Sep
Other references · 31
- 2017Attention Is All You Needcited by 59,932
- 2023GPT-4 Technical Reportcited by 18,132
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
- 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2017End-to-end Driving via Conditional Imitation Learningcited by 479
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
- 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
19 more
- 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
- 2016Time Series Classification from Scratch with Deep Neural Networks: A Strong Baselinecited by 358
- 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
- 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
- 2017LSTM Fully Convolutional Networks for Time Series Classificationcited by 135
- 2018Multivariate LSTM-FCNs for Time Series Classificationcited by 120
- 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
- 2023V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecastingcited by 114
- 2023Hidden Biases of End-to-End Driving Modelscited by 95
- 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Drivingcited by 88
- 2018Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysiscited by 61
- 2018Deep Gated Recurrent and Convolutional Network Hybrid Model for Univariate Time Series Classificationcited by 26
- 2025RAD: Retrieval-Augmented Decision-Making of Meta-Actions with Vision-Language Models in Autonomous Drivingcited by 18
- 2025Characteristics Analysis of Autonomous Vehicle Pre-crash Scenarioscited by 8
- 2024RAC3: Retrieval-Augmented Corner Case Comprehension for Autonomous Driving with Vision-Language Modelscited by 7
- 2025Trust-MARL: Trust-Based Multi-Agent Reinforcement Learning Framework for Cooperative On-Ramp Merging Control in Heterogeneous Traffic Flowcited by 3
- 2025BIDA: A Bi-level Interaction Decision-making Algorithm for Autonomous Vehicles in Dynamic Traffic Scenarioscited by 2
- 2023Exploring Driving Behavior for Autonomous Vehicles Based on Gramian Angular Field Vision Transformercited by 1
- 2025TGLD: A Trust-Aware Game-Theoretic Lane-Changing Decision Framework for Automated Vehicles in Heterogeneous Trafficcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Large Foundation Models for Trajectory Prediction in Autonomous Driving: A Comprehensive Survey
- 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving
- 2023A Survey on Multimodal Large Language Models for Autonomous Driving
- 2023Empowering Autonomous Driving with Large Language Models: A Safety Perspective
- 2023LLM4Drive: A Survey of Large Language Models for Autonomous Driving
- 2024Leveraging Large Language Models for Enhancing Autonomous Vehicle Perception
- 2024Multimodal LLM for Intelligent Transportation Systems
- 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving