mlsanityData to 7 Oct 2026

E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving

arXiv 2512.04733 ↗First posted 4 Dec 2025cs.CV cs.AIcited by 2

On the map: End-to-end autonomous driving → Reasoning VLA and VLM driving policies

Abstract (arXiv)

End-to-end autonomous driving (AD) systems increasingly adopt vision-language-action (VLA) models, yet they typically ignore the passenger's emotional state, which is central to comfort and AD acceptance. We introduce Open-Domain End-to-End (OD-E2E) autonomous driving, where an autonomous vehicle (AV) must interpret free-form natural-language commands, infer the emotion, and plan a physically feasible trajectory. We propose E3AD, an emotion-aware VLA framework that augments semantic understanding with two cognitively inspired components: a continuous Valenc-Arousal-Dominance (VAD) emotion model that captures tone and urgency from language, and a dual-pathway spatial reasoning module that fuses egocentric and allocentric views for human-like spatial cognition. A consistency-oriented training scheme, combining modality pretraining with preference-based alignment, further enforces coherence between emotional intent and driving actions. Across real-world datasets, E3AD improves visual grounding and waypoint planning and achieves state-of-the-art (SOTA) VAD correlation for emotion estimation. These evaluation results show that injecting emotion into VLA-style driving yields more human-aligned grounding, planning, and feedback.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning VLA and VLM driving policies, one of 9 directions in End-to-end autonomous driving. Builds on 59 of its 87 references that are in the map. First posted on arXiv 4 Dec 2025.

Builds on

59 of 87 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving31 Aug
  2. 2025A Survey on Vision-Language-Action Models for Autonomous Driving30 Jun
  3. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning16 Jun
  4. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving9 Jun
  5. 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving22 May
  6. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model30 Mar
  7. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment12 Mar
  8. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving29 Oct
  9. 2024VLP: Vision Language Planning for Autonomous Driving10 Jan
  10. 2023GPT-4 Enhanced Multimodal Grounding for Autonomous Driving: Leveraging Cross-Modal Attention with Large Language Models6 Dec

Other references · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  11. 2022Planning-oriented Autonomous Drivingcited by 1,068
  12. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
37 more
  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  3. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  4. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  5. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  6. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  7. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  8. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  9. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  10. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  11. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  12. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  13. 2020From Goals, Waypoints & Paths To Long Term Human Trajectory Forecastingcited by 166
  14. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  15. 2019Talk2Car: Taking Control of Your Self-Driving Carcited by 141
  16. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
  17. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  18. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  19. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 72
  20. 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
  21. 2020Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimationcited by 57
  22. 2024ITINERA: Integrating Spatial Optimization with Large Language Models for Open-domain Urban Itinerary Planningcited by 33
  23. 2024Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoningcited by 33
  24. 2025CoT-Drive: Efficient Motion Forecasting for Autonomous Driving with LLMs and Chain-of-Thought Promptingcited by 26
  25. 2023Human-Centric Autonomous Systems With LLMs for User Command Reasoningcited by 22
  26. 2025Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Drivingcited by 19
  27. 2020AttnGrounder: Talking to Cars with Attentioncited by 10
  28. 2020Cosine meets Softmax: A tough-to-beat baseline for visual groundingcited by 10
  29. 2024A Cognitive-Driven Trajectory Prediction Model for Autonomous Driving in Mixed Autonomy Environmentcited by 10
  30. 2025Minds on the Move: Decoding Trajectory Prediction in Autonomous Driving with Cognitive Insightscited by 10
  31. 2025Steerable Adversarial Scenario Generation through Test-Time Preference Alignmentcited by 8
  32. 2025Mamba-VA: A Mamba-based Approach for Continuous Emotion Recognition in Valence-Arousal Spacecited by 6
  33. 2022Grounding Commands for Autonomous Vehicles via Layer Fusion with Region-specific Dynamic Layer Attentioncited by 4
  34. 2021Predicting Physical World Destinations for Commands Given to Self-Driving Carscited by 3
  35. 2023RouteKG: A knowledge graph-based framework for route prediction on road networkscited by 2
  36. 2025INTENT: Trajectory Prediction Framework with Intention-Guided Contrastive Clusteringcited by 2
  37. 2025Think Before You Drive: World Model-Inspired Multimodal Grounding for Autonomous Vehiclescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Listen, Look, Drive: Coupling Audio Instructions for User-aware VLA-based Autonomous Driving
  2. 2026Less Language, More Latents: Annotation-Efficient VLAs for Driving
  3. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
  4. 2026Unifying Language-Action Understanding and Generation for Autonomous Driving
  5. 2025DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Models
  6. 2026Vega: Learning to Drive with Natural Language Instructions
  7. 2025DriveCritic: Towards Context-Aware, Human-Aligned Evaluation for Autonomous Driving with Vision-Language Models
  8. 2026A Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Driving