mlsanityData to 14 Oct 2026

SEAL: Vision-Language Model-Based Safe End-to-End Cooperative Autonomous Driving with Adaptive Long-Tail Modeling

arXiv 2506.21041 ↗First posted 26 Jun 2025cs.RO cs.AI cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 1 more
Abstract (arXiv)

Autonomous driving technologies face significant safety challenges while operating under rare, diverse, and visually degraded weather scenarios. These challenges become more critical in cooperative settings, where vehicles and infrastructure jointly perceive and reason across complex environments. To address these issues, we propose SEAL, a vision-language model-based framework with adaptive multimodal learning for robust cooperative autonomous driving under long-tail scenarios. SEAL introduces three core innovations: (i) a prompt-driven long-tail scenario generation and evaluation pipeline that leverages foundation models to synthesize realistic long-tail conditions such as snow and fog across vehicle- and infrastructure-side views, enriching training diversity efficiently; (ii) a gated multi-scenario adaptive attention module that modulates the visual stream using scenario priors to recalibrate ambiguous or corrupted features; and (iii) a multi-task scenario-aware contrastive learning objective that improves multimodal alignment and promotes cross-scenario feature separability. Extensive experiments demonstrate that SEAL significantly outperforms existing baselines in reasoning, safety, and planning accuracy under complex, challenging driving conditions, advancing the safety, robustness, and scalability of autonomous driving.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 45 references that are in the map. First posted on arXiv 26 Jun 2025.

Builds on

24 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Driving20 Dec
  2. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving19 Dec

Other references · 22

  1. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  2. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  3. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  4. 2020V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predictioncited by 292
  5. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
  6. 2019Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Cloudscited by 165
  7. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  8. 2022COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehiclescited by 95
  9. 2024Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Drivingcited by 56
  10. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  11. 2015Learning to Generate Images with Perceptual Similarity Metricscited by 39
  12. 2024CMP: Cooperative Motion Prediction with Multi-Agent Communicationcited by 25
10 more
  1. 2025VERDI: VLM-Embedded Reasoning for Autonomous Drivingcited by 16
  2. 2023MACP: Efficient Model Adaptation for Cooperative Perceptioncited by 14
  3. 2025Co-MTP: A Cooperative Trajectory Prediction Framework with Multi-Temporal Fusion for Autonomous Drivingcited by 14
  4. 2023Dynamically Conservative Self-Driving Planner for Long-Tail Casescited by 13
  5. 2023Interactive Joint Planning for Autonomous Vehiclescited by 13
  6. 2024CooPre: Cooperative Pretraining for V2X Cooperative Perceptioncited by 11
  7. 2025LiloDriver: A Lifelong Learning Framework for Closed-loop Motion Planning in Long-tail Autonomous Driving Scenarioscited by 7
  8. 2025RiskNet: Interaction-Aware Risk Forecasting for Autonomous Driving in Long-Tail Scenarioscited by 4
  9. 2025LTDA-Drive: LLMs-guided Generative Models based Long-tail Data Augmentation for Autonomous Drivingcited by 4
  10. 2023Long-Tail Learning with Rebalanced Contrastive Losscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generation
  2. 2026VLM-CASE: Vision-Language Model Enabled Context-Adaptive Safety Envelopes for Anticipatory Safe Autonomous Driving
  3. 2025V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models and Graph-of-Thoughts
  4. 2025V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models
  5. 2026HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Driving
  6. 2025LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving
  7. 2026UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving
  8. 2025CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models