mlsanityData to 14 Oct 2026

FLARE: Learning Future-Aware Latent Representations from Vision-Language Models for Autonomous Driving

arXiv 2601.05611 ↗First posted 9 Jan 2026cs.CVcited by 10

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 1 more
Abstract (arXiv)

While Vision-Language Models (VLMs) offer rich world knowledge for end-to-end autonomous driving, current approaches heavily rely on labor-intensive language annotations (e.g., VQA) to bridge perception and control. This paradigm suffers from a fundamental mismatch between discrete linguistic tokens and continuous driving trajectories, often leading to suboptimal control policies and inefficient utilization of pre-trained knowledge. To address these challenges, we propose FLARE (Future-aware LAtent REpresentation), a novel framework that activates the visual-semantic capabilities of pre-trained VLMs without requiring language supervision. Instead of aligning with text, we introduce a self-supervised future feature prediction objective. This mechanism compels the model to anticipate scene dynamics and ego-motion directly in the latent space, enabling the learning of robust driving representations from large-scale unlabeled trajectory data. Furthermore, we integrate Group Relative Policy Optimization (GRPO) into the planning process to refine decision-making quality. Extensive experiments on the NAVSIM benchmark demonstrate that FLARE achieves state-of-the-art performance, validating the effectiveness of leveraging VLM knowledge via predictive self-supervision rather than explicit language generation.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 41 references that are in the map. First posted on arXiv 9 Jan 2026.

Builds on

35 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving9 Jun
  2. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models19 Feb
  3. 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
  4. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct

Other references · 31

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2022Planning-oriented Autonomous Drivingcited by 1,087
  7. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  8. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  9. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  10. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  11. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  12. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
19 more
  1. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  2. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  3. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  4. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
  5. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  6. 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
  7. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 181
  8. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
  9. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 151
  10. 2025Pseudo-Simulation for Autonomous Drivingcited by 150
  11. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 134
  12. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 117
  13. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  14. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  15. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 96
  16. 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Drivingcited by 86
  17. 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 67
  18. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  19. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving
  2. 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model
  3. 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges
  4. 2026PRISM: Privileged Probabilistic Latent Supervision for End-to-End Autonomous Driving Motion Planning
  5. 2026RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving
  6. 2026LWDrive: Layer-Wise World-Model-Guided Vision-Language Model Planning for Autonomous Driving
  7. 2026DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving
  8. 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving