FLARE: Learning Future-Aware Latent Representations from Vision-Language Models for Autonomous Driving
On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection
also in 1 more
Abstract (arXiv)
While Vision-Language Models (VLMs) offer rich world knowledge for end-to-end autonomous driving, current approaches heavily rely on labor-intensive language annotations (e.g., VQA) to bridge perception and control. This paradigm suffers from a fundamental mismatch between discrete linguistic tokens and continuous driving trajectories, often leading to suboptimal control policies and inefficient utilization of pre-trained knowledge. To address these challenges, we propose FLARE (Future-aware LAtent REpresentation), a novel framework that activates the visual-semantic capabilities of pre-trained VLMs without requiring language supervision. Instead of aligning with text, we introduce a self-supervised future feature prediction objective. This mechanism compels the model to anticipate scene dynamics and ego-motion directly in the latent space, enabling the learning of robust driving representations from large-scale unlabeled trajectory data. Furthermore, we integrate Group Relative Policy Optimization (GRPO) into the planning process to refine decision-making quality. Extensive experiments on the NAVSIM benchmark demonstrate that FLARE achieves state-of-the-art performance, validating the effectiveness of leveraging VLM knowledge via predictive self-supervision rather than explicit language generation.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 41 references that are in the map. First posted on arXiv 9 Jan 2026.
Builds on
35 of 41 references.
Closest ancestors · 4
Same direction, newest first
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving9 Jun
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models19 Feb
- 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
- 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct
Other references · 31
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
- 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
- 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
19 more
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
- 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
- 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
- 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
- 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
- 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 181
- 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 151
- 2025Pseudo-Simulation for Autonomous Drivingcited by 150
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 134
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 117
- 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
- 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
- 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 96
- 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Drivingcited by 86
- 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 67
- 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
- 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 36
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
- 2026LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformerscited by 2
- 2026LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matchingcited by 2
- 2026Post-Training in End-to-End Autonomous Drivingcited by 2
- 2026UniTeD: Unified Temporal Diffusion for Joint Perception and Planning in Autonomous Drivingcited by 1
- 2026DH-VLM: Dual-Horizon Cooperative Latent Reasoning for Autonomous Drivingcited by 1
- 2026VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Drivingcited by 0
- 2026Evaluation Is All You Need for Multi-Modal Autonomous Drivingcited by 0
- 2026ReWAM: Reciprocal World Action Models for Interactive Autonomous Drivingcited by 0
- 2026Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?cited by 0
- 2026Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?7 Oct 2026
- 2026ReWAM: Reciprocal World Action Models for Interactive Autonomous Driving30 Sep 2026
- 2026Evaluation Is All You Need for Multi-Modal Autonomous Driving25 Sep 2026
- 2026DH-VLM: Dual-Horizon Cooperative Latent Reasoning for Autonomous Driving10 Aug 2026
- 2026Post-Training in End-to-End Autonomous Driving9 Jul 2026
- 2026UniTeD: Unified Temporal Diffusion for Joint Perception and Planning in Autonomous Driving24 Jun 2026
- 2026LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching9 Jun 2026
- 2026VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving9 May 2026
- 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook2 Apr 2026
- 2026LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformers2 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model
- 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges
- 2026PRISM: Privileged Probabilistic Latent Supervision for End-to-End Autonomous Driving Motion Planning
- 2026RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving
- 2026LWDrive: Layer-Wise World-Model-Guided Vision-Language Model Planning for Autonomous Driving
- 2026DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving
- 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving