mlsanityData to 7 Oct 2026

Your Vision-Language-Action Model Already Has Attention Heads For Path Deviation Detection

arXiv 2603.13782 ↗First posted 14 Mar 2026cs.RO cs.CVcited by 3

On the map: Vision-language-action models → Failure recovery and robustness

Abstract (arXiv)

Vision-Language-Action (VLA) models have demonstrated strong potential for predicting semantic actions in navigation tasks, demonstrating the ability to reason over complex linguistic instructions and visual contexts. However, they are fundamentally hindered by visual-reasoning hallucinations that lead to trajectory deviations. Addressing this issue has conventionally required training external critic modules or relying on complex uncertainty heuristics. In this work, we discover that monitoring a few attention heads within a frozen VLA model can accurately detect path deviations without incurring additional computational overhead. We refer to these heads, which inherently capture the spatiotemporal causality between historical visual sequences and linguistic instructions, as Navigation Heads. Using these heads, we propose an intuitive, training-free anomaly-detection framework that monitors their signals to detect hallucinations in real time. Surprisingly, among over a thousand attention heads, a combination of just three is sufficient to achieve a 44.6 % deviation detection rate with a low false-positive rate of 11.7 %. Furthermore, upon detecting a deviation, we bypass the heavy VLA model and trigger a lightweight Reinforcement Learning (RL) policy to safely execute a shortest-path rollback. By integrating this entire detection-to-recovery pipeline onto a physical robot, we demonstrate its practical robustness. All source code will be publicly available.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models. Builds on 48 of its 59 references that are in the map. First posted on arXiv 14 Mar 2026.

Builds on

48 of 59 references.

References · 48

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  5. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  6. 2019Are Sixteen Heads Really Better than One?cited by 803
  7. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  8. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 629
  9. 2018On Evaluation of Embodied Navigation Agentscited by 526
  10. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  11. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  12. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
36 more
  1. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  2. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  3. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  4. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  5. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
  6. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  7. 2016LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detectioncited by 213
  8. 2017A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencodercited by 209
  9. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  10. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  11. 2022Robot Operating System 2: Design, Architecture, and Uses In The Wildcited by 190
  12. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  13. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  14. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 171
  15. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  16. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  17. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 115
  18. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
  19. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 104
  20. 2020BADGR: An Autonomous Self-Supervised Learning-Based Navigation Systemcited by 97
  21. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  22. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  23. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Groundingcited by 78
  24. 2024Attention Heads of Large Language Models: A Surveycited by 65
  25. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 63
  26. 2023SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environmentscited by 53
  27. 2025Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decodingcited by 42
  28. 2024FAST-LIVO2: Fast, Direct LiDAR-Inertial-Visual Odometrycited by 40
  29. 2022skrl: Modular and Flexible Library for Reinforcement Learningcited by 35
  30. 2020Safe Robot Navigation via Multi-Modal Anomaly Detectioncited by 33
  31. 2020LaND: Learning to Navigate from Disengagementscited by 24
  32. 2022Proactive Anomaly Detection for Robot Navigation with Multi-Sensor Fusioncited by 15
  33. 2024Mind the Error! Detection and Localization of Instruction Errors in Vision-and-Language Navigationcited by 11
  34. 2024Robot Navigation Using Physically Grounded Vision-Language Models in Outdoor Environmentscited by 9
  35. 2025AgriChrono: A Multi-modal Dataset Capturing Crop Growth and Lighting Variability with a Field Robotcited by 1
  36. 2025Vision-Guided Targeted Grasping and Vibration for Robotic Pollination in Controlled Environmentscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior
  2. 2026Act, Think or Abstain: Complexity-Aware Adaptive Inference for Vision-Language-Action Models
  3. 2026VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing
  4. 2026From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation
  5. 2026Mitigating Hallucinations in Large Vision-Language Models via Causal Route Gating
  6. 2026Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMs
  7. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
  8. 2026When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models