Shifting Uncertainty to Critical Moments: Towards Reliable Uncertainty Quantification for VLA Model
On the map: Vision-language-action models → Failure recovery and robustness
Abstract (arXiv)
Vision-Language-Action (VLA) models enable general-purpose robotic policies by mapping visual observations and language instructions to low-level actions, but they often lack reliable introspection. A common practice is to compute a token-level uncertainty signal and take its mean over a rollout. However, mean aggregation can dilute short-lived but safety-critical uncertainty spikes in continuous control. In particular, successful rollouts may contain localized high-entropy segments due to benign noise or non-critical micro-adjustments, while failure rollouts can appear low-entropy for most timesteps and only exhibit brief spikes near the onset of failure. We propose a unified uncertainty quantification approach for predicting rollout success versus failure that (1) uses max-based sliding window pooling to preserve transient risk signals, (2) applies motion-aware stability weighting to emphasize high-frequency action oscillations associated with unstable behaviors, and (3) performs DoF-adaptive calibration via Bayesian Optimization to prioritize kinematically critical axes. Experiments on the LIBERO benchmark show that our method substantially improves failure prediction accuracy and yields more reliable signals for failure detection, which can support downstream human-in-the-loop interventions.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models. Builds on 18 of its 23 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
18 of 23 references.
References · 18
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
- 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
6 more
- 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
- 2025Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecastingcited by 53
- 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
- 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
- 2025Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Basecited by 6
- 2025Multi-Cache Enhanced Prototype Learning for Test-Time Generalization of Vision-Language Modelscited by 6
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agentscited by 12
- 2026Where Fake Citations Are Made: Tracing Field-Level Hallucination to Specific Neurons in LLMscited by 0
- 2026Making two action heads agree: coordination mechanisms and a runtime collapse certificate for flow-matching policiescited by 0
- 2026LM-X: Explainable Vision--Language--Action Modeling via Progress, Event, and Uncertainty Predictioncited by 0
- 2026Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Modelscited by 0
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directionscited by 0
- 2026Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Models3 Sep 2026
- 2026LM-X: Explainable Vision--Language--Action Modeling via Progress, Event, and Uncertainty Prediction26 Aug 2026
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions19 Aug 2026
- 2026Making two action heads agree: coordination mechanisms and a runtime collapse certificate for flow-matching policies16 Aug 2026
- 2026Where Fake Citations Are Made: Tracing Field-Level Hallucination to Specific Neurons in LLMs20 Apr 2026
- 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents12 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Perturbation-Based Epistemic Uncertainty for Failure Detection in Vision-Language-Action Models
- 2026ReconVLA: An Uncertainty-Guided and Failure-Aware Vision-Language-Action Framework for Robotic Control
- 2026Uncertainty Quantification for Flow-Based Vision-Language-Action Models
- 2025Confidence Calibration in Vision-Language-Action Models
- 2026Reshaping Action Error Distributions for Reliable Vision-Language-Action Models
- 2026Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Models
- 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models
- 2026Taming VLAs under Robot Execution Errors: Self-Compensation and Stress Testing