# Perturbation-Based Epistemic Uncertainty for Failure Detection in Vision-Language-Action Models

arXiv: https://arxiv.org/abs/2606.20754 · HTML: https://mlsanity.com/paper/2606.20754/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models.
- Builds on 24 of its 28 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Placed in

- Vision-language-action models / [Failure recovery and robustness](https://mlsanity.com/vla/failure-recovery-and-robustness/)

## Builds on (most cited first, 24 of 24)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization](https://mlsanity.com/paper/2510.03827/) (2025)
- [Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors](https://mlsanity.com/paper/2005.07186/) (2020)
- [SAFE: Multitask Failure Detection for Vision-Language-Action Models](https://mlsanity.com/paper/2506.09937/) (2025)
- [Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress](https://mlsanity.com/paper/2410.04640/) (2024)
- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models](https://mlsanity.com/paper/2406.11675/) (2024)
- [Failure Prediction at Runtime for Generative Robot Policies](https://mlsanity.com/paper/2510.09459/) (2025)
- [Multi-Task Interactive Robot Fleet Learning with Visual World Models](https://mlsanity.com/paper/2410.22689/) (2024)
- [Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation](https://mlsanity.com/paper/2508.06426/) (2025)
- [Enhancing Hallucination Detection through Noise Injection](https://mlsanity.com/paper/2502.03799/) (2025)
- [Training-Free Bayesianization for Low-Rank Adapters of Large Language Models](https://mlsanity.com/paper/2412.05723/) (2024)
- [Epistemic Uncertainty Quantification For Pre-trained Neural Network](https://mlsanity.com/paper/2404.10124/) (2024)
- [Singular Bayesian Neural Networks](https://mlsanity.com/paper/2602.00387/) (2026)
- [Uncertainty Quantification for Flow-Based Vision-Language-Action Models](https://mlsanity.com/paper/2606.18043/) (2026)
- [Epistemic Uncertainty for Generated Image Detection](https://mlsanity.com/paper/2412.05897/) (2024)

## Abstract (arXiv)

Vision-Language-Action (VLA) models have shown strong performance in robotic manipulation, but reliable uncertainty quantification remains challenging, particularly under distribution shift. Unlike autoregressive policies, many modern VLA models generate continuous actions through regression or flow-based generation, where explicit predictive probabilities are unavailable. Moreover, stochastic action sampling primarily captures action-generation variability under a fixed model, while failure detection under distribution shift can benefit from capturing uncertainty in the model itself. Motivated by Bayesian perspectives on local model variations, we propose perturbation-based failure detection (PFD), a training-free framework for estimating epistemic uncertainty in VLA models through low-rank weight perturbations. Specifically, we inject random low-rank perturbations into selected transformer weight matrices and estimate epistemic uncertainty from disagreement across perturbed action predictions. Experiments on LIBERO-PRO show that PFD achieves the highest average AUROC and balanced accuracy among the evaluated methods while consistently outperforming stochastic action sampling across distribution shifts. Real-world robot experiments further demonstrate that PFD provides a competitive failure-detection signal under an unseen object shift.