mlsanityData to 14 Oct 2026

Early Warning Signals for OpenVLA Failure under Visual Distribution Shift

arXiv 2606.29699 ↗First posted 29 Jun 2026cs.CV cs.AI cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

Visual shifts can cause a vision-language-action policy to fail after initially plausible behavior. We ask whether OpenVLA's internal activations contain signals associated with the steps before failure. We freeze the policy, record one MLP activation per LIBERO-10 step, and fit two linear monitors. Occlusion reduces task success from 57% to 17%. Within failed matched-reset trajectories, a layer-16 logistic probe attains AUROC 0.972 and AUPRC 0.352, whereas action disagreement attains AUROC 0.496. Without refitting, the occlusion-trained probe reaches AUROC 0.689 on failed camera-jitter episodes. In a calibration check, however, the same layer-16 monitor averages 3.32 warning onsets per clean episode. This contrast shows that strong retrospective discrimination does not imply operationally quiet warning behavior. Because fitting and evaluation share tasks, resets, and seed, these results establish retrospective separability rather than prediction on independent episodes.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 42 of its 48 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

42 of 48 references.

References · 42

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  12. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
30 more
  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  2. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  4. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  5. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  7. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  10. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  11. 2022A Generalist Agentcited by 808
  12. 2019Designing and Interpreting Probes with Control Taskscited by 640
  13. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  14. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  15. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  16. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
  17. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  18. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  19. 2020Information-Theoretic Probing for Linguistic Structurecited by 207
  20. 2023Vision-Language Models as Success Detectorscited by 142
  21. 2024AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulationcited by 125
  22. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 111
  23. 2025SAFE: Multitask Failure Detection for Vision-Language-Action Modelscited by 80
  24. 2025Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policiescited by 52
  25. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 38
  26. 2022A System-Level View on Out-of-Distribution Data in Roboticscited by 33
  27. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 33
  28. 2025Mechanistic interpretability for steering vision-language-action modelscited by 31
  29. 2023Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptationcited by 16
  30. 2022Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehiclescited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026The Gate, Not the Cache: Gate Provenance Bounds the Closed-Loop Reliability of Training-Free VLA Token Skippingcited by 2
  2. 2026VLA-Scope: Shift-Aware Failure Prediction for Vision-Language-Action Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models
  2. 2026ConflictVLA-Bench: Benchmarking Behavioral Responses of Vision-Language-Action Models to Premise Conflicts
  3. 2026How VLAs Fail Differently: Black-Box Action Monitoring Reveals Architecture-Specific Failure Signatures
  4. 2026Natural State-Prediction Accuracy can Hide Weak Controlled Responsiveness in VLA Readouts
  5. 2026Are Vision-Language-Action Models Robust to One-Step Observation Perturbations?
  6. 2026Does a Learned Corrector Beat a Simple Retreat? Evidence from a Frozen VLA
  7. 2026CARE: Certifying Acceleration for Vision-Language-Action Inference
  8. 2026Blackout vs. Freeze: Analyzing Physical Failure Modes of VLAs under Camera Faults