mlsanityData to 14 Oct 2026

Sensing Which Modality Matters: Evidence-Gated Regularization for Robust VLA Policies

arXiv 2609.03142 ↗First posted 2 Sep 2026cs.RO cs.CV cs.LGcited by 0

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

Vision-Language-Action (VLA) policies fuse multimodal sensory inputs, but training on limited and homogeneous robot demonstrations encourages spurious inter-sensor correlations rather than task-relevant signal, a failure we term modality entanglement. Under real-world occlusions and distractors, this manifests as nuisance sensitivity to corruption of uninformative sensors and single-modality insufficiency when only one informative sensor remains intact. We propose Evidence-Gated Regularization (EGR), a modality-agnostic training objective that introduces zero inference-time overhead. EGR derives a per-frame and per-sensor task-relevance signal to gate two state-conditional consistency objectives: invariance on low-evidence sensors, and single-sensor sufficiency on high-evidence ones. We introduce a benchmark based on BEHAVIOR-1K, comprising a fast inference-only diagnostic suite and 47 rollout-based skills targeting modality entanglement. We validate EGR on this benchmark and on two real-robot setups with fundamentally different embodiments: a bi-manual setup with two Kinova arms and three RGB cameras, and a single-arm MELFA ASSISTA setup combining vision and GelSight tactile sensors. EGR improves simulation success rates (SR) from 12.5% to 16.4% under full modalities (+31%), from 9.4% to 16.5% under uninformative-sensor corruption (+75%), and from 2.8% to 6.1% under single-sensor fallback (+120%). Under physical-object distractors, EGR boosts SR from 30% to 85% on the bi-manual setup (+183%) and from 55% to 70% on the tactile setup (+27%).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 31 of its 37 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

31 of 37 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization12 May
  2. 2025π0.5: a Vision-Language-Action Model with Open-World Generalization22 Apr
  3. 2024Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust2 Oct
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Model13 Jun
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 26

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  5. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  6. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  7. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  8. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 303
  9. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 264
  10. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
  11. 2021Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answeringcited by 71
  12. 2023On Uni-Modal Feature Learning in Supervised Multi-Modal Learningcited by 53
14 more
  1. 2026When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAscited by 30
  2. 2023Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptationcited by 29
  3. 2025RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbationscited by 25
  4. 2024Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulationcited by 20
  5. 2025Diagnosing and Mitigating Modality Interference in Multimodal Large Language Modelscited by 19
  6. 2026TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulationcited by 19
  7. 2017Learning End-to-end Multimodal Sensor Policies for Autonomous Navigationcited by 18
  8. 2026LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policiescited by 14
  9. 2025Openpi Comet: Competition Solution For 2025 BEHAVIOR Challengecited by 9
  10. 2025BOSS: Benchmark for Observation Space Shift in Long-Horizon Taskcited by 7
  11. 2022Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrationscited by 6
  12. 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Modelscited by 6
  13. 2024ARCADE: Scalable Demonstration Collection and Generation via Augmented Reality for Imitation Learningcited by 5
  14. 2025Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Robust Bimanual Vision-Language-Action Models via Embarrassingly Simple Modality Masking
  2. 2026A Causality-aware Infer-diagnose-refine Framework for Test-time Modality Adaptation in VLA Models
  3. 2026Natural State-Prediction Accuracy can Hide Weak Controlled Responsiveness in VLA Readouts
  4. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context
  5. 2026Rethinking Visual-Language-Action Model Scaling: Alignment, Mixture, and Regularization
  6. 2026Decoupling Vision, Language, and Action for Efficient Multi-Task Robot Policies
  7. 2026PerturBot: Breaking Shortcut Priors in Vision-Language-Action Models with Perturbative Training
  8. 2026IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies