mlsanityData to 14 Oct 2026

LIBERO-VIFO: Benchmarking the Capability and Safety of Visual Cue Following in Vision-Language-Action Models

arXiv 2608.17600 ↗First posted 18 Aug 2026cs.ROcited by 0

On the map: Vision-language-action models → Failure recovery and robustness

Abstract (arXiv)

Visual cues are increasingly adopted to guide robot learning, but whether Vision-Language-Action (VLA) models can reliably follow authorized cues while disregarding unauthorized ones remains unclear. Existing work covers only a narrow range of cue forms and focuses on final task success, providing only a coarse assessment of cue-following capability. Treating all visual cues as authorized also leaves safety risks of unauthorized following unexplored. To address these gaps, we introduce LIBERO-VIFO, a benchmark to evaluate both the capability and safety of visual cue following in VLA models. LIBERO-VIFO defines eight visual cue families spanning diverse forms. A total of four protocols in two parts are defined: Part I tests cue understanding and authorized following, while Part II evaluates unauthorized visual cue following under language-cue conflict and empty language conditions. Evaluating seven VLA models reveals that although visual cue understanding does not reliably translate into execution, current VLAs are able to execute cue-indicated tasks without language instruction, exposing an emerging risk of unauthorized visual cue following. Extended experiments on scene-instantiated cues, safety-critical settings, and real-robot deployment corroborate these findings. LIBERO-VIFO brings both the capability and safety of visual cue following into systematic evaluation, establishing visual-centric safety as a new perspective for the VLA community.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models. Builds on 19 of its 23 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

19 of 23 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models15 Oct
  2. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization4 Oct

Other references · 17

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  5. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
  6. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  7. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  8. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  9. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 304
  10. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 133
  11. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 129
  12. 2025InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulationcited by 72
5 more
  1. 2026MolmoAct2: Action Reasoning Models for Real-world Deploymentcited by 50
  2. 2024RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketchescited by 41
  3. 2026InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalizationcited by 20
  4. 2024VIP: Vision Instructed Pre-training for Robotic Manipulationcited by 11
  5. 2026Action-Sketcher: From Reasoning to Action via Visual Sketches for Long-Horizon Robotic Manipulationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
  2. 2026When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs
  3. 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
  4. 2026Encoded but Not in Control: Revealing the Grounding Gap in Vision-Language Robot Policies
  5. 2026When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models
  6. 2025VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models
  7. 2026LIBERO-X: Robustness Litmus for Vision-Language-Action Models
  8. 2026MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation