mlsanityData to 7 Oct 2026

OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature Extraction

arXiv 2503.03734 ↗First posted 5 Mar 2025cs.RO cs.CVcited by 54

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

Vision-Language-Action (VLA) models aim to predict robotic actions based on visual observations and language instructions. Existing approaches require fine-tuning pre-trained visionlanguage models (VLMs) as visual and language features are independently fed into downstream policies, degrading the pre-trained semantic alignments. We propose OTTER, a novel VLA architecture that leverages these existing alignments through explicit, text-aware visual feature extraction. Instead of processing all visual features, OTTER selectively extracts and passes only task-relevant visual features that are semantically aligned with the language instruction to the policy transformer. This allows OTTER to keep the pre-trained vision-language encoders frozen. Thereby, OTTER preserves and utilizes the rich semantic understanding learned from large-scale pre-training, enabling strong zero-shot generalization capabilities. In simulation and real-world experiments, OTTER significantly outperforms existing VLA models, demonstrating strong zeroshot generalization to novel objects and environments. Video, code, checkpoints, and dataset: https://ottervla.github.io/.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 37 of its 48 references that are in the map. First posted on arXiv 5 Mar 2025.

Builds on

37 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2024The Llama 3 Herd of Modelscited by 14,192
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2016Layer Normalizationcited by 5,579
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
24 more
  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  6. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  7. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  8. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  9. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  10. 2022A Generalist Agentcited by 802
  11. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  12. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  13. 2023LERF: Language Embedded Radiance Fieldscited by 538
  14. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  15. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  16. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  17. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  18. 2024What matters when building vision-language models?cited by 321
  19. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  20. 2020Self-Supervised MultiModal Versatile Networkscited by 276
  21. 2023ViNT: A Foundation Model for Visual Navigationcited by 210
  22. 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
  23. 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 112
  24. 2024In-Context Imitation Learning via Next-Token Predictioncited by 65

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 30; 2026: 24.

54 papers.

Sort:

By area:

  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  2. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  3. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
  4. 2025Training Strategies for Efficient Embodied Reasoningcited by 53
  5. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
  6. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  7. 2025HAMLET: Switch your Vision-Language-Action Model into a History-Aware Policycited by 42
  8. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Contextcited by 34
  9. 2025Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentationcited by 31
  10. 20253D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Taskscited by 30
  11. 2025PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policiescited by 29
  12. 2025Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Surveycited by 25
38 more
  1. 2026RLDX-1 Technical Reportcited by 25
  2. 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
  3. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  4. 2025VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Votingcited by 22
  5. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  6. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policycited by 17
  7. 2026Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignmentcited by 15
  8. 2026VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agentscited by 9
  9. 2025VLAs are Confined yet Capable of Generalizing to Novel Instructionscited by 8
  10. 2025Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learningcited by 8
  11. 2025Compressor-VLA: Instruction-Guided Visual Token Compression for Efficient Robotic Manipulationcited by 7
  12. 2025Transport Discrepancy as a Reliability Signal for Vision-Language-Action Modelscited by 6
  13. 2026PEAfowl: Perception-Enhanced Multi-View Vision-Language-Action for Bimanual Manipulationcited by 6
  14. 2026VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memorycited by 6
  15. 2025MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalizationcited by 5
  16. 2026DTP: A Simple yet Effective Distracting Token Pruning Framework for Vision-Language Action Modelscited by 5
  17. 2026APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policiescited by 5
  18. 2025IA-VLA: Input Augmentation for Vision-Language-Action models in settings with semantically complex taskscited by 4
  19. 2025MetaVLA: Unified Meta Co-training For Efficient Embodied Adaptioncited by 4
  20. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  21. 2025CIVIL: Causal and Intuitive Visual Imitation Learningcited by 3
  22. 2026ST-VLA: Enabling 4D-Aware Spatiotemporal Understanding for General Robot Manipulationcited by 3
  23. 2026MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalizationcited by 3
  24. 2025Generalization Capability for Imitation Learningcited by 2
  25. 2026Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Modelscited by 2
  26. 2025AURA: Autonomous Upskilling with Retrieval-Augmented Agentscited by 1
  27. 2025IGen: Scalable Data Generation for Robot Learning from Open-World Imagescited by 1
  28. 2026Vision-Language Models Unlock Task-Centric Latent Actionscited by 1
  29. 2026HMVLA: Hyperbolic Multimodal Fusion for Vision-Language-Action Modelscited by 1
  30. 2026Robust Bimanual Vision-Language-Action Models via Embarrassingly Simple Modality Maskingcited by 1
  31. 2026LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policiescited by 1
  32. 2025Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensemblescited by 0
  33. 2025Learning to Grasp Anything by Playing with Random Toyscited by 0
  34. 2026Think Proprioceptively: State-Grounded Visual Token Selection for VLA Policiescited by 0
  35. 2026Differentiate-and-Inject: Enhancing VLAs via Functional Differentiation Induced by In-Parameter Structural Reasoningcited by 0
  36. 2026MotionBits: Video Segmentation through Motion-Level Analysis of Rigid Bodiescited by 0
  37. 2026Spatially Conditioned Diffusion Policy: Learning Precise and Robust Manipulation with a Single RGB Cameracited by 0
  38. 2026LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Actioncited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
  2. 2025NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks
  3. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model
  4. 2025Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations
  5. 2025From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models
  6. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
  7. 2025XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations
  8. 2025VLA-0: Building State-of-the-Art VLAs with Zero Modification