mlsanityData to 14 Oct 2026

Robota-Embed: Unified VLA Embeddings for Robot In-Context Learning and Evolution

arXiv 2610.19495 ↗First posted 14 Oct 2026cs.RO cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Retrieving useful context for a robot requires matching its task, execution stage, and the continuation that should follow. We introduce Robota-Embed, to our knowledge the first vision-language-action (VLA) embedding model to jointly encode all three modalities in both current-state queries and recorded future continuations. Predictive retrieval training aligns these endpoints, enabling historical continuations to serve as context at inference time. We construct a multi-source corpus of 1,695,684 cleaned training anchors and introduce Robota-Embed-Bench, with 4,000 queries and a shared 8,000-entry gallery. Robota-Embed achieves 47.33% clip-level and 81.77% episode-level R@1, exceeding action-free Gemini Embedding 2 by 22.09 and 33.47 percentage points. Controlled retrieval studies show that removing native action degrades retrieval more than removing vision. We study two complementary applications. Embed2ICL uses retrieved experience for training-free and training-based policy conditioning, increasing success from 47.0% to 56.3% in DROID-Sim and from 86.4% to 92.8% on the LIBERO difficult subset, while improving task progress on real robots. Embed2Evolve uses execution feedback to grow task-local memory and refine context use around a frozen GPT-6 Astra policy. Across five ManiSkill tasks, it achieves 68% success, compared with 24% for GPT-6 Astra Direct and 33% for a harness independently evolved with Omni-Embed-Nemotron-3B under the same protocol. Relative to the latter, it uses approximately 79.5% fewer policy tokens per test rollout, counting both successes and failures. These results show that a unified, action-aware retrieval space can support robot in-context learning and task-local context evolution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 39 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

35 of 39 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  5. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  6. 2025SAM 3: Segment Anything with Conceptscited by 906
  7. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  8. 2026World Action Models are Zero-shot Policiescited by 465
  9. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 349
  10. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  11. 2026Causal World Modeling for Robot Controlcited by 321
  12. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
23 more
  1. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
  2. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  3. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
  4. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
  5. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  6. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 116
  7. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 38
  8. 2024Retrieval-Augmented Embodied Agentscited by 31
  9. 2025RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Modelscited by 22
  10. 2023Contrastive Language, Action, and State Pre-training for Robot Learningcited by 16
  11. 2024REGENT: A Retrieval-Augmented Generalist Agent That Can Act In-Context in New Environmentscited by 16
  12. 2025Omni-Embed-Nemotron: A Unified Multimodal Retrieval Model for Text, Image, Audio, and Videocited by 16
  13. 2026T-Rex: Tactile-Reactive Dexterous Manipulationcited by 13
  14. 2026Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalizationcited by 13
  15. 2026Gemini Embedding 2: A Native Multimodal Embedding Model from Geminicited by 10
  16. 2026WeMM-Embedding: WeChat Multi-Modal Embedding Technical Reportcited by 8
  17. 2026HOST:Robots Acquire Manipulation Skills in Seconds from a Single Human Videocited by 7
  18. 2025VLA-R: Vision-Language Action Retrieval toward Open-World End-to-End Autonomous Drivingcited by 5
  19. 2015Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language and Trajectoriescited by 4
  20. 2026Zeva: In-Context Causal Learning for Generalizable Embodied Manipulationcited by 3
  21. 2026Retrieval-Augmented Robots via Retrieve-Reason-Actcited by 2
  22. 2026Grounding Robot Generalization in Training Data via Retrieval-Augmented VLMscited by 2
  23. 2026ROSER: Few-Shot Robotic Sequence Retrieval for Scalable Robot Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Does Retrieval Help? A Study of In-Context Adaptation in Vision-Language-Action Models
  2. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context
  3. 2026JEPA-VLA: Video Predictive Embedding is Needed for VLA Models
  4. 2026Semantic Anchoring for Robotic Action Representations
  5. 2025ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and Retrieval
  6. 2026Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
  7. 2025Token Expand-Merge: Training-Free Token Compression for Vision-Language-Action Models
  8. 2026G0.5: One Autoregressive Stream for Robot Reasoning and Action