mlsanityData to 14 Oct 2026

Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

arXiv 2607.12894 ↗First posted 14 Jul 2026cs.CVcited by 8

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy-Embodied-VLM-1.0, an efficient and powerful embodied foundation model specifically designed for embodied agents operating in the physical world. To cultivate such capabilities from the pre-training stage onward, we define an action-centric capability taxonomy comprising three progressive dimensions: Action-Relevant State Understanding, Action-Transition Reasoning, and Sequential and Adaptive Reasoning. Guided by this taxonomy, we develop a systematic data pipeline and curate data mixtures spanning both pre-training and post-training. To deliver strong physical-world understanding and interaction capabilities while supporting latency-sensitive deployment, we build our model on the Hy3-A3B language backbone and the Hy-ViT2 vision encoder. Its efficient Mixture-of-Experts architecture combines strong model capacity with high inference efficiency. We evaluate Hy-Embodied-VLM-1.0 on a comprehensive suite of 38 benchmarks covering embodied perception, physical-world understanding, and embodied reasoning. The model achieves the best performance among similarly sized models on 19 of the 38 benchmarks and substantially outperforms strong competitors, including Qwen3.6-A3B and Cosmos 3. Compared with the previous-generation Hy-Embodied-0.5 MoT-2B, Hy-Embodied-VLM-1.0 improves average performance by 8.4%. Despite activating only 3B parameters, it achieves performance close to that of the previous-generation model with 32B activated parameters. Beyond static benchmark evaluation, Hy-Embodied-VLM-1.0 also demonstrates strong performance on embodied agentic tasks requiring multi-turn interaction and long-horizon reasoning.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 61 of its 76 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

61 of 76 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents8 Apr

Other references · 60

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  10. 2024Depth Anything V2cited by 1,422
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  12. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
38 more
  1. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  2. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  3. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  4. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  5. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  6. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  7. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  8. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  9. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  10. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  11. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  12. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
  13. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  14. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
  15. 2017Learning to Detect Human-Object Interactionscited by 240
  16. 2022Egocentric Video-Language Pretrainingcited by 232
  17. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
  18. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
  19. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 175
  20. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
  21. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 139
  22. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 136
  23. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 135
  24. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 127
  25. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 120
  26. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 118
  27. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
  28. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 107
  29. 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 62
  30. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 44
  31. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  32. 2023VL-Grasp: a 6-Dof Interactive Grasp Policy for Language-Oriented Objects in Cluttered Indoor Scenescited by 42
  33. 2024EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarioscited by 41
  34. 2025RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correctioncited by 41
  35. 2022PartAfford: Part-level Affordance Discovery from 3D Objectscited by 36
  36. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulationcited by 35
  37. 2025PointArena: Probing Multimodal Grounding Through Language-Guided Pointingcited by 31
  38. 2025SITE: towards Spatial Intelligence Thorough Evaluationcited by 29

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026How Foundational Skills Influence VLM-based Embodied Agents:A Native Perspective
  2. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
  3. 2025ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning
  4. 2025PhysiAgent: An Embodied Agent Framework in Physical World
  5. 2026Plan-Bench: Can Existing VLMs Go Well Beyond Perception to Reason and Plan in Embodied Worlds?
  6. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
  7. 2026DM0: An Embodied-Native Vision-Language-Action Model towards Physical AI
  8. 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence