Hy-Embodied-VLM-1.0: Efficient Physical-World Agents
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy-Embodied-VLM-1.0, an efficient and powerful embodied foundation model specifically designed for embodied agents operating in the physical world. To cultivate such capabilities from the pre-training stage onward, we define an action-centric capability taxonomy comprising three progressive dimensions: Action-Relevant State Understanding, Action-Transition Reasoning, and Sequential and Adaptive Reasoning. Guided by this taxonomy, we develop a systematic data pipeline and curate data mixtures spanning both pre-training and post-training. To deliver strong physical-world understanding and interaction capabilities while supporting latency-sensitive deployment, we build our model on the Hy3-A3B language backbone and the Hy-ViT2 vision encoder. Its efficient Mixture-of-Experts architecture combines strong model capacity with high inference efficiency. We evaluate Hy-Embodied-VLM-1.0 on a comprehensive suite of 38 benchmarks covering embodied perception, physical-world understanding, and embodied reasoning. The model achieves the best performance among similarly sized models on 19 of the 38 benchmarks and substantially outperforms strong competitors, including Qwen3.6-A3B and Cosmos 3. Compared with the previous-generation Hy-Embodied-0.5 MoT-2B, Hy-Embodied-VLM-1.0 improves average performance by 8.4%. Despite activating only 3B parameters, it achieves performance close to that of the previous-generation model with 32B activated parameters. Beyond static benchmark evaluation, Hy-Embodied-VLM-1.0 also demonstrates strong performance on embodied agentic tasks requiring multi-turn interaction and long-horizon reasoning.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 61 of its 76 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
61 of 76 references.
Closest ancestors · 1
Same direction, newest first
Other references · 60
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Visual Instruction Tuningcited by 7,869
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2024Depth Anything V2cited by 1,422
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
38 more
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
- 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
- 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
- 2017Learning to Detect Human-Object Interactionscited by 240
- 2022Egocentric Video-Language Pretrainingcited by 232
- 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
- 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 175
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 139
- 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 136
- 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 135
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 127
- 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 120
- 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 118
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
- 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 107
- 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 62
- 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 44
- 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
- 2023VL-Grasp: a 6-Dof Interactive Grasp Policy for Language-Oriented Objects in Cluttered Indoor Scenescited by 42
- 2024EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarioscited by 41
- 2025RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correctioncited by 41
- 2022PartAfford: Part-level Affordance Discovery from 3D Objectscited by 36
- 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulationcited by 35
- 2025PointArena: Probing Multimodal Grounding Through Language-Guided Pointingcited by 31
- 2025SITE: towards Spatial Intelligence Thorough Evaluationcited by 29
50 most cited of 60 All 60 (JSON)
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulationcited by 2
- 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordinationcited by 1
- 2026Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligencecited by 0
- 2026N0-Foundation: Towards the Age of Tactile Intelligencecited by 0
- 2026FailBench: How Reliable are VLMs at Judging Robot Task Success?cited by 0
- 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Modelscited by 0
- 2026MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulationcited by 0
- 2026When Left Is Right? Camera-Motion Understanding beyond Screencited by 0
- 2026When Left Is Right? Camera-Motion Understanding beyond Screen12 Oct 2026
- 2026MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation29 Sep 2026
- 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordination21 Sep 2026
- 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models14 Sep 2026
- 2026FailBench: How Reliable are VLMs at Judging Robot Task Success?3 Sep 2026
- 2026N0-Foundation: Towards the Age of Tactile Intelligence30 Aug 2026
- 2026Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence7 Aug 2026
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation26 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026How Foundational Skills Influence VLM-based Embodied Agents:A Native Perspective
- 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
- 2025ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning
- 2025PhysiAgent: An Embodied Agent Framework in Physical World
- 2026Plan-Bench: Can Existing VLMs Go Well Beyond Perception to Reason and Plan in Embodied Worlds?
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
- 2026DM0: An Embodied-Native Vision-Language-Action Model towards Physical AI
- 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence