HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Vision-Language Models (VLMs) and the demands of embodied agents, our models are developed to enhance the core capabilities required by embodied intelligence: spatial and temporal visual perception, alongside advanced embodied reasoning for prediction, interaction, and planning. The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning. To support the fine-grained visual perception essential for embodied tasks, we adopt a Mixture-of-Transformers (MoT) architecture to enable modality-specific computing. By incorporating latent tokens, this design effectively enhances the perceptual representation of the models. To improve reasoning capabilities, we introduce an iterative, self-evolving post-training paradigm. Furthermore, we employ on-policy distillation to transfer the advanced capabilities of the large model to the smaller variant, thereby maximizing the performance potential of the compact model. Extensive evaluations across 22 benchmarks, spanning visual perception, spatial reasoning, and embodied understanding, demonstrate the effectiveness of our approach. Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro. In downstream robot control experiments, we leverage our robust VLM foundation to train an effective Vision-Language-Action (VLA) model, achieving compelling results in real-world physical evaluations. Code and models are open-sourced at https://github.com/Tencent-Hunyuan/HY-Embodied.
Key facts. Cited by 18 papers in the map (data through 13 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 61 of its 80 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
61 of 80 references.
References · 61
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
- 2017Proximal Policy Optimization Algorithmscited by 14,592
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
- 2023Segment Anythingcited by 8,195
- 2023Visual Instruction Tuningcited by 7,827
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,541
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,463
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,096
- 2025Qwen3-VL Technical Reportcited by 2,663
- 2019Towards VQA Models That Can Readcited by 1,705
38 more
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,676
- 2024Depth Anything V2cited by 1,399
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,295
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,154
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,154
- 2016Modeling Context in Referring Expressionscited by 1,057
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,041
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 873
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 857
- 2023Vision Transformers Need Registerscited by 850
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 827
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 818
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 752
- 2024PaliGemma: A versatile 3B VLM for transfercited by 686
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 588
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 577
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 534
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 470
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 445
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 397
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 323
- 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 295
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 243
- 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 227
- 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
- 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 193
- 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 188
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 172
- 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 168
- 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 156
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 135
- 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 135
- 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 133
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
- 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 117
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 111
- 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 111
- 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 101
50 most cited of 61 All 61 (JSON)
Built on by
Citing papers per year: 2026: 18.
18 papers.
Sort:
By area:
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 153
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?cited by 38
- 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Modelscited by 16
- 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulationcited by 10
- 2026PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformationcited by 7
- 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agentscited by 7
- 2026Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Modelscited by 5
- 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AIcited by 5
- 2026Data Pyramid for Embodied Manipulation: A Surveycited by 5
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulationcited by 2
- 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imaginationcited by 1
- 2026SpatialCLI: Learning to Reason With Spatial Tools, Then Without Themcited by 1
6 more
- 2026PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloudcited by 1
- 2026CometVLA: Co-Training on an Embodied Data Pyramid towards Physical Understandingcited by 1
- 2026When Video Misreads: Closed-Loop Distillation of Reading Heuristics for Exploratory Manipulation Trace QAcited by 0
- 2026N0-Foundation: Towards the Age of Tactile Intelligencecited by 0
- 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Modelscited by 0
- 2026MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulationcited by 0
- 2026MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation29 Sep 2026
- 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models14 Sep 2026
- 2026CometVLA: Co-Training on an Embodied Data Pyramid towards Physical Understanding31 Aug 2026
- 2026N0-Foundation: Towards the Age of Tactile Intelligence30 Aug 2026
- 2026PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud4 Aug 2026
- 2026SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them30 Jul 2026
- 2026Data Pyramid for Embodied Manipulation: A Survey27 Jul 2026
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation26 Jul 2026
- 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination15 Jul 2026
- 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agents14 Jul 2026
- 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI5 Jul 2026
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?17 Jun 2026
6 more
- 2026PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation15 Jun 2026
- 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models11 Jun 2026
- 2026Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models9 Jun 2026
- 2026When Video Misreads: Closed-Loop Distillation of Reading Heuristics for Exploratory Manipulation Trace QA7 Jun 2026
- 2026Cosmos 3: Omnimodal World Models for Physical AI1 Jun 2026
- 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation29 Apr 2026
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 153
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?cited by 38
- 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Modelscited by 16
- 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulationcited by 10
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulationcited by 2
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation26 Jul 2026
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?17 Jun 2026
- 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models11 Jun 2026
- 2026Cosmos 3: Omnimodal World Models for Physical AI1 Jun 2026
- 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation29 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
- 2025How Good are Foundation Models in Step-by-Step Embodied Reasoning?
- 2025Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning
- 2025BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
- 2026Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack
- 2025ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning
- 2025EmbodiedBrain: Expanding Performance Boundaries of Task Planning for Embodied Intelligence
- 2026Towards general embodied intelligence: integrating large language models, knowledge bases, and reasoning capabilities to build the next generation of AI agents