mlsanityData to 13 Oct 2026

HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents

arXiv 2604.07430 ↗First posted 8 Apr 2026cs.CVcited by 18

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Vision-Language Models (VLMs) and the demands of embodied agents, our models are developed to enhance the core capabilities required by embodied intelligence: spatial and temporal visual perception, alongside advanced embodied reasoning for prediction, interaction, and planning. The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning. To support the fine-grained visual perception essential for embodied tasks, we adopt a Mixture-of-Transformers (MoT) architecture to enable modality-specific computing. By incorporating latent tokens, this design effectively enhances the perceptual representation of the models. To improve reasoning capabilities, we introduce an iterative, self-evolving post-training paradigm. Furthermore, we employ on-policy distillation to transfer the advanced capabilities of the large model to the smaller variant, thereby maximizing the performance potential of the compact model. Extensive evaluations across 22 benchmarks, spanning visual perception, spatial reasoning, and embodied understanding, demonstrate the effectiveness of our approach. Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro. In downstream robot control experiments, we leverage our robust VLM foundation to train an effective Vision-Language-Action (VLA) model, achieving compelling results in real-world physical evaluations. Code and models are open-sourced at https://github.com/Tencent-Hunyuan/HY-Embodied.

Key facts. Cited by 18 papers in the map (data through 13 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 61 of its 80 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

61 of 80 references.

References · 61

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  2. 2017Proximal Policy Optimization Algorithmscited by 14,592
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
  6. 2023Segment Anythingcited by 8,195
  7. 2023Visual Instruction Tuningcited by 7,827
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,541
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,463
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,096
  11. 2025Qwen3-VL Technical Reportcited by 2,663
  12. 2019Towards VQA Models That Can Readcited by 1,705
38 more
  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,676
  2. 2024Depth Anything V2cited by 1,399
  3. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,295
  4. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,154
  5. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,154
  6. 2016Modeling Context in Referring Expressionscited by 1,057
  7. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,041
  8. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 873
  9. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 857
  10. 2023Vision Transformers Need Registerscited by 850
  11. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 827
  12. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 818
  13. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 752
  14. 2024PaliGemma: A versatile 3B VLM for transfercited by 686
  15. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 588
  16. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 577
  17. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 534
  18. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 470
  19. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 445
  20. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 397
  21. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 323
  22. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 295
  23. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 243
  24. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 227
  25. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
  26. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 193
  27. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 188
  28. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 172
  29. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 168
  30. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 156
  31. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 135
  32. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 135
  33. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 133
  34. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
  35. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 117
  36. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 111
  37. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 111
  38. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 101

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 18.

18 papers.

Sort:

By area:

  1. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 153
  2. 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?cited by 38
  3. 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Modelscited by 16
  4. 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulationcited by 10
  5. 2026PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformationcited by 7
  6. 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agentscited by 7
  7. 2026Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Modelscited by 5
  8. 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AIcited by 5
  9. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 5
  10. 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulationcited by 2
  11. 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imaginationcited by 1
  12. 2026SpatialCLI: Learning to Reason With Spatial Tools, Then Without Themcited by 1
6 more
  1. 2026PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloudcited by 1
  2. 2026CometVLA: Co-Training on an Embodied Data Pyramid towards Physical Understandingcited by 1
  3. 2026When Video Misreads: Closed-Loop Distillation of Reading Heuristics for Exploratory Manipulation Trace QAcited by 0
  4. 2026N0-Foundation: Towards the Age of Tactile Intelligencecited by 0
  5. 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Modelscited by 0
  6. 2026MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
  2. 2025How Good are Foundation Models in Step-by-Step Embodied Reasoning?
  3. 2025Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning
  4. 2025BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
  5. 2026Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack
  6. 2025ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning
  7. 2025EmbodiedBrain: Expanding Performance Boundaries of Task Planning for Embodied Intelligence
  8. 2026Towards general embodied intelligence: integrating large language models, knowledge bases, and reasoning capabilities to build the next generation of AI agents