iFLYTEK-Embodied-Omni Technical Report
Not on the map. Why?
Abstract (arXiv)
General-purpose embodied agents must understand multimodal instructions, anticipate how their environment will evolve, and produce precise control actions over extended horizons. Existing approaches typically specialize in visual-language reasoning, video-based world modeling, or action generation, while cascaded pipelines that first synthesize future observations and then infer actions can introduce interface bottlenecks and compound prediction errors. We present iFLYTEK-Embodied-Omni, a unified multimodal foundation model that jointly models vision(videos and images), language, and action within a single Omni framework. Its modality-specific visual-language, video-generation, and action-generation components communicate through shared multimodal self-attention. This design establishes brain-cerebellum collaboration: the vision-language modeland video generation model form a high-level brain for instruction understanding, task planning, progress tracking, and future visual-state prediction, whereas the action generation modelserves as a low-level cerebellum that directly converts planned subgoals and shared multimodal context into executable action chunks. To develop these capabilities, we combine action-annotated and action-free embodied videos from human demonstrations and robot interactions with embodied reasoning, embodied perception, and general-purpose image-text data to construct a comprehensive dataset. We further adopt a four-stage strategy that progressively trains the VLM, VGM, and AGM before jointly fine-tuning the complete model.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 24 Jun 2026.
Builds on
12 of 22 references.
References · 12
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
- 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
- 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
- 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Igniting VLMs toward the Embodied Space
- 2026Pelican-Unify 1.0: A Unified Embodied Intelligence Model for Understanding, Reasoning, Imagination and Action
- 2025RynnEC: Bringing MLLMs into Embodied World
- 2025ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
- 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
- 2024Can-Do! A Dataset and Neuro-Symbolic Grounded Framework for Embodied Planning with Large Multimodal Models
- 2026MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence
- 2025BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning