mlsanityData to 7 Oct 2026

iFLYTEK-Embodied-Omni Technical Report

arXiv 2607.02542 ↗First posted 24 Jun 2026cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

General-purpose embodied agents must understand multimodal instructions, anticipate how their environment will evolve, and produce precise control actions over extended horizons. Existing approaches typically specialize in visual-language reasoning, video-based world modeling, or action generation, while cascaded pipelines that first synthesize future observations and then infer actions can introduce interface bottlenecks and compound prediction errors. We present iFLYTEK-Embodied-Omni, a unified multimodal foundation model that jointly models vision(videos and images), language, and action within a single Omni framework. Its modality-specific visual-language, video-generation, and action-generation components communicate through shared multimodal self-attention. This design establishes brain-cerebellum collaboration: the vision-language modeland video generation model form a high-level brain for instruction understanding, task planning, progress tracking, and future visual-state prediction, whereas the action generation modelserves as a low-level cerebellum that directly converts planned subgoals and shared multimodal context into executable action chunks. To develop these capabilities, we combine action-annotated and action-free embodied videos from human demonstrations and robot interactions with embodied reasoning, embodied perception, and general-purpose image-text data to construct a comprehensive dataset. We further adopt a four-stage strategy that progressively trains the VLM, VGM, and AGM before jointly fine-tuning the complete model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

12 of 22 references.

References · 12

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  4. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  5. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  6. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  7. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  8. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  9. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  10. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  11. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  12. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Igniting VLMs toward the Embodied Space
  2. 2026Pelican-Unify 1.0: A Unified Embodied Intelligence Model for Understanding, Reasoning, Imagination and Action
  3. 2025RynnEC: Bringing MLLMs into Embodied World
  4. 2025ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
  5. 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
  6. 2024Can-Do! A Dataset and Neuro-Symbolic Grounded Framework for Embodied Planning with Large Multimodal Models
  7. 2026MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence
  8. 2025BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning