mlsanityData to 14 Oct 2026

RoboEgo System Card: An Omnimodal Model with Native Full Duplexity

arXiv 2506.01934 ↗First posted 2 Jun 2025cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Humans naturally process real-world multimodal information in a full-duplex manner. In artificial intelligence, replicating this capability is essential for advancing model development and deployment, particularly in embodied contexts. The development of multimodal models faces two primary challenges: (1) effectively handling more than three modalities-such as vision, audio, and text; and (2) delivering full-duplex responses to rapidly evolving human instructions. To facilitate research on models that support both omnimodal processing and full duplexity, we present RoboEgo (alias: FLM-Ego), a unified model system designed to address both challenges. RoboEgo incorporates a backbone architecture and algorithms that natively support full duplexity, achieving a theoretical duplex latency of 80 ms. In streaming visually grounded conversations under real-world conditions, RoboEgo exhibits superior responsiveness and speech naturalness, while maintaining comparable content qualities to state-of-the-art semi-duplex omnimodal models-a feat previously considered unattainable by native full-duplex systems.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

32 of 41 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  10. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  11. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
20 more
  1. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  2. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  3. 2025Qwen2.5-Omni Technical Reportcited by 736
  4. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  5. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  6. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  7. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  8. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  9. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
  10. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  11. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  12. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  13. 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 94
  14. 2025MinMo: A Multimodal Large Language Model for Seamless Voice Interactioncited by 79
  15. 2024OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversationcited by 60
  16. 2024Scaling Speech-Text Pre-training with Synthetic Interleaved Datacited by 47
  17. 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
  18. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  19. 2024Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Modelscited by 26
  20. 2025Toward Embodied AGI: A Review of Embodied AI and the Road Aheadcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.