RoboEgo System Card: An Omnimodal Model with Native Full Duplexity
Not on the map. Why?
Abstract (arXiv)
Humans naturally process real-world multimodal information in a full-duplex manner. In artificial intelligence, replicating this capability is essential for advancing model development and deployment, particularly in embodied contexts. The development of multimodal models faces two primary challenges: (1) effectively handling more than three modalities-such as vision, audio, and text; and (2) delivering full-duplex responses to rapidly evolving human instructions. To facilitate research on models that support both omnimodal processing and full duplexity, we present RoboEgo (alias: FLM-Ego), a unified model system designed to address both challenges. RoboEgo incorporates a backbone architecture and algorithms that natively support full duplexity, achieving a theoretical duplex latency of 80 ms. In streaming visually grounded conversations under real-world conditions, RoboEgo exhibits superior responsiveness and speech naturalness, while maintaining comparable content qualities to state-of-the-art semi-duplex omnimodal models-a feat previously considered unattainable by native full-duplex systems.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 2 Jun 2025.
Builds on
32 of 41 references.
References · 32
- 2017Attention Is All You Needcited by 60,361
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023GPT-4 Technical Reportcited by 18,396
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
20 more
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
- 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
- 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
- 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
- 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
- 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
- 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
- 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
- 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 94
- 2025MinMo: A Multimodal Large Language Model for Seamless Voice Interactioncited by 79
- 2024OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversationcited by 60
- 2024Scaling Speech-Text Pre-training with Synthetic Interleaved Datacited by 47
- 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
- 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
- 2024Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Modelscited by 26
- 2025Toward Embodied AGI: A Review of Embodied AI and the Road Aheadcited by 8
Built on by
Citing papers per year: 2025: 3; 2026: 2.
5 papers.
- 2026Do Audio-Visual Large Language Models Really See and Hear?cited by 9
- 2025EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Modelscited by 5
- 2025End-to-end Listen, Look, Speak and Actcited by 5
- 2025TurnGuide: Enhancing Meaningful Full Duplex Spoken Interactions via Dynamic Turn-Level Text-Speech Interleavingcited by 1
- 2026MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversationcited by 1
- 2026MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversation29 Sep 2026
- 2026Do Audio-Visual Large Language Models Really See and Hear?3 Apr 2026
- 2025End-to-end Listen, Look, Speak and Act19 Oct 2025
- 2025EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models15 Sep 2025
- 2025TurnGuide: Enhancing Meaningful Full Duplex Spoken Interactions via Dynamic Turn-Level Text-Speech Interleaving10 Aug 2025