# DuplexOmni: Real-Time Listening, Seeing, Thinking, and Speaking for Full-Duplex Interaction

arXiv: https://arxiv.org/abs/2606.09186 · HTML: https://mlsanity.com/paper/2606.09186/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2026.

## Built on by (most cited first, 8 of 8)

- [Realtime-Venus: A full-duplex interaction system with asynchronous delegation](https://mlsanity.com/paper/2609.13814/) (2026)
- [Multimodal Duplex Interaction Agent](https://mlsanity.com/paper/2609.08977/) (2026)
- [SALMONN-duo: Adaptive Dual-System Coordination for Full-Duplex Voice Agents](https://mlsanity.com/paper/2609.34247/) (2026)
- [Voice Memory for Agentic Speech Recognition](https://mlsanity.com/paper/2607.26410/) (2026)
- [LLMs are General Asynchronous Agents](https://mlsanity.com/paper/2609.35427/) (2026)
- [EvolvingAvatar: Interactive 3D Head Generation That Adapts as Conversations Unfold](https://mlsanity.com/paper/2609.35616/) (2026)
- [FlowAct-R2: Beyond Talking Avatar via Streaming Multimodal References and Proactive Agent Planning](https://mlsanity.com/paper/2609.35728/) (2026)
- [NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime](https://mlsanity.com/paper/2610.10787/) (2026)

## Abstract (arXiv)

Human interaction is continuous, multimodal, and full-duplex by nature. Although recent omni models have made substantial progress in unified speech, vision, and text modeling, combining seamless real-time interaction with complex reasoning and tool use remains challenging. We present DuplexOmni, a method for real-time multimodal full-duplex interaction. DuplexOmni separates model capability into an interaction layer and a thinking layer, which collaborate asynchronously in parallel. The interaction layer is implemented by the DuplexOmni model, an end-to-end system that processes streaming audio and video inputs while generating text and speech responses in real time. The thinking layer is a pluggable module that provides complex reasoning and tool-use capabilities. To support this method, we further develop a Writer-Director pipeline for constructing continuous-interaction training data. Experiments show that DuplexOmni achieves strong performance on multiple public benchmarks and exhibits natural full-duplex interaction ability.