# SALMONN-duo: Adaptive Dual-System Coordination for Full-Duplex Voice Agents arXiv: https://arxiv.org/abs/2609.34247 · HTML: https://mlsanity.com/paper/2609.34247/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. - Builds on 42 of its 47 references that are in the map. - First posted on arXiv 28 Sep 2026. ## Placed in - LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/) ## Builds on (most cited first, 30 of 42) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025) - [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018) - [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017) - [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023) - [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025) - [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024) - [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024) - [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025) - [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023) - [CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models](https://mlsanity.com/paper/2412.10117/) (2024) - [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023) - [GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot](https://mlsanity.com/paper/2412.02612/) (2024) - [Kimi-Audio Technical Report](https://mlsanity.com/paper/2504.18425/) (2025) - [Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM](https://mlsanity.com/paper/2411.00774/) (2024) - [Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph](https://mlsanity.com/paper/2406.15627/) (2024) - [Step-Audio 2 Technical Report](https://mlsanity.com/paper/2507.16632/) (2025) - [Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context](https://mlsanity.com/paper/2309.08105/) (2023) - [MinMo: A Multimodal Large Language Model for Seamless Voice Interaction](https://mlsanity.com/paper/2501.06282/) (2025) - [MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction](https://mlsanity.com/paper/2604.27393/) (2026) - [Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning](https://mlsanity.com/paper/2506.09033/) (2025) - [Baichuan-Audio: A Unified Framework for End-to-End Speech Interaction](https://mlsanity.com/paper/2502.17239/) (2025) - [ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration](https://mlsanity.com/paper/2511.21689/) (2025) - [τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains](https://mlsanity.com/paper/2603.13686/) (2026) - [SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Model](https://mlsanity.com/paper/2505.15670/) (2025) - [SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversation](https://mlsanity.com/paper/2505.17060/) (2025) - [When to Retrieve: Teaching LLMs to Utilize Information Retrieval Effectively](https://mlsanity.com/paper/2404.19705/) (2024) - [From Self-Evolving Synthetic Data to Verifiable-Reward RL: Post-Training Multi-turn Interactive Tool-Using Agents](https://mlsanity.com/paper/2601.22607/) (2026) - [Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation](https://mlsanity.com/paper/2603.03116/) (2026) - [Can Speech LLMs Think while Listening?](https://mlsanity.com/paper/2510.07497/) (2025) ## Built on by (most cited first, 1 of 1) - [Lost in the Fallback: How Reliably Do Typed Decision Models Use Fallback Options?](https://mlsanity.com/paper/2610.19856/) (2026) ## Abstract (arXiv) Full-duplex speech large language models (LLMs) enable low-latency, natural voice interaction. However, real-world agents must also use tools and perform deliberative reasoning-operations whose variable latency and computational cost conflict with the stringent timing requirements of real-time conversation. To reconcile these demands, we propose SALMONN-duo, an adaptive dual-system voice agent inspired by dual-process theories of cognition. SALMONN-duo separates real-time interaction from deliberative computation by pairing an always-on, fast-thinking full-duplex speech LLM (system 1) with a powerful asynchronous slow-thinking LLM agent (system 2). Beyond handling real-time interaction, system 1 learns when to answer directly and when to delegate, remaining responsive during backend execution and seamlessly integrating returned information into the ongoing dialogue without exposing tool traces or losing conversational context. Evaluations on single-turn spoken question answering (QA) and multi-turn conversations demonstrate that adaptive delegation substantially improves accuracy on knowledge-intensive and multi-hop reasoning questions, while knowledge-boundary-aware training avoids unnecessary system 2 invocations. On a customized version of τ-Voice, SALMONN-duo further demonstrates its ability to complete environment-grounded, policy-constrained tasks through multi-turn interactions in realistic business scenarios. Finally, cost-aware reinforcement learning further enhances the trade-off between task performance and backend usage across the QA and conversation tasks, while improving task success and response safety on τ-Voice with an acceptable increase in the delegation rate.