# DuCCAE: A Hybrid Engine for Immersive Conversation via Collaboration, Augmentation, and Evolution

arXiv: https://arxiv.org/abs/2603.19248 · HTML: https://mlsanity.com/paper/2603.19248/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 74 references that are in the map.
- First posted on arXiv 25 Feb 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 32)

- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions](https://mlsanity.com/paper/2503.23278/) (2025)
- [Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks](https://mlsanity.com/paper/2503.09572/) (2025)
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey](https://mlsanity.com/paper/2404.11584/) (2024)
- [Scaling Long-Horizon LLM Agent via Context-Folding](https://mlsanity.com/paper/2510.11967/) (2025)
- [Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction](https://mlsanity.com/paper/2501.03218/) (2025)
- [AIOS: LLM Agent Operating System](https://mlsanity.com/paper/2403.16971/) (2024)
- [Searching for Best Practices in Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01219/) (2024)
- [Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation](https://mlsanity.com/paper/2406.07867/) (2024)
- [Agents Thinking Fast and Slow: A Talker-Reasoner Architecture](https://mlsanity.com/paper/2410.08328/) (2024)
- [Guided Self-Evolving LLMs with Minimal Human Supervision](https://mlsanity.com/paper/2512.02472/) (2025)
- [Embodied AI: From LLMs to World Models](https://mlsanity.com/paper/2509.20021/) (2025)
- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Model](https://mlsanity.com/paper/2410.10696/) (2024)
- [EmpathyEar: An Open-source Avatar Multimodal Empathetic Chatbot](https://mlsanity.com/paper/2406.15177/) (2024)
- [MNER-QG: An End-to-End MRC framework for Multimodal Named Entity Recognition with Query Grounding](https://mlsanity.com/paper/2211.14739/) (2022)
- [ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer](https://mlsanity.com/paper/2408.03284/) (2024)
- [Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimization](https://mlsanity.com/paper/2109.06471/) (2021)
- [The Orchestration of Multi-Agent Systems: Architectures, Protocols, and Enterprise Adoption](https://mlsanity.com/paper/2601.13671/) (2026)
- [Learning to Select Context in a Hierarchical and Global Perspective for Open-domain Dialogue Generation](https://mlsanity.com/paper/2102.09282/) (2021)
- [Asynchronous Tool Usage for Real-Time Agents](https://mlsanity.com/paper/2410.21620/) (2024)
- [Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents](https://mlsanity.com/paper/2507.22352/) (2025)
- [When to Reason: Semantic Router for vLLM](https://mlsanity.com/paper/2510.08731/) (2025)
- [Chameleon: Adaptive Caching and Scheduling for Many-Adapter LLM Inference Environments](https://mlsanity.com/paper/2411.17741/) (2024)
- [LLIA -- Enabling Low-Latency Interactive Avatars: Real-Time Audio-Driven Portrait Video Generation with Diffusion Models](https://mlsanity.com/paper/2506.05806/) (2025)
- [ALAS: A Stateful Multi-LLM Agent Framework for Disruption-Aware Planning](https://mlsanity.com/paper/2505.12501/) (2025)
- [Spoken Conversational Agents with Large Language Models](https://mlsanity.com/paper/2512.02593/) (2025)
- [The Six Sigma Agent: Achieving Enterprise-Grade Reliability in LLM Systems Through Consensus-Driven Decomposed Execution](https://mlsanity.com/paper/2601.22290/) (2026)

## Built on by (most cited first, 1 of 1)

- [Learning User-Aware Recall: Personalized Retrieval in Long-Term Conversational Memory](https://mlsanity.com/paper/2607.00017/) (2026)

## Abstract (arXiv)

Immersive conversational systems in production face a persistent trade-off between responsiveness and long-horizon task capability. Real-time interaction is achievable for lightweight turns, but requests involving planning and tool invocation (e.g., search and media generation) produce heavy-tail execution latency that degrades turn-taking, persona consistency, and user trust. To address this challenge, we propose DuCCAE (Conversation while Collaboration with Augmentation and Evolution), a hybrid engine for immersive conversation deployed within Baidu Search, serving millions of users. DuCCAE decouples real-time response generation from asynchronous agentic execution and synchronizes them via a shared state that maintains session context and execution traces, enabling asynchronous results to be integrated back into the ongoing dialogue. The system orchestrates five subsystems-Info, Conversation, Collaboration, Augmentation, and Evolution-to support multi-agent collaboration and continuous improvement. We evaluate DuCCAE through a comprehensive framework that combines offline benchmarking on the Du-Interact dataset and large-scale production evaluation within Baidu Search. Experimental results demonstrate that DuCCAE outperforms strong baselines in agentic execution reliability and dialogue quality while reducing latency to fit strict real-time budgets. Crucially, deployment metrics since June 2025 confirm substantial real-world effectiveness, evidenced by a tripling of Day-7 user retention to 34.2% and a surge in the complex task completion rate to 65.2%. Our hybrid architecture successfully preserves conversational continuity while enabling reliable agentic execution, offering practical guidelines for deploying scalable agentic systems in industrial settings.