Data to 14 Oct 2026

From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale

arXiv 2609.05758 ↗First posted 4 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Conversational assistants can blend retrieval, action selection, escalation, and wording in a single model path, or separate those roles. We report a production migration of a customer-support assistant at a large accommodation marketplace (millions of conversations per month, 11 languages, 10-second P90). Dynamic Response (DR) replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools plus a smaller generator that writes from a backend-validated context contract. Because the migration also changed prompts, alignment, and serving, we attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns: typed entity selection moves the reservation selector to a precision-first operating point (precision 8.3% to 89.1%, recall 75.2% to 67.3%), and typed action IDs with a membership check remove observed structured-action hallucination (2.14% to 0.0%). A low-ramp A/B test reproduces the replay escalation reductions: hard-escalation responses fall from 5.60% to 3.08% and soft-escalation responses from 9.56% to 2.49%, while production handoff volume holds roughly steady; self-solve is directional (+5.1 points, 95% CI [-2, +12]). Serving optimizations cut orchestrator P90 latency from 3.87s to 2.24s on a GPU footprint reduced by roughly one-third, and self-hosting reduces estimated annual model-serving cost by more than an order of magnitude.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 26 of its 33 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

26 of 33 references.

Closest ancestors · 1

Same direction, newest first

2025Agent-in-the-Loop: A Data Flywheel for Continuous Improvement in LLM-based Customer Support8 Oct

Other references · 25

2022Training language models to follow instructions with human feedbackcited by 15,971
2017Proximal Policy Optimization Algorithmscited by 15,215
2015Distilling the Knowledge in a Neural Networkcited by 10,324
2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988

13 more

2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
2022Red Teaming Language Models with Language Modelscited by 1,026
2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
2025ECom-Bench: Can LLM Agent Resolve Real-World E-commerce Customer Support Issues?cited by 27
2024ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questionscited by 10
2025Incremental Summarization for Customer Support via Progressive Note-Taking and Agent Feedbackcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way