mlsanityData to 7 Oct 2026

Detect, Explain, Escalate: Sustainable Dialogue Breakdown Management for LLM Agents

arXiv 2504.18839 ↗First posted 26 Apr 2025cs.CLcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated substantial capabilities in conversational AI applications, yet their susceptibility to dialogue breakdowns poses significant challenges to deployment reliability and user trust. This paper introduces a "Detect, Explain, Escalate" framework to manage dialogue breakdowns in LLM-powered agents, emphasizing resource-efficient operation. Our approach integrates two key strategies: (1) We fine-tune a compact 8B-parameter model, augmented with teacher-generated reasoning traces, which serves as an efficient real-time breakdown detector and explainer. This model demonstrates robust classification and calibration on English and Japanese dialogues, and generalizes to the BETOLD dataset, improving accuracy by 7% over its baseline. (2) We systematically evaluate frontier LLMs using advanced prompting (few-shot, chain-of-thought, analogical reasoning) for high-fidelity breakdown assessment. These are integrated into an "escalation" architecture where our efficient detector defers to larger models only when necessary, substantially reducing operational costs and computational overhead. Our fine-tuned model and prompting strategies achieve state-of-the-art performance on DBDC5 and strong results on BETOLD, outperforming specialized classifiers on DBDC5 and narrowing the performance gap to larger proprietary models. The proposed monitor-escalate pipeline reduces inference costs by 54%, providing a cost-effective and interpretable solution for robust conversational AI in high-impact domains. Code and models are publicly available.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 31 of its 58 references that are in the map. First posted on arXiv 26 Apr 2025.

Builds on

31 of 58 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  12. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
19 more
  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  3. 2024Mixtral of Expertscited by 1,785
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  5. 2024A Survey on LLM-as-a-Judgecited by 1,384
  6. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  7. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  8. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  9. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
  10. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  11. 2023Large Language Models as Analogical Reasonerscited by 107
  12. 2020SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustnesscited by 94
  13. 2024Online Cascade Learning for Efficient Inference over Streamscited by 19
  14. 2024CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Modelscited by 19
  15. 2023Leveraging Large Language Models for Automated Dialogue Analysiscited by 14
  16. 2022Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systemscited by 9
  17. 2020Improving Dialogue Breakdown Detection with Semi-Supervised Learningcited by 6
  18. 2023Chatbots and ChatGPT: A Bibliometric Analysis and Systematic Review of Publications in Web of Science and Scopus Databasescited by 4
  19. 2022A Semi-supervised Learning Approach with Two Teachers to Improve Breakdown Identification in Dialoguescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration
  2. 2024Leveraging LLMs for Dialogue Quality Measurement
  3. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions
  4. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents
  5. 2025ClarifyMT-Bench: Benchmarking and Improving Multi-Turn Clarification for Conversational Large Language Models
  6. 2024From LLM to Conversational Agent: A Memory Enhanced Architecture with Fine-Tuning of Large Language Models
  7. 2025Beyond Single Models: Enhancing LLM Detection of Ambiguity in Requests through Debate
  8. 2025Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logs