Enhancing Cluster Resilience: LLM-agent Based Autonomous Intelligent Cluster Diagnosis System and Evaluation Framework
On the map: Retrieval-augmented generation → Agentic RAG, agents and memory
also in 1 more
Abstract (arXiv)
Recent advancements in Large Language Models (LLMs) and related technologies such as Retrieval-Augmented Generation (RAG) and Diagram of Thought (DoT) have enabled the creation of autonomous intelligent systems capable of performing cluster diagnostics and troubleshooting. By integrating these technologies with self-play methodologies, we have developed an LLM-agent system designed to autonomously diagnose and resolve issues within AI clusters. Our innovations include a knowledge base tailored for cluster diagnostics, enhanced LLM algorithms, practical deployment strategies for agents, and a benchmark specifically designed for evaluating LLM capabilities in this domain. Through extensive experimentation across multiple dimensions, we have demonstrated the superiority of our system in addressing the challenges faced in cluster diagnostics, particularly in detecting and rectifying performance issues more efficiently and accurately than traditional methods.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 22 of its 36 references that are in the map. First posted on arXiv 8 Nov 2024.
Builds on
22 of 36 references.
References · 22
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023Mistral 7Bcited by 3,065
- 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
10 more
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
- 2024Qwen2 Technical Reportcited by 2,076
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
- 2024Nemotron-4 340B Technical Reportcited by 143
- 2024On the Diagram of Thoughtcited by 16
- 2023CloudEval-YAML: A Practical Benchmark for Cloud Configuration Generationcited by 8
- 2023DLRover-RM: Resource Optimization for Deep Recommendation Models Training in the Cloudcited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
- 2023A Survey on Large Language Model based Autonomous Agents
- 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey
- 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
- 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents
- 2024Practical Considerations for Agentic LLM Systems
- 2024Coalitions of Large Language Models Increase the Robustness of AI Agents
- 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review