mlsanityData to 7 Oct 2026

Bian Que: An Agentic Framework with Flexible Skill Arrangement for Online System Operations

arXiv 2604.26805 ↗First posted 29 Apr 2026cs.AI cs.MAcited by 2

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Operating and maintaining (O&M) large-scale online engine systems (eg, search, recommendation and advertising) demands substantial human effort for release monitoring, alert response, and root cause analysis. Despite the inherent suitability of LLM-based agents for such operational scenarios, the critical bottleneck impeding their practical deployment lies not in reasoning, but in orchestration capability - specifically, the precise selection of relevant data (encompassing metrics, logs, and change events) and applicable knowledge (including handbook-defined rules and empirically derived practitioner experience) tailored to each individual operational event. Feeding all signals indiscriminately causes dilution and hallucination, while manually curating the event-to-(data, knowledge) mapping is intractable under dozens of daily releases. Here we present Bian Que, an agentic operating framework with three contributions: (i) The unified operational paradigm, which abstracts routine daily O&M actions into three canonical patterns: release interception, proactive inspection, and alert root cause analysis; (ii) The flexible Skill Arrangement, each predefined Skill explicitly defines the requisite data and operational knowledge for each specific context. Such Skills can be automatically generated and updated by LLM agents, and can also be iteratively optimized by on-call engineers via natural language instructions. (iii) The unified self-evolving mechanism, where each correction signal enables two parallel evolutionary pathways: distilling event memory into knowledge, and targeted refinement of Skills. Deployed on the e-commerce search engine of KuaiShou, Bian Que reduces alert volume by 75%, achieves 80% root-cause analysis accuracy, cuts mean time to resolution by over 50%, and attains a 99.0% pass rate on offline evaluations. Codes are at https://github.com/benchen4395/BianQue_Assistant.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 37 of its 46 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

37 of 46 references.

References · 37

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  9. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  10. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  11. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  12. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 250
25 more
  1. 2025OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignmentcited by 167
  2. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  3. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 129
  4. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  5. 2023TaskWeaver: A Code-First Agent Frameworkcited by 85
  6. 2023RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Modelscited by 62
  7. 2024Exploring LLM-based Agents for Root Cause Analysiscited by 60
  8. 2020Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Artcited by 55
  9. 2021Anomaly Detection and Failure Root Cause Analysis in (Micro)Service-Based Cloud Applications: A Surveycited by 47
  10. 2025OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Searchcited by 24
  11. 2023RAGLog: Log Anomaly Detection using Retrieval Augmented Generationcited by 19
  12. 2025STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Cloudscited by 18
  13. 2025A Survey of AIOps in the Era of Large Language Modelscited by 18
  14. 2024A Survey of AIOps for Failure Management in the Era of Large Language Modelscited by 16
  15. 2025LLM-based event log analysis techniques: A surveycited by 15
  16. 2025UniECS: Unified Multimodal E-Commerce Search Framework with Gated Cross-modal Fusioncited by 10
  17. 2025MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Toolscited by 9
  18. 2026Stalled, Biased, and Confused: Uncovering Reasoning Failures in LLMs for Cloud-Based Root Cause Analysiscited by 8
  19. 2026Live-Evo: Online Evolution of Agentic Memory from Continuous Feedbackcited by 8
  20. 2026IG-Search: Step-Level Information Gain Rewards for Search-Augmented Reasoningcited by 8
  21. 2026OneSearch-V2: The Latent Reasoning Enhanced Self-distillation Generative Search Frameworkcited by 6
  22. 2024Retrieval Augmented Generation-Based Incident Resolution Recommendation System for IT Supportcited by 2
  23. 2025Empowering AIOps: Leveraging Large Language Models for IT Operations Managementcited by 2
  24. 2026CSMCIR: CoT-Enhanced Symmetric Alignment with Memory Bank for Composed Image Retrievalcited by 1
  25. 2026AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosiscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Tool-Making and Self-Evolving LLM Agents in Low-Latency Systemscited by 2
  2. 2026Plan Before Search: Search Agents Need Plancited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026The Agent Operating System (AOS): A Reference Operating Architecture for Distributed Agentic Systems
  2. 2026Agentic RCA for Internet-Scale Services Using Constrained Creativity
  3. 2026Organizational Control Layer: Governance Infrastructure at the Execution Boundary of LLM Agent Systems
  4. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites
  5. 2025Multi-Agent LLM Orchestration Achieves Deterministic, High-Quality Decision Support for Incident Response
  6. 2026AgentTrace: A Structured Logging Framework for Agent System Observability
  7. 2026Apodex 1.1: Scaling Agentic Intelligence for Complex Work
  8. 2026Agent Operating Systems (AOS): Integrating Agentic Control Planes into, and Beyond, Traditional Operating Systems