# Bian Que: An Agentic Framework with Flexible Skill Arrangement for Online System Operations arXiv: https://arxiv.org/abs/2604.26805 ยท HTML: https://mlsanity.com/paper/2604.26805/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. - Builds on 37 of its 46 references that are in the map. - First posted on arXiv 29 Apr 2026. ## Placed in - LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/) ## Builds on (most cited first, 30 of 37) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023) - [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023) - [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023) - [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023) - [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023) - [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023) - [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023) - [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025) - [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023) - [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024) - [Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions](https://mlsanity.com/paper/2306.02224/) (2023) - [OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment](https://mlsanity.com/paper/2502.18965/) (2025) - [MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents](https://mlsanity.com/paper/2602.02474/) (2026) - [Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory](https://mlsanity.com/paper/2511.20857/) (2025) - [A Survey on Self-Evolution of Large Language Models](https://mlsanity.com/paper/2404.14387/) (2024) - [TaskWeaver: A Code-First Agent Framework](https://mlsanity.com/paper/2311.17541/) (2023) - [RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models](https://mlsanity.com/paper/2310.16340/) (2023) - [Exploring LLM-based Agents for Root Cause Analysis](https://mlsanity.com/paper/2403.04123/) (2024) - [Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art](https://mlsanity.com/paper/2004.00433/) (2020) - [Anomaly Detection and Failure Root Cause Analysis in (Micro)Service-Based Cloud Applications: A Survey](https://mlsanity.com/paper/2105.12378/) (2021) - [OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Search](https://mlsanity.com/paper/2509.03236/) (2025) - [RAGLog: Log Anomaly Detection using Retrieval Augmented Generation](https://mlsanity.com/paper/2311.05261/) (2023) - [STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds](https://mlsanity.com/paper/2506.02009/) (2025) - [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025) - [A Survey of AIOps for Failure Management in the Era of Large Language Models](https://mlsanity.com/paper/2406.11213/) (2024) - [LLM-based event log analysis techniques: A survey](https://mlsanity.com/paper/2502.00677/) (2025) - [MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Tools](https://mlsanity.com/paper/2510.24284/) (2025) - [UniECS: Unified Multimodal E-Commerce Search Framework with Gated Cross-modal Fusion](https://mlsanity.com/paper/2508.13843/) (2025) - [IG-Search: Step-Level Information Gain Rewards for Search-Augmented Reasoning](https://mlsanity.com/paper/2604.15148/) (2026) ## Built on by (most cited first, 2 of 2) - [Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems](https://mlsanity.com/paper/2607.08010/) (2026) - [Plan Before Search: Search Agents Need Plan](https://mlsanity.com/paper/2605.28354/) (2026) ## Abstract (arXiv) Operating and maintaining (O&M) large-scale online engine systems (eg, search, recommendation and advertising) demands substantial human effort for release monitoring, alert response, and root cause analysis. Despite the inherent suitability of LLM-based agents for such operational scenarios, the critical bottleneck impeding their practical deployment lies not in reasoning, but in orchestration capability - specifically, the precise selection of relevant data (encompassing metrics, logs, and change events) and applicable knowledge (including handbook-defined rules and empirically derived practitioner experience) tailored to each individual operational event. Feeding all signals indiscriminately causes dilution and hallucination, while manually curating the event-to-(data, knowledge) mapping is intractable under dozens of daily releases. Here we present Bian Que, an agentic operating framework with three contributions: (i) The unified operational paradigm, which abstracts routine daily O&M actions into three canonical patterns: release interception, proactive inspection, and alert root cause analysis; (ii) The flexible Skill Arrangement, each predefined Skill explicitly defines the requisite data and operational knowledge for each specific context. Such Skills can be automatically generated and updated by LLM agents, and can also be iteratively optimized by on-call engineers via natural language instructions. (iii) The unified self-evolving mechanism, where each correction signal enables two parallel evolutionary pathways: distilling event memory into knowledge, and targeted refinement of Skills. Deployed on the e-commerce search engine of KuaiShou, Bian Que reduces alert volume by 75%, achieves 80% root-cause analysis accuracy, cuts mean time to resolution by over 50%, and attains a 99.0% pass rate on offline evaluations. Codes are at https://github.com/benchen4395/BianQue_Assistant.