# Building AI Agents for Autonomous Clouds: Challenges and Design Principles

arXiv: https://arxiv.org/abs/2407.12165 · HTML: https://mlsanity.com/paper/2407.12165/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 119 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 22 of 22)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [ADBench: Anomaly Detection Benchmark](https://mlsanity.com/paper/2206.09426/) (2022)
- [Automatic Root Cause Analysis via Large Language Models for Cloud Incidents](https://mlsanity.com/paper/2305.15778/) (2023)
- [RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models](https://mlsanity.com/paper/2310.16340/) (2023)
- [Tools and Benchmarks for Automated Log Parsing](https://mlsanity.com/paper/1811.03509/) (2018)
- [Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4](https://mlsanity.com/paper/2401.13810/) (2024)
- [Assess and Summarize: Improve Outage Understanding with Large Language Models](https://mlsanity.com/paper/2305.18084/) (2023)
- [DeCaf: Diagnosing and Triaging Performance Issues in Large-Scale Cloud Services](https://mlsanity.com/paper/1910.05339/) (2019)
- [Robust Multimodal Failure Detection for Microservice Systems](https://mlsanity.com/paper/2305.18985/) (2023)
- [AutoTSG: Learning and Synthesis for Incident Troubleshooting](https://mlsanity.com/paper/2205.13457/) (2022)
- [BigDataBench: a Big Data Benchmark Suite from Internet Services](https://mlsanity.com/paper/1401.1406/) (2014)
- [Observability and Chaos Engineering on System Calls for Containerized Applications in Docker](https://mlsanity.com/paper/1907.13039/) (2019)
- [DeepAnalyze: Learning to Localize Crashes at Scale](https://mlsanity.com/paper/2109.14326/) (2021)
- [The Workflow Trace Archive: Open-Access Data from Public and Private Computing Infrastructures -- Technical Report](https://mlsanity.com/paper/1906.07471/) (2019)

## Built on by (most cited first, 19 of 19)

- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [JITServe: SLO-aware LLM Serving with Imprecise Request Information](https://mlsanity.com/paper/2504.20068/) (2025)
- [A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System](https://mlsanity.com/paper/2510.09721/) (2025)
- [Cloud Infrastructure Management in the Age of AI Agents](https://mlsanity.com/paper/2506.12270/) (2025)
- [Unveiling the Landscape of LLM Deployment in the Wild: An Empirical Study](https://mlsanity.com/paper/2505.02502/) (2025)
- [ORCA-bench: How Ready Are Language Model Agents for Oncall?](https://mlsanity.com/paper/2607.28545/) (2026)
- [Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage](https://mlsanity.com/paper/2506.06472/) (2025)
- [Generative Caching for Structurally Similar Prompts and Responses](https://mlsanity.com/paper/2511.17565/) (2025)
- [Towards a Playground to Democratize Experimentation and Benchmarking of AI Agents for Network Troubleshooting](https://mlsanity.com/paper/2507.01997/) (2025)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [What Challenges Do Developers Face in AI Agent Systems? An Empirical Study on Stack Overflow & GitHub Issues](https://mlsanity.com/paper/2510.25423/) (2025)
- [RIVA: Leveraging LLM Agents for Reliable Configuration Drift Detection](https://mlsanity.com/paper/2603.02345/) (2026)
- [A Feature Engineering Approach for Business Impact-Oriented Failure Detection in Distributed Instant Payment Systems](https://mlsanity.com/paper/2510.21710/) (2025)
- [GSAR: Typed Grounding for Hallucination Detection and Recovery in Multi-Agent LLMs](https://mlsanity.com/paper/2604.23366/) (2026)
- [InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk](https://mlsanity.com/paper/2608.11234/) (2026)
- [Interactive Clarification for Cloud Infrastructure-as-Code Synthesis](https://mlsanity.com/paper/2604.02382/) (2026)
- [Towards a Systems Foundation for Agentic Cloud Management](https://mlsanity.com/paper/2607.25883/) (2026)
- [Towards Agentic Cloud Engineering: Graph and Loop Engineering with a Zero-Trust Agent Harness](https://mlsanity.com/paper/2609.00050/) (2026)
- [LLMs are General Asynchronous Agents](https://mlsanity.com/paper/2609.35427/) (2026)

## Abstract (arXiv)

The rapid growth in the use of Large Language Models (LLMs) and AI Agents as part of software development and deployment is revolutionizing the information technology landscape. While code generation receives significant attention, a higher-impact application lies in using AI agents for operational resilience of cloud services, which currently require significant human effort and domain knowledge. There is a growing interest in AI for IT Operations (AIOps) which aims to automate complex operational tasks, like fault localization and root cause analysis, thereby reducing human intervention and customer impact. However, achieving the vision of autonomous and self-healing clouds through AIOps is hampered by the lack of standardized frameworks for building, evaluating, and improving AIOps agents. This vision paper lays the groundwork for such a framework by first framing the requirements and then discussing design decisions that satisfy them. We also propose AIOpsLab, a prototype implementation leveraging agent-cloud-interface that orchestrates an application, injects real-time faults using chaos engineering, and interfaces with an agent to localize and resolve the faults. We report promising results and lay the groundwork to build a modular and robust framework for building, evaluating, and improving agents for autonomous clouds.