mlsanityData to 7 Oct 2026

Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective

arXiv 2511.00739 ↗First posted 1 Nov 2025cs.AI cs.LG cs.MAcited by 3

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Agentic AI serving converts monolithic LLM-based inference to autonomous problem-solvers that can plan, call tools, perform reasoning, and adapt on the fly. Due to diverse task execution need, such serving heavily rely on heterogeneous CPU-GPU systems with majority of the external tools responsible for agentic capability, either run on or are orchestrated by the CPU. Towards having a deeper understanding of its role, this paper aims to characterize and analyze the system bottlenecks introduced by agentic AI workloads from a largely overlooked CPU-centric perspective. We first present a compile-time characterization of agentic AI execution and choose representative workloads to capture the algorithmic diversity. We then perform runtime characterization of the representative workloads analyzing the end-to-end latency and throughput on two different hardware systems to isolate respective architectural bottlenecks. Based on the insights on the bottlenecks, we finally present two scheduling optimizations, namely, 1. CPU-Aware Overlapped Micro-Batching (COMB) and 2. Mixed Agentic Scheduling (MAS) on homogeneous and heterogeneous agentic workloads, respectively. In specific, these methods optimize for improved CPU-GPU concurrent utilization while reducing skewed resource allocation for heterogeneous execution. Experimental evaluations on the two hardware systems demonstrate the efficacy of COMB in yielding up to 1.7x lower P50 latency in standalone homogeneous workload execution and up to 3.9x/1.8x lower service/total latency under homogeneous open-loop load. Additionally, for heterogeneous open-loop load, MAS can reduce the total latency for minority request-type by up to 2.37x/2.49x at P50/P90 percentile.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 75 references that are in the map. First posted on arXiv 1 Nov 2025.

Builds on

43 of 75 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Efficient and Scalable Agentic AI with Heterogeneous Systems25 Jul
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
29 more
  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  2. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  3. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  4. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  5. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  7. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  8. 2023Textbooks Are All You Needcited by 561
  9. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  10. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  11. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  12. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  13. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  14. 2025Small Language Models are the Future of Agentic AIcited by 254
  15. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 250
  16. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  17. 2021Internet-Augmented Dialogue Generationcited by 217
  18. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  19. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  20. 2023Large Language Models in Education: Vision and Opportunitiescited by 54
  21. 2024Large Language Models Can Self-Improve At Web Agent Taskscited by 54
  22. 2023GeoGPT: Understanding and Processing Geospatial Tasks through An Autonomous GPTcited by 40
  23. 2025The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspectivecited by 38
  24. 2024Teola: Towards End-to-End Optimization of LLM-based Applicationscited by 32
  25. 2025Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inferencecited by 32
  26. 2024Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Executioncited by 16
  27. 2024Accelerating Retrieval-Augmented Generationcited by 14
  28. 2025Characterizing and Optimizing LLM Inference Workloads on CPU-GPU Coupled Architecturescited by 8
  29. 2021LMStream: When Distributed Micro-Batch Stream Processing Systems Meet GPUcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systemscited by 5
  2. 2026Energy per Successful Goal: Goal-Level Energy Accounting for Agentic AI Systemscited by 1
  3. 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SwarmX: Agentic Scheduling for Low-Latency Agentic Systems
  2. 2026Architectural Implications of Agentic AI Workflows
  3. 2026MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems
  4. 2026SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters
  5. 2026Heterogeneous Computing: The Key to Powering the Future of AI Agent Inference
  6. 2026Agentic AI Workload Characteristics
  7. 2026Benchmarking LLM Serving Systems for Agentic AI Workloads with XPerf
  8. 2026End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services