mlsanityData to 14 Oct 2026

ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management

arXiv 2601.21473 ↗First posted 29 Jan 2026cs.AI cs.DCcited by 2

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

LLM-based multi-agent simulations are increasingly adopted across application domains, but remain difficult to scale due to GPU memory pressure. Each agent maintains private GPU-resident states, including models, prefix caches, and adapters, which quickly exhaust device memory as the agent count grows. We identify two key properties of these workloads: sparse agent activation and an estimable agent invocation order. Based on an analysis of representative workload classes, we introduce invocation distance, a unified abstraction that estimates the relative order in which agents will issue future LLM requests. Leveraging this abstraction, we present ScaleSim, a memory-efficient LLM serving system for large-scale multi-agent simulations. ScaleSim enables proactive prefetching and priority-based eviction, supports diverse agent-specific memory through a modular interface, and achieves up to 1.74x speedup over SGLang on simulation benchmarks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 47 of its 57 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

47 of 57 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Pie: A Programmable Serving System for Emerging LLM Applications28 Oct
  2. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows10 Jul

Other references · 45

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  7. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  8. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  9. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  10. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
  11. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  12. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
33 more
  1. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  2. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  3. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 289
  4. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
  5. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  6. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
  7. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 249
  8. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 233
  9. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 203
  10. 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 186
  11. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 185
  12. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  13. 2023EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activitiescited by 154
  14. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Modelscited by 153
  15. 2023Online Speculative Decodingcited by 113
  16. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 99
  17. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 98
  18. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 86
  19. 2023Punica: Multi-Tenant LoRA Servingcited by 83
  20. 2023Humanoid Agents: Platform for Simulating Human-like Generative Agentscited by 73
  21. 2024Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agentcited by 52
  22. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 50
  23. 2024Project Sid: Many-agent simulations toward AI civilizationcited by 49
  24. 2024Emergence of Social Norms in Generative Agent Societies: Principles and Architecturecited by 41
  25. 2024InferCept: Efficient Intercept Support for Augmented Large Language Model Inferencecited by 33
  26. 2024BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batchingcited by 32
  27. 2024Marconi: Prefix Caching for the Era of Hybrid LLMscited by 31
  28. 2024Large Language Model-driven Multi-Agent Simulation for News Diffusion Under Different Network Structurescited by 14
  29. 2024AI Metropolis: Scaling Large Language Model-based Multi-Agent Simulation with Out-of-order Executioncited by 14
  30. 2025TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrievalcited by 9
  31. 2024LiPUP-MA: A Residential Experience-centric Multi-Agent Framework for Living-in-the-loop Participatory Urban Planningcited by 8
  32. 2025Parallelized Planning-Acting for Efficient LLM-based Multi-Agent Systems in Minecraftcited by 5
  33. 2025Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Managementcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharingcited by 8
  2. 2026SmoothAgent: Efficient Long-Horizon LLM-Based Agent Serving with Lookahead Context Engineeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications
  2. 2026Pancake: Hierarchical Memory System for Multi-Agent LLM Serving
  3. 2026Scaling LLM-Driven Multi-Agent Systems: Design Principles and Architectural Scalability Analysis
  4. 2026Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control
  5. 2026JointServe: Serving Multi-Model Agents on Memory-Constrained GPUs
  6. 2026Benchmark Test-Time Scaling of General LLM Agents
  7. 2026LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents
  8. 2026CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control