mlsanityData to 14 Oct 2026

Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics

arXiv 2604.07144 ↗First posted 8 Apr 2026cs.DCcited by 1

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Modern Large Language Model (LLM) serving operates in highly volatile environments characterized by severe runtime dynamics, such as workload fluctuations and elastic cluster autoscaling. Traditional serving systems rely on static, human-engineered serving policies (e.g., scheduling algorithms and rescheduling strategies) to manage these dynamics. However, these policies must navigate deeply intertwined runtime trade-offs (e.g., scheduling overhead vs. execution efficiency, rescheduling frequency vs. reconfiguration overhead), whose optimal balance is workload-specific and shifts continuously as runtime conditions evolve, rendering any fixed policy fundamentally unable to adapt. We propose Autopoiesis, a novel online self-evolving system that shifts LLM serving from static policy deployment to continuous online policy evolution. First, Autopoiesis introduces an LLM-driven program synthesis workflow to evolve serving policies with respect to real-time observed dynamics, where the evolved policies reflect the optimal decision in navigating the complex, multi-dimensional trade-off space. Second, Autopoiesis enables this synthesis process to operate continuously during serving, observing real-world system behavior, and rewriting the policy code as runtime trade-offs shift, thereby transforming policy design from a one-time offline endeavor into an ongoing system component, enabling autonomous adaptation to evolving runtime conditions. Together, we establish a new paradigm: Serving policies are no longer static artifacts designed by humans before deployment, but living code that LLMs continuously evolve throughout deployment to navigate runtime trade-offs beyond human design. We evaluate Autopoiesis across diverse runtime dynamics and show up to 53% and on average 34% improvements over state-of-the-art LLM serving systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 46 of its 57 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

46 of 57 references.

Closest ancestors · 19

Same direction, newest first

  1. 2026OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration12 Feb
  2. 2026BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization11 Feb
  3. 2025Parallax: Efficient LLM Inference Service over Decentralized Environment30 Sep
  4. 2025LMDeploy Accelerates Mixed-Precision LLM Inference with TurboMind21 Aug
  5. 2025Cascadia: An Efficient Cascade Serving System for Large Language Models4 Jun
  6. 2025ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments13 Feb
  7. 2025Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs2 Feb
  8. 2024KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving24 Dec
  9. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving24 Jun
  10. 2024Llumnix: Dynamic Scheduling for Large Language Model Serving5 Jun
  11. 2024LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism15 Apr
  12. 2024Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction12 Apr
7 more
  1. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  2. 2024LLM Inference Unveiled: Survey and Roofline Model Insights26 Feb
  3. 2024BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems31 Jan
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan
  5. 2023Fairness in Serving Large Language Models31 Dec
  6. 2023SpotServe: Serving Generative Large Language Models on Preemptible Instances27 Nov
  7. 2023Fast Distributed Inference Serving for Large Language Models10 May

Other references · 27

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  6. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  7. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  8. 2015Illuminating search spaces by mapping elitescited by 425
  9. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
  10. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 245
  11. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 80
  12. 2020Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Providercited by 66
15 more
  1. 2026EvoX: Meta-Evolution for Automated Discoverycited by 60
  2. 2025Barbarians at the Gate: How AI is Upending Systems Researchcited by 54
  3. 2022Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelismcited by 53
  4. 2025CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimizationcited by 44
  5. 2024SkyServe: Serving AI Models across Regions and Clouds with Spot Instancescited by 20
  6. 2025HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environmentcited by 20
  7. 2023Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collectionscited by 16
  8. 2023Improving Automatic Parallel Training via Balanced Memory Workload Optimizationcited by 14
  9. 2022OSDP: Optimal Sharded Data Parallel for Distributed Deep Learningcited by 12
  10. 2024HexiScale: Facilitating Large Language Model Training over Heterogeneous Hardwarecited by 7
  11. 2026Efficient Multi-round LLM Inference over Disaggregated Servingcited by 7
  12. 2025Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accuratelycited by 6
  13. 2025Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUscited by 6
  14. 2025HEXGEN-FLOW: Optimizing LLM Inference Request Scheduling for Agentic Text-to-SQLcited by 3
  15. 2025Evolution of Kernels: Automated RISC-V Kernel Optimization with Large Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Schedulingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Position: Agentic Evolution is the Path to Evolving LLMs
  2. 2026From Inference Engine to Inference Control Plane: Connecting vLLM, llm-d, and the Evolution of Efficient Distributed LLM Serving
  3. 2026When Is Enough Enough in Self-Evolving LLM Systems?
  4. 2025Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
  5. 2026LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs
  6. 2026FineServe: A Fine-Grained Dataset and Characterization of Global LLM Serving Workloads
  7. 2026EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems
  8. 2026A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing