mlsanityData to 14 Oct 2026

SparseEngine: Sparse-First Inference Engine

arXiv 2609.39068 ↗First posted 30 Sep 2026cs.LGcited by 0

On the map: LLM agents and tool use → Agent memory systems

also in 1 more
Abstract (arXiv)

Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these costs, heterogeneous cache representations and workflows hinder integration with existing inference engines, while prior sparse-serving abstractions support only specific layouts or workflows. We present SparseEngine, a ground-up, sparse-first inference engine whose shared lifecycle contract lets each method control its KV representation and computation while coordinating state transitions with common serving infrastructure. SparseEngine supports 15 methods across four categories and enables cross-request state management through Chain Cache, which resumes KV-eviction methods from retained history, and controllable Prefix-Cache Pruning, which removes KV from selected history regions while preserving logical-prefix matching. While maintaining method quality, SparseEngine delivers over 10x higher throughput with KV eviction, over 2.5x faster decoding at matched concurrency than vLLM, and over 2x end-to-end speedup on agent benchmarks. The code is available at https://github.com/CURRENTF/SparseEngine.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 40 of its 50 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

40 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory7 Feb

Other references · 39

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  8. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  9. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  10. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  11. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  12. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
27 more
  1. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  2. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  3. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  4. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  5. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  6. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  7. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  8. 2026Gemma 4 Technical Reportcited by 259
  9. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  10. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 125
  11. 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agentscited by 64
  12. 2026The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligencecited by 61
  13. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflowscited by 56
  14. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 54
  15. 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategycited by 51
  16. 2026IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reusecited by 31
  17. 2025The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMscited by 29
  18. 2025RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inferencecited by 13
  19. 2026Efficient LLM Serving for Agentic Workflows: A Data Systems Perspectivecited by 11
  20. 2025SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Modelscited by 5
  21. 2026Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Servingcited by 4
  22. 2026MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inferencecited by 3
  23. 2026HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Managementcited by 3
  24. 2026Tangram: Unlocking Non-Uniform KV Cache Compression for Efficient Multi-turn LLM Servingcited by 2
  25. 2026FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Servingcited by 2
  26. 2026Vortex: Efficient and Programmable Sparse Attention Serving for AI Agentscited by 1
  27. 2026vToken: Token-Level Virtualization for Reclaimable KV Cachescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
  2. 2026StepKV: Step-Aware KV Cache Compression for LLM Agents
  3. 2026ActKV: Efficient LLM Agents through Action-Guided KV Cache Management
  4. 2026RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse
  5. 2026Practical Online KV Cache Compaction for LLM Agents: An Empirical Study
  6. 2026Akashic: A Low-Overhead LLM Inference Service with MemAttention
  7. 2025KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
  8. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agents