mlsanityData to 14 Oct 2026

Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents

arXiv 2606.06453 ↗First posted 4 Jun 2026cs.AIcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Sparse attention is becoming increasingly important for serving large language models (LLMs) as generation lengths continue to grow. However, deploying and evaluating new sparse attention algorithms at scale remains highly engineering-intensive, slowing both human researchers and AI agents in exploring the sparse attention design. To address this challenge, we present Vortex, a system that combines a Python-embedded frontend language atop a page-centric tensor abstraction for expressing a broad range of sparse attention algorithms, with an efficient backend tightly integrated into modern LLM serving stacks. Vortex enables rapid prototyping, deployment, and evaluation of sparse attention algorithms, effectively translating their theoretical efficiency gains into real-world throughput improvements. As a result, Vortex substantially accelerates the design and iteration of sparse attention algorithms. First, AI agents use Vortex to automatically generate and refine diverse algorithms, the best reaching up to 3.46× higher throughput than full attention while preserving accuracy. Second, Vortex extends sparse attention to emerging architectures and very large models that are otherwise hard to experiment with, reaching up to 4.7× higher throughput on the MLA-based GLM-4.7-Flash and 1.37× on the 229B-parameter MiniMax-M2.7 on NVIDIA B200 GPUs.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 62 of its 81 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

62 of 81 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence26 May
  2. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models8 Aug

Other references · 60

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  9. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  10. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  11. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  12. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
38 more
  1. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  2. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  3. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  4. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  5. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  6. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  7. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  8. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  9. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  10. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  11. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  12. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  13. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  14. 2024The Faiss librarycited by 376
  15. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 204
  16. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 198
  17. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  18. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
  19. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 185
  20. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 134
  21. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inferencecited by 126
  22. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 124
  23. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 121
  24. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
  25. 2014Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)cited by 109
  26. 2024DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inferencecited by 106
  27. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 102
  28. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 95
  29. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 89
  30. 2023SpotServe: Serving Generative Large Language Models on Preemptible Instancescited by 87
  31. 2024Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCachecited by 73
  32. 2023FlashDecoding++: Faster Large Language Model Inference on GPUscited by 71
  33. 2026FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scalingcited by 66
  34. 2023KDEformer: Accelerating Transformers via Kernel Density Estimationcited by 58
  35. 2024Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flowcited by 55
  36. 2025Twilight: Adaptive Attention Sparsity with Hierarchical Top-p Pruningcited by 49
  37. 2022SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learningcited by 47
  38. 2024Post-Training Sparse Attention with Double Sparsitycited by 45

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SparseEngine: Sparse-First Inference Enginecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A.X K2 Technical Report
  2. 2026AB-Sparse: Sparse Attention with Adaptive Block Size for Accurate and Efficient Long-Context Inference
  3. 2025Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing
  4. 2026Steered Generation via Gradient-Based Optimization on Sparse Query Features
  5. 2025Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
  6. 2026TuringLLM: Efficiently Scaling Foundation Models Toward Physical AI
  7. 2026S-HPLB: Efficient LLM Attention Serving via Sparsity-Aware Head Parallelism Load Balance
  8. 2026Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference