mlsanityData to 14 Oct 2026

AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference

arXiv 2504.10326 ↗First posted 14 Apr 2025cs.AI cs.DB cs.IRcited by 3

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

AlayaDB is a cutting-edge vector database system natively architected for efficient and effective long-context inference for Large Language Models (LLMs) at AlayaDB AI. Specifically, it decouples the KV cache and attention computation from the LLM inference systems, and encapsulates them into a novel vector database system. For the Model as a Service providers (MaaS), AlayaDB consumes fewer hardware resources and offers higher generation quality for various workloads with different kinds of Service Level Objectives (SLOs), when comparing with the existing alternative solutions (e.g., KV cache disaggregation, retrieval-based sparse attention). The crux of AlayaDB is that it abstracts the attention computation and cache management for LLM inference into a query processing procedure, and optimizes the performance via a native query optimizer. In this work, we demonstrate the effectiveness of AlayaDB via (i) three use cases from our industry partners, and (ii) extensive experimental results on LLM inference benchmarks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 26 of its 53 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

26 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving24 Jun
  2. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 24

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  9. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  10. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  11. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  12. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
12 more
  1. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 288
  2. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  3. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 203
  4. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  5. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 116
  6. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
  7. 2017Fast Approximate Nearest Neighbor Search With The Navigating Spreading-out Graphcited by 104
  8. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 95
  9. 2024ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Datacited by 32
  10. 2023Model-as-a-Service (MaaS): A Surveycited by 25
  11. 2024RoarGraph: A Projected Bipartite Graph for Efficient Cross-Modal Approximate Nearest Neighbor Searchcited by 12
  12. 2021Greenplum: A Hybrid Database for Transactional and Analytical Workloadscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inferencecited by 13
  2. 2026QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Servingcited by 3
  3. 2026Accelerating Cross-Model LLM Inference via Attention Map Reusecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Inference Unveiled: Survey and Roofline Model Insights
  2. 2025xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference
  3. 2023SparQ Attention: Bandwidth-Efficient LLM Inference
  4. 2025MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
  5. 2023Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline
  6. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methods
  7. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques
  8. 2025Taming the Titans: A Survey of Efficient LLM Inference Serving