mlsanityData to 14 Oct 2026

DynaKV: Enabling Accurate and Efficient Long-Sequence LLM Decoding on Smartphones

arXiv 2511.07427 ↗First posted 20 Oct 2025cs.DC cs.AIcited by 4

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

As the demand for human-like reasoning, multi-turn dialogues, and long-form responses grows, large language models (LLMs) are increasingly expected to support efficient and effective long-sequence decoding. However, due to limited DRAM capacity, long-seuqence LLM decoding on smartphones is constrained by the key-value cache (KVCache), whose memory footprint increases linearly with sequence length. Retrieval-based methods mitigate DRAM pressure by offloading KVCache to flash and retrieving query-relevant entries through cluster-based indexing. Unfortunately, as decoding progresses, KVCache distribution shifts render static or local cluster updates progressively misaligned, excluding essential entries or fetching redundant ones. These issues are further exacerbated by smartphone-specific limitations in bandwidth, IOPS, and memory capacity. We propose DynaKV, the first adaptive KVCache management approach that jointly addresses accuracy and efficiency for long-sequence decoding on smartphones. DynaKV integrates three key techniques: (1) Migration-Free Cluster Adaptation, which adaptively splits clusters during retrieval without incurring additional transfers; (2) Continuity-Centric Flash Management, which co-locates correlated entries and clusters and employs a dual-head layout for efficient updates; and (3) Memory-Efficient Cache Design, which virtualizes cache space across DRAM and flash and extends replacement policies to align with cluster-level access patterns. Evaluations demonstrate that DynaKV improves retrieval accuracy and reduces end-to-end latency compared to state-of-the-art solutions, achieving average gains of 1.38× in accuracy and 1.47× speedups. Furthermore, the insights of DynaKV naturally extend to other long-context workloads and multi-tier memory hierarchies, underscoring its broader applicability.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 43 of its 60 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

43 of 60 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025CentroidKV: Efficient Long-Context LLM Inference via KV Cache Clustering13 Jun
  2. 2024ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression4 Dec
  3. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference28 Oct
  4. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  6. 2023SparQ Attention: Bandwidth-Efficient LLM Inference8 Dec
  7. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  8. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
  9. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 34

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  8. 2018Hierarchical Neural Story Generationcited by 1,193
  9. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  10. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  11. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  12. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
22 more
  1. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  2. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  3. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  4. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 252
  5. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  6. 2023Using an LLM to Help With Code Understandingcited by 228
  7. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  8. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 177
  9. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 116
  10. 2024PowerInfer-2: Fast Large Language Model Inference on a Smartphonecited by 103
  11. 2024LongLaMP: A Benchmark for Personalized Long-form Text Generationcited by 97
  12. 2024SEED-Story: Multimodal Long Story Generation with Large Language Modelcited by 73
  13. 2023LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Codingcited by 64
  14. 2024Fast On-device LLM Inference with NPUscited by 62
  15. 2024Squeezed Attention: Accelerating Long Context Length LLM Inferencecited by 57
  16. 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgecited by 55
  17. 2024HelloBench: Evaluating Long Text Generation Capabilities of Large Language Modelscited by 44
  18. 2025Bitnet.cpp: Efficient Edge Inference for Ternary LLMscited by 22
  19. 2023Parrot: Enhancing Multi-Turn Instruction Following for Large Language Modelscited by 16
  20. 2024Neuralink: Fast LLM Inference on Smartphones with Neuron Co-Activation Linkingcited by 5
  21. 2025Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flashcited by 4
  22. 2025Long Exposure: Accelerating Parameter-Efficient Fine-Tuning for LLMs under Shadowy Sparsitycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Lever: Speculative LLM Inference on Smartphonescited by 2
  2. 2025KVNAND: Efficient On-Device Large Language Model Inference Using DRAM-Free In-Flash Computingcited by 1
  3. 2026S4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Cachingcited by 0
  4. 2026SPLASH: Co-Designing Sparse Attention with High-Bandwidth Flash for Efficient Long-Context Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching
  2. 2026Don't Waste Bits! Adaptive KV-Cache Quantization for Lightweight On-Device LLMs
  3. 2026Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM
  4. 2026ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs
  5. 2026Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
  6. 2026TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference
  7. 2026ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
  8. 2025SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs