mlsanityData to 7 Oct 2026

You Need an Encoder for Native Position-Independent Caching

arXiv 2602.01519 ↗First posted 2 Feb 2026cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

The Key-Value (KV) cache of Large Language Models (LLMs) is prefix-based, making it highly inefficient for processing contexts retrieved in arbitrary order. Position-Independent Caching (PIC) has been proposed to enable KV reuse without positional constraints; however, existing approaches often incur substantial accuracy degradation, limiting their practical adoption. To address this issue, we propose native PIC by reintroducing the encoder to prevalent decoder-only LLMs and explicitly training it to support PIC. We further develop COMB, a PIC-aware caching system that integrates seamlessly with existing inference frameworks. Experimental results show that COMB reduces Time-to-First-Token (TTFT) by 51-94% and increases throughput by 3× with comparable accuracy. Furthermore, the quality improvement when using DeepSeek-V2-Lite-Chat demonstrates the applicability of COMB to other types of decoder-only LLMs. Our code is available at https://github.com/shijuzhao/Comb.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

33 of 44 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  10. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  11. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  12. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
21 more
  1. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  2. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  3. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  4. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  5. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 391
  6. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  7. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
  8. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  9. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  10. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  11. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Textcited by 60
  12. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reusecited by 60
  13. 2023Stateful Large Language Model Serving with Pensievecited by 59
  14. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 45
  15. 2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generationcited by 36
  16. 2024Block-Attention for Efficient Prefillingcited by 34
  17. 2023NTK-SAP: Improving neural network pruning by aligning training dynamicscited by 30
  18. 2024Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Modelscited by 26
  19. 2025RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoningcited by 19
  20. 2025MPIC: Position-Independent Multimodal Context Caching System for Efficient MLLM Servingcited by 8
  21. 2025Directional Diffusion-Style Code Editing Pre-trainingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026HYPIC: Accelerating Hybrid-Attention LLM Serving with Position-Independent Cachingcited by 7
  2. 2026LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMscited by 3
  3. 2026SemPIC: Learning Semantic Position-Independent KV Cachescited by 2
  4. 2026Evaluating the accuracy of KV cache reuse techniquescited by 0
  5. 2026Memory as a cache: Exact context reuse and deletion by constructioncited by 0
  6. 2026ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptationcited by 0