mlsanityData to 14 Oct 2026

AdapShot: Adaptive Many-Shot In-Context Learning with Semantic-Aware KV Cache Reuse

arXiv 2605.03644 ↗First posted 5 May 2026cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Many-Shot In-Context Learning (ICL) has emerged as a promising paradigm, leveraging extensive examples to unlock the reasoning potential of Large Language Models (LLMs). However, existing methods typically rely on a predetermined, fixed number of shots. This static approach often fails to adapt to the varying difficulty of different queries, leading to either insufficient context or interference from noise. Furthermore, the prohibitive computational and memory costs of long contexts severely limit Many-Shot's feasibility. To address the above limitations, we propose AdapShot, which dynamically optimizes shot counts and leverages KV cache reuse for efficient inference. Specifically, we design a probe-based evaluation mechanism that utilizes output entropy to determine the optimal number of shots. To bypass the redundant prefilling computation during both the probing and inference phases, we incorporate a semantics-aware KV cache reuse strategy. Within this reuse strategy, to address positional encoding incompatibilities, we introduce a decoupling and re-encoding method that enables the flexible reordering of cached key-value pairs. Extensive experiments demonstrate that AdapShot achieves an average performance gain of around 10% and a 4.64x speedup compared to state-of-the-art DBSA.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 23 of its 29 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

23 of 29 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  5. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  6. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  7. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 128
  8. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 108
  9. 2025Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognitioncited by 62
  10. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 60
  11. 2023In-Context Learning with Many Demonstration Examplescited by 39
  12. 2024Star Attention: Efficient LLM Inference over Long Sequencescited by 25
11 more
  1. 2024Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuningcited by 21
  2. 2025From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generationcited by 21
  3. 2026Learning Global Hypothesis Space for Enhancing Synergistic Reasoning Chaincited by 17
  4. 2024When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Trainingcited by 15
  5. 2025MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learningcited by 13
  6. 2025Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attentioncited by 12
  7. 2024On Many-Shot In-Context Learning for Long-Context Evaluationcited by 10
  8. 2026Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Modelscited by 10
  9. 2025Scaling Laws for Many-Shot In-Context Learning with Self-Generated Annotationscited by 7
  10. 2024Many-Shot In-Context Learning for Molecular Inverse Designcited by 6
  11. 2025Towards Compute-Optimal Many-Shot In-Context Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Many-Shot In-Context Learning
  2. 2026C2KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
  3. 2024Training-Free Exponential Context Extension via Cascading KV Cache
  4. 2026ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
  5. 2026ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compaction
  6. 2025Compressing Many-Shots in In-Context Learning
  7. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
  8. 2025More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectives