mlsanityData to 14 Oct 2026

UT-ACA: Uncertainty-Triggered Adaptive Context Allocation for Long-Context Inference

arXiv 2603.18446 ↗First posted 19 Mar 2026cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Long-context inference remains challenging for large language models due to attention dilution and out-of-distribution degradation. Context selection mitigates this limitation by attending to a subset of key-value cache entries, yet most methods allocate a fixed context budget throughout decoding despite highly non-uniform token-level contextual demands. To address this issue, we propose Uncertainty-Triggered Adaptive Context Allocation (UT-ACA), an inference-time framework that dynamically adjusts the context window based on token-wise uncertainty. UT-ACA learns an uncertainty detector that combines semantic embeddings with logit-based confidence while accounting for uncertainty accumulation across decoding steps. When insufficient evidence is indicated, UT-ACA selectively rolls back, expands the context window, and regenerates the token with additional support. Experiments show that UT-ACA substantially reduces average context usage while preserving generation quality in long-context settings. Code and dataset are available at https://github.com/Tommy307/UT-ACA.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 57 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

45 of 57 references.

References · 45

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  3. 2024Qwen2 Technical Reportcited by 2,120
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  6. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  7. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  9. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  10. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  11. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  12. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
33 more
  1. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 348
  2. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  3. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
  4. 2024Long-context LLMs Struggle with Long In-context Learningcited by 286
  5. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  6. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 202
  7. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  8. 2024How to Train Long-Context Language Models (Effectively)cited by 146
  9. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 137
  10. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 136
  11. 2024Make Your LLM Fully Utilize the Contextcited by 119
  12. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 116
  13. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 115
  14. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 95
  15. 2024What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formationcited by 89
  16. 2024LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inferencecited by 70
  17. 2024Squeezed Attention: Accelerating Long Context Length LLM Inferencecited by 57
  18. 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inferencecited by 53
  19. 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
  20. 2024TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selectioncited by 39
  21. 2025InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptationcited by 32
  22. 2023Are Large Language Models Temporally Grounded?cited by 27
  23. 2024LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Modelscited by 23
  24. 2024SCOPE: Optimizing Key-Value Cache Compression in Long-context Generationcited by 20
  25. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inferencecited by 20
  26. 2025FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Accelerationcited by 14
  27. 2024E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoningcited by 13
  28. 2025LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMscited by 11
  29. 2024FocusLLM: Precise Understanding of Long Context by Dynamic Condensingcited by 10
  30. 2025When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMscited by 9
  31. 2025FreqKV: Key-Value Compression in Frequency Domain for Context Window Extensioncited by 7
  32. 2025Speculative Decoding for Multi-Sample Inferencecited by 5
  33. 2025AcademicEval: Live Long-Context LLM Benchmarkcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026PAGE: Partition-Aware Gated KV-Cache Evictioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Training-free Context-adaptive Attention for Efficient Long Context Modeling
  2. 2026ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
  3. 2026CHESS: Context-aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inference
  4. 2025Retrieval-augmented Decoding for Improving Truthfulness in Open-ended Generation
  5. 2026CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration
  6. 2026End-to-End Context Compression at Scale
  7. 2022Confident Adaptive Language Modeling
  8. 2025From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models