mlsanityData to 7 Oct 2026

Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression

arXiv 2608.01631 ↗First posted 3 Aug 2026cs.CLcited by 2

On the map: Efficient LLM inference and serving → Efficient reasoning and agent inference

Abstract (arXiv)

KV cache compression is commonly evaluated by final-answer accuracy, implicitly assuming that preserving the answer also preserves the reasoning that supports it. We test this assumption for large reasoning models and show that it can fail: under compression, correct answers and the validity of their visible supporting rationales can be preserved at different rates. We study this failure with a controlled fixed-trace replay protocol, which holds reasoning content fixed and isolates whether compression preserves usable information from an already available trace. We evaluate ten token-eviction KV compression methods and one quantization method on three models across mathematical reasoning, scientific QA, clinical calculation, and long-context retrieval. We measure final accuracy, answer-chain consistency, and perturbation faithfulness. Across tasks, token-eviction methods can preserve competitive final-answer accuracy while substantially degrading chain support or perturbation faithfulness. We call this the answer-evidence gap. A coverage-preserving quantization control is substantially less affected, suggesting that the failure is tied less to KV memory reduction itself than to losing access to parts of the reasoning trace. Code is available at https://github.com/famous-blue-raincoat/Safe_KV_Compress.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Efficient reasoning and agent inference, one of 12 directions in Efficient LLM inference and serving. Builds on 40 of its 51 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

40 of 51 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Models30 May

Other references · 39

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  6. 2024A Survey on LLM-as-a-Judgecited by 1,384
  7. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  8. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  10. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  11. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  12. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
27 more
  1. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  2. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  3. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  4. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
  5. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  6. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  7. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 186
  8. 2024Distilling System 2 into System 1cited by 152
  9. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 117
  10. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  11. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approachcited by 95
  12. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 89
  13. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compressioncited by 84
  14. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantizationcited by 74
  15. 2025ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoningcited by 70
  16. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distributioncited by 70
  17. 2024MedCalc-Bench: Evaluating Large Language Models for Medical Calculationscited by 64
  18. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 62
  19. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 57
  20. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 43
  21. 2025Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Modelscited by 21
  22. 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbositycited by 15
  23. 2026ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inferencecited by 8
  24. 2025LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Importantcited by 7
  25. 2026MKA: Memory-Keyed Attention for Efficient Long-Context Reasoningcited by 6
  26. 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokenscited by 3
  27. 2026HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inferencecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Cartridges++: KV Cache Compression without Off-Context Derailmentcited by 1
  2. 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression
  2. 2026KV-Rescue: Recovering Reasoning Language Model KV Eviction Loss via Stepwise Interleaving
  3. 2026CASK: Core-Aware Selective KV Compression for Reasoning Traces
  4. 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning
  5. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
  6. 2026A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models
  7. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression
  8. 2025When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models