mlsanityData to 14 Oct 2026

Counterexamples to Local Reconstruction Gain as a Proxy for Final Fidelity in Residual Completion

arXiv 2609.34063 ↗First posted 28 Sep 2026cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Residual completion augments query-aware sparse attention by estimating the contribution of tokens omitted from the exact sparse computation. We ask whether improving a layer's attention-output reconstruction on the same incoming Q/K/V and selected support necessarily improves the fidelity of the final model output. We study training-free RESA and learned Top-K+φ with frozen backbone language models. A prespecified single-layer screen yields two Qwen3-0.6B/Multi-LexSum interventions for which direct-runtime measurements show positive prespecified request-aggregate local reconstruction gain but worse final KL fidelity than the corresponding all-abstain Exact Top-K baseline on both discovery and prompt-token-disjoint holdout requests. Exact restoration at the same layer instead improves final fidelity, showing that the reversal is specific to approximate completion in these cases. In complementary multi-layer experiments, a task-independent local diagnostic often repairs the tested completion estimators, although the repaired models do not consistently outperform Exact Top-K. Together, these results show that better local reconstruction need not translate into better final-model fidelity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 22 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

19 of 22 references.

References · 19

  1. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  2. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  3. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  4. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  5. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
  6. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 245
  7. 2019BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarizationcited by 156
  8. 2023SparQ Attention: Bandwidth-Efficient LLM Inferencecited by 119
  9. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
  10. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inferencecited by 75
  11. 2022Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularitiescited by 47
  12. 2026PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformerscited by 17
7 more
  1. 2026ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothingcited by 9
  2. 2025Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correctioncited by 7
  3. 2026ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compressioncited by 3
  4. 2026Residual-Mass Accounting for Partial-KV Decodingcited by 1
  5. 2026Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluationcited by 1
  6. 2026RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagationcited by 1
  7. 2026KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Modelscited by 1

Built on by

Not cited yet.