mlsanityData to 14 Oct 2026

Runtime Observability for Heterogeneous Attention Memory

arXiv 2608.05863 ↗First posted 6 Aug 2026cs.AIcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Modern models no longer keep a plain KV cache: latent caches, learned sparse selectors and recurrent states each carry the model's memory in a different form, and each fails differently under compression. We give a runtime observability contract that covers all four memory classes with three operators, instantiate it on six model configurations across five architecture families, and compose the per-stage bounds into an executable request-level risk ledger. Contracts carry their error metric as a type -- composition is only defined when metrics match, and this check rejected our own first composed chain; the repaired chain crosses metrics through two proved bridges, and whatever no formal system can certify is measured instead, dropping the composed tier to empirical automatically: every claim is certified, partially certified, or empirical, composition inherits the weakest tier, and the tier is decided by the machine. Replayed over 12.4M entry reads and run under eight-way concurrency with per-request budgets and fail-closed identity attribution, the ledger quantifies the honest trade-off on today's witness and holds its risk budget with zero violations. A fused always-on probe observes a declared one-layer subset under CUDA graphs inside the serving noise floor. Applied to a served DeepSeek-V4 stack with a packed compressed-KV prototype, the same machinery localizes a silent corruption to a precise structural boundary -- exact in the eviction-free, identity-isolated regime, with every observed failure in an eviction or slot-reuse regime -- through a machine-adjudicated discrimination campaign whose calculus rejected two of our own confounded inferences along the way. All artifacts, guards, and the Lean development are released at https://github.com/metask-ai/witprobe-attention-memory; every number in this paper regenerates from the shipped artifacts by one command.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 13 of its 22 references that are in the map. First posted on arXiv 6 Aug 2026.

Builds on

13 of 22 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  2. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM8 Mar
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  4. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 9

  1. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  3. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  4. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  5. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  6. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
  7. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  8. 2022Game-theoretic statistics and safe anytime-valid inferencecited by 155
  9. 2026WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Runtime-Certified Bounded-Error Quantized Attention
  2. 2025Investigating Memory in Model-Free RL with POPGym Arcade
  3. 2026AI Observability for Large Language Model Systems: A Multi-Layer Analysis of Monitoring Approaches from Confidence Calibration to Infrastructure Tracing ↗
  4. 2026OGAM: Connecting Systematic Testing to Runtime Assurance through Object-Grounded Attention Monitoring for VLA Policies
  5. 2025Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domains
  6. 2026Auditing Long-Term Memory Evaluation: Repeated Judging, Reader Variation, and Negative Controls
  7. 2026D2ACCI: A Dual-Loop Diagnostic Protocol for Evidence-Preserving Agent Memory
  8. 2026CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cache