mlsanityData to 7 Oct 2026

MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel

arXiv 2607.19456 ↗First posted 21 Jul 2026cs.LG cs.AIcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

We derive four memory-optimal inference artifacts for transformer attention using the Mathematics of Arrays (MoA), each following directly from the forward-pass Denotational Normal Form (DNF) of with the query-row index fixed to the current decode step. The artifacts are: (1)~a single-query decode DNF in which the psi-reduction eliminates the K^top buffer algebraically, achieving (dk + ndk+ ndv+ dv)×4,B Dynamic Random Access Memory (DRAM) traffic result numerically verified to |err|_≤2×10-7; (2)~a C/OpenACC Graphics Processing Unit (GPU) kernel with Operational Normal Form (ONF) stride arithmetic and hardware-coalesced memory access, verified to |err|_∞=0 (exact IEEE-754 floating-point arithmetic); (3)~a multi-step KV-cache with O(dk+dv) per-step append via MoA concatenation #; and (4)~Grouped-Query Attention (GQA) and Multi-Query Attention (MQA) derived via psi-selection, achieving a proven frac hq hkv reduction in KV traffic. All programs are verified against PyTorch scaleddotproductattention.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 7 of its 12 references that are in the map. First posted on arXiv 21 Jul 2026.

Builds on

7 of 12 references.

References · 7

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  4. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  5. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  6. 2023From array algebra to energy efficiency on GPUs: Data and hardware shapes with dimension-lifting to optimize memory-processor layoutscited by 1
  7. 2026Attention at the Theoretical Minimum: A Mathematics of Arrays Framework for Memory-Optimal Transformer Kernelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Validating Memory-Optimal Transformer Kernels on Real Hardware: From Formal Derivation to Measured Performance Across Two HPC Clusterscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025NOSA: Native and Offloadable Sparse Attention
  2. 2025End-to-End Transformer Acceleration Through Processing-in-Memory Architectures
  3. 2025Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decoding
  4. 2026Heterogeneous LLM Serving with General-Purpose Processing-Near-Memory for Retrieval-Based Sparse Attention
  5. 2025HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs
  6. 2026Stochastic Sparse Attention for Memory-Bound Inference
  7. 2026Predictive Multi-Tier Memory Management for KV Cache in Large-Scale GPU Inference
  8. 2026Learning how to Forget: Fine-tuning for Long-Context Sparse Attention