mlsanityData to 14 Oct 2026

GQLA: Group-Query Latent Attention for Hardware-Adaptive Decoding

arXiv 2605.15250 ↗First posted 14 May 2026cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Decoding efficiency depends on KV-cache traffic, computation, and parallelism. Multi-head Latent Attention (MLA) offers absorbed and head-expanded forms, but its head-specific projections limit grouped KV sharing. We introduce Group-Query Latent Attention (GQLA), whose group-shared up-projections preserve a joint latent and enable a shardable expanded cache. The same weights support equivalent MQA-absorb and GQA execution using existing kernels. Roofline analysis relates path preference to GPU compute-to-bandwidth ratio and tensor parallelism (TP): absorption trades traffic for computation and replicates the latent across ranks; GQA shards KV groups. Pretraining with eight query heads per group yields terminal training losses close to MLA's across the tested model scales at matched tokens. TransGQLA converts GQA and MLA checkpoints using calibration data. Without gradient updates, converted DeepSeek-V3.1-Base achieves 77.79% eight-task accuracy versus the source's 77.84%, while WikiText-2 perplexity rises from 3.21 to 6.05. On eight H20-3e GPUs, its GQA path improves output throughput over MQA-absorb by 22.42--34.97% at matched resident batches across 8K--64K inputs. Its smaller per-rank cache permits larger batches, giving 50.07% in a separate capacity-tuned comparison. These results support multiple execution paths as an architectural means of adapting decoding to hardware and parallelism.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

14 of 24 references.

References · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  3. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  5. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  6. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  7. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  8. 2022Efficiently Scaling Transformer Inferencecited by 539
  9. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  10. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  11. 2024AI and Memory Wallcited by 92
  12. 2025Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMscited by 27
2 more
  1. 2026HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attentioncited by 12
  2. 2025Hardware-Efficient Attention for Fast Decodingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Think Before You Grid-Search: Floor-First Triage for LLM Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Virtual Head Attention
  2. 2026Multi-Head Low-Rank Attention
  3. 2025Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding
  4. 2024Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA
  5. 2025GTA: Grouped-head latenT Attention
  6. 2025Hardware-Centric Analysis of DeepSeek's Multi-Head Latent Attention
  7. 2025TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference
  8. 2025EG-MLA: Embedding-Gated Multi-head Latent Attention for Scalable and Efficient LLMs