mlsanityData to 14 Oct 2026

Online normalizer calculation for softmax

arXiv 1805.02867 ↗First posted 8 May 2018cs.PF cs.AI cs.CLcited by 162

Not on the map. Why?

Cited by 22 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The Softmax function is ubiquitous in machine learning, multiple previous works suggested faster alternatives for it. In this paper we propose a way to compute classical Softmax with fewer memory accesses and hypothesize that this reduction in memory accesses should improve Softmax performance on actual hardware. The benchmarks confirm this hypothesis: Softmax accelerates by up to 1.3x and Softmax+TopK combined and fused by up to 5x.

Key facts. Cited by 162 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 11; 2024: 30; 2025: 48; 2026: 70.

162 papers.

Sort:

By area:

  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  2. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  3. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  4. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
  5. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 198
  6. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 160
  7. 2026Kimi K3: Open Frontier Intelligencecited by 138
  8. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 137
  9. 2025Test-Time Training Done Rightcited by 130
  10. 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 120
  11. 2024SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantizationcited by 114
  12. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 102
38 more
  1. 2024Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architecturescited by 98
  2. 2024ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partitioncited by 82
  3. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 80
  4. 2023Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Surveycited by 72
  5. 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generationcited by 69
  6. 2025SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Trainingcited by 66
  7. 2026Attention Residualscited by 65
  8. 2025SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attentioncited by 58
  9. 2024Cut Your Losses in Large-Vocabulary Language Modelscited by 40
  10. 2025PaTH Attention: Position Encoding via Accumulating Householder Transformationscited by 34
  11. 2023FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Corescited by 33
  12. 2024BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batchingcited by 32
  13. 2023DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Trainingcited by 29
  14. 2024Context Parallelism for Scalable Million-Token Inferencecited by 26
  15. 2024Star Attention: Efficient LLM Inference over Long Sequencescited by 25
  16. 2023Efficient Attention via Control Variatescited by 23
  17. 2025SageAttention2++: A More Efficient Implementation of SageAttention2cited by 22
  18. 2024FlashMask: Efficient and Rich Mask Extension of FlashAttentioncited by 20
  19. 2025Terminal Velocity Matchingcited by 20
  20. 2021A Full-Stack Search Technique for Domain Optimized Deep Learning Acceleratorscited by 19
  21. 2025Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Lightcited by 18
  22. 2024Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformerscited by 16
  23. 2025Adaptive Self-improvement LLM Agentic System for ML Library Developmentcited by 16
  24. 2023A Hardware Evaluation Framework for Large Language Model Inferencecited by 14
  25. 2024Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Levelcited by 14
  26. 2024RelayAttention for Efficient Large Language Model Serving with Long System Promptscited by 13
  27. 2025Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernelscited by 13
  28. 2023Blockwise Parallel Transformer for Large Context Modelscited by 12
  29. 2025AdaSplash: Adaptive Sparse Flash Attentioncited by 12
  30. 2025FlashBias: Fast Computation of Attention with Biascited by 11
  31. 2025Scaling LLM Test-Time Compute with Mobile NPU on Smartphonescited by 11
  32. 2025LiteAttention: A Temporal Sparse Attention for Diffusion Transformerscited by 11
  33. 2026FlashSinkhorn: IO-Aware Entropic Optimal Transport on GPUcited by 11
  34. 2024BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequencescited by 10
  35. 2024INT-FlashAttention: Enabling Flash Attention for INT8 Quantizationcited by 9
  36. 2024Optimizing the Deployment of Tiny Transformers on Low-Power MCUscited by 8
  37. 2025Fast weight programming and linear transformers: from machine learning to neurobiologycited by 8
  38. 2025Pushing up to the Limit of Memory Bandwidth and Capacity Utilization for Efficient LLM Decoding on Embedded FPGAcited by 7

50 most cited of 162 All 162 (JSON)