mlsanityData to 7 Oct 2026

Softmax is not Enough (for Sharp Size Generalisation)

arXiv 2410.01104 ↗First posted 1 Oct 2024cs.LG cs.AI cs.ITcited by 42

Not on the map. Why?

Abstract (arXiv)

A key property of reasoning systems is the ability to make sharp decisions on their input data. For contemporary AI systems, a key carrier of sharp behaviour is the softmax function, with its capability to perform differentiable query-key lookups. It is a common belief that the predictive power of networks leveraging softmax arises from "circuits" which sharply perform certain kinds of computations consistently across many diverse inputs. However, for these circuits to be robust, they would need to generalise well to arbitrary valid inputs. In this paper, we dispel this myth: even for tasks as simple as finding the maximum key, any learned circuitry must disperse as the number of items grows at test time. We attribute this to a fundamental limitation of the softmax function to robustly approximate sharp functions with increasing problem size, prove this phenomenon theoretically, and propose adaptive temperature as an ad-hoc technique for improving the sharpness of softmax at inference time.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 40 of its 62 references that are in the map. First posted on arXiv 1 Oct 2024.

Builds on

40 of 62 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2017Graph Attention Networkscited by 8,854
  6. 2016Layer Normalizationcited by 5,579
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  10. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  11. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  12. 2022In-context Learning and Induction Headscited by 930
28 more
  1. 2019Attention is not Explanationcited by 916
  2. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  3. 2023Vision Transformers Need Registerscited by 844
  4. 2021How Attentive are Graph Attention Networks?cited by 818
  5. 2014Neural Turing Machinescited by 630
  6. 2014Recurrent Models of Visual Attentioncited by 344
  7. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 337
  8. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
  9. 2022Exploring Length Generalization in Large Language Modelscited by 204
  10. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  11. 2019Sparse Sequence-to-Sequence Modelscited by 167
  12. 2019What Can Neural Networks Reason About?cited by 155
  13. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  14. 2018DeepInf: Social Influence Prediction with Deep Learningcited by 128
  15. 2020From Local Structures to Size Generalization in Graph Neural Networkscited by 119
  16. 2022Overcoming a Theoretical Limitation of Self-Attentioncited by 110
  17. 2024Transformers need glasses! Information over-squashing in language taskscited by 87
  18. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  19. 2024Theory, Analysis, and Best Practices for Sigmoid Self-Attentioncited by 62
  20. 2024MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereocited by 49
  21. 2024Improving fine-grained understanding in image-text pre-trainingcited by 44
  22. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
  23. 2020Neural Execution Engines: Learning to Execute Subroutinescited by 36
  24. 2024The CLRS-Text Algorithmic Reasoning Language Benchmarkcited by 28
  25. 2022Learnable Commutative Monoids for Graph Neural Networkscited by 13
  26. 2023Asynchronous Algorithmic Alignment with Cocyclescited by 13
  27. 2024Your Context Is Not an Array: Unveiling Random Access Limitations in Transformerscited by 11
  28. 2024Penzai + Treescope: A Toolkit for Interpreting, Visualizing, and Editing Models As Datacited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 11; 2026: 27.

42 papers.

Sort:

By area:

  1. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 111
  2. 2026A Mechanistic Analysis of Looped Reasoning Language Modelscited by 27
  3. 2026The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinkscited by 23
  4. 2025Long-Context Generalization with Sparse Attentioncited by 18
  5. 2025Interpreting the Repeated Token Phenomenon in Large Language Modelscited by 15
  6. 2025Long Context In-Context Compression by Getting to the Gist of Gistingcited by 12
  7. 2024Understanding Synthetic Context Extension via Retrieval Headscited by 10
  8. 2026Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Taskscited by 10
  9. 2025What Happens Next? Anticipating Future Motion by Generating Point Trajectoriescited by 9
  10. 2025On Vanishing Variance in Transformer Length Generalizationcited by 6
  11. 2026WildCat: Near-Linear Attention in Theory and Practicecited by 6
  12. 2026Attention Smoothing Is All You Need For Unlearningcited by 6
30 more
  1. 2026TabDPT-Turbo: Efficient In-Context Learning for Tabular Predictioncited by 4
  2. 2024AERO: Entropy-Guided Framework for Private LLM Inferencecited by 3
  3. 2024RAM: Replace Attention with MLP for Efficient Multivariate Time Series Forecastingcited by 3
  4. 2025Bayesian Principles Improve Prompt Learning In Vision-Language Modelscited by 3
  5. 2026The Dual-Stream Transformer: Channelized Architecture for Interpretable Language Modelingcited by 3
  6. 2026AdaSplash-2: Faster Differentiable Sparse Attentioncited by 3
  7. 2026DashAttention: Differentiable and Adaptive Sparse Hierarchical Attentioncited by 3
  8. 2026Sparse Attention as Compact Kernel Regressioncited by 2
  9. 2026Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scalecited by 2
  10. 2025SEMA: a Scalable and Efficient Mamba like Attention via Token Localization and Averagingcited by 1
  11. 2025Optimal Attention Temperature Improves the Robustness of In-Context Learning under Distribution Shift in High Dimensionscited by 1
  12. 2026The Library Theorem: How External Organization Governs Agentic Reasoning Capacitycited by 1
  13. 2026SepSeq: A Training-Free Framework for Long Numerical Sequence Processing in LLMscited by 1
  14. 2026Scaling Limits of Long-Context Transformerscited by 1
  15. 2026Remember to Forget: Gated Adaptive Positional Encodingcited by 1
  16. 2026ATMA: Long-Context Language Modeling via Polar Attention and Gated-Delta Compression Memorycited by 1
  17. 2025Softplus Attention with Re-weighting Boosts Length Extrapolation in Large Language Modelscited by 0
  18. 2025Converting Transformers into DGNNs Formcited by 0
  19. 2025Knowledge Distillation for Speech Denoising by Latent Representation Alignment with Cosine Distancecited by 0
  20. 2026MAST: Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfercited by 0
  21. 2026Quantamination: Dynamic Quantization Leaks Your Data Across the Batchcited by 0
  22. 2026Breaking the Chains of Probability: Neutrosophic Logic as a New Framework for Epistemic Uncertainty in Large Language Modelscited by 0
  23. 2026Express Language Modelingcited by 0
  24. 2026The Devil is in the Spectrum: Mitigating Representation Collapse in LLMs via Topologically Regularized Side-Pathcited by 0
  25. 2026Beyond Sparse Weights: When Is Attention Compressible?cited by 0
  26. 2026When Do Frozen VLMs Respond to Image-Free Object-Token Edits? An Answer-Key-Free Protocol and What It Revealscited by 0
  27. 2026SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterizationcited by 0
  28. 2026List Counting Failures Are Not One Phenomenoncited by 0
  29. 2026Why Do Conventional World Models Fail to Learn Cellular Automata?cited by 0
  30. 2026SharpDraft: Accelerating Long-Context Speculative Decoding with Cardinality-Aware Query Scalingcited by 0