mlsanityData to 14 Oct 2026

A Quantum Roadmap for Softmax Attention: Exact Born-Rule Analogs for Softmax Attention on the Probability Simplex

arXiv 2608.11173 ↗First posted 11 Aug 2026quant-ph cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of π/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is ε-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 54 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

15 of 54 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2020Rethinking Attention with Performerscited by 1,601
  5. 2016Language Modeling with Gated Convolutional Networkscited by 702
  6. 2018Barren plateaus in quantum neural network training landscapescited by 630
  7. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 341
  8. 2020The effect of data encoding on the expressive power of variational quantum machine learning modelscited by 333
  9. 2023Flow Matching on General Geometriescited by 265
  10. 2022Continuous diffusion for categorical datacited by 214
  11. 2019Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernelcited by 213
  12. 2019Sparse Sequence-to-Sequence Modelscited by 170
3 more
  1. 2019Adaptively Sparse Transformerscited by 155
  2. 2019The Lean mathematical librarycited by 134
  3. 2022Quantum Vision Transformerscited by 48

Built on by

Not cited yet.