# Stability of Measure-to-Measure Transformers on Sub-Gaussian Data

arXiv: https://arxiv.org/abs/2610.07717 · HTML: https://mlsanity.com/paper/2610.07717/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 66 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Builds on (most cited first, 22 of 22)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [A Mathematical Theory of Attention](https://mlsanity.com/paper/2007.02876/) (2020)
- [Quantitative Clustering in Mean-Field Transformer Models](https://mlsanity.com/paper/2504.14697/) (2025)
- [On the Regularity of Attention](https://mlsanity.com/paper/2102.05628/) (2021)
- [In-Context Operator Learning on the Space of Probability Measures](https://mlsanity.com/paper/2601.09979/) (2026)
- [Towards a Physics Foundation Model](https://mlsanity.com/paper/2509.13805/) (2025)
- [Transformers through the lens of support-preserving maps between measures](https://mlsanity.com/paper/2509.25611/) (2025)
- [Learning Probabilistic Filters with Strictly Proper Scoring Rules](https://mlsanity.com/paper/2606.26497/) (2026)
- [Stability of Transformers under Layer Normalization](https://mlsanity.com/paper/2510.09904/) (2025)
- [Continuous transformations of probability measures and their transport representations](https://mlsanity.com/paper/2604.16653/) (2026)
- [One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operators](https://mlsanity.com/paper/2605.06873/) (2026)
- [Learning where to learn: Training data distribution optimization for scientific machine learning](https://mlsanity.com/paper/2505.21626/) (2025)
- [Scaling Limits of Long-Context Transformers](https://mlsanity.com/paper/2605.08505/) (2026)
- [Propagation of Chaos in Contextual Flow Maps](https://mlsanity.com/paper/2605.16747/) (2026)
- [Any-Dimensional Invariant Universality](https://mlsanity.com/paper/2605.23156/) (2026)
- [On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers](https://mlsanity.com/paper/2607.18584/) (2026)
- [Distribution-Conditioned Transport](https://mlsanity.com/paper/2603.04736/) (2026)
- [Global Attention with Linear Complexity for Exascale Generative Data Assimilation in Earth System Prediction](https://mlsanity.com/paper/2604.16590/) (2026)
- [Gradient Flow Structure and Quantitative Dynamics of Multi-Head Self-Attention](https://mlsanity.com/paper/2605.04279/) (2026)
- [Universal Approximation of Measure-to-Measure Operators by Pushforwards](https://mlsanity.com/paper/2609.35483/) (2026)
- [Attention Kernels for Learning Maps Between Heavy-Tailed Measures](https://mlsanity.com/paper/2610.00564/) (2026)

## Abstract (arXiv)

Transformers have exhibited impressive empirical success across various domains, but their theoretical foundations remain less developed. This work constitutes a mathematical study of the measure-to-measure operators defined by transformers. We show that transformers map sub-Gaussian inputs to sub-Gaussian outputs; this ensures that taking arbitrary-length compositions of the softmax operator is well-defined. We then show that transformers are H"older continuous with respect to the 1-Wasserstein distance on appropriate spaces of sub-Gaussian inputs. This allows us to establish estimates on the error propagation along a transformer between a sub-Gaussian input and its empirical approximation. We also study a mean-field analog of the cross-attention mechanism, which is an operator from a pair of probability measures to a single probability measure. We show that cross-attention exhibits different H"older regularity and sample-complexity in its two input arguments. Last, we apply our results to deduce approximation guarantees for measure-to-measure transformers. Together, these results provide a firm stability and finite-sample theory for transformers on sub-Gaussian data.