# On the Mathematical Impossibility of Safe Universal Approximators

arXiv: https://arxiv.org/abs/2507.03031 · HTML: https://mlsanity.com/paper/2507.03031/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 16 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 12 of 12)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Are Transformers universal approximators of sequence-to-sequence functions?](https://mlsanity.com/paper/1912.10077/) (2019)
- [On the equivalence between graph isomorphism testing and function approximation with GNNs](https://mlsanity.com/paper/1905.12560/) (2019)
- [Stabilizing Transformer Training by Preventing Attention Entropy Collapse](https://mlsanity.com/paper/2303.06296/) (2023)
- [Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing](https://mlsanity.com/paper/2306.12929/) (2023)
- [Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach](https://mlsanity.com/paper/1806.01316/) (2018)
- [Scalable-Softmax Is Superior for Attention](https://mlsanity.com/paper/2501.19399/) (2025)
- [Stabilize Deep ResNet with A Sharp Scaling Factor τ](https://mlsanity.com/paper/1903.07120/) (2019)
- [On the Nonlinearity of Layer Normalization](https://mlsanity.com/paper/2406.01255/) (2024)
- [Bridgeout: stochastic bridge regularization for deep neural networks](https://mlsanity.com/paper/1804.08042/) (2018)
- [On the Training Instability of Shuffling SGD with Batch Normalization](https://mlsanity.com/paper/2302.12444/) (2023)
- [Dropout in Training Neural Networks: Flatness of Solution and Noise Structure](https://mlsanity.com/paper/2111.01022/) (2021)

## Abstract (arXiv)

We establish fundamental mathematical limits on universal approximation theorem (UAT) system alignment by proving that catastrophic failures are an inescapable feature of any useful computational system. Our central thesis is that for any universal approximator, the expressive power required for useful computation is inextricably linked to a dense set of instabilities that make perfect, reliable control a mathematical impossibility. We prove this through a three-level argument that leaves no escape routes for any class of universal approximator architecture. i) Combinatorial Necessity: For the vast majority of practical universal approximators (e.g., those using ReLU activations), we prove that the density of catastrophic failure points is directly proportional to the network's expressive power. ii) Topological Necessity: For any theoretical universal approximator, we use singularity theory to prove that the ability to approximate generic functions requires the ability to implement the dense, catastrophic singularities that characterize them. iii) Empirical Necessity: We prove that the universal existence of adversarial examples is empirical evidence that real-world tasks are themselves catastrophic, forcing any successful model to learn and replicate these instabilities. These results, combined with a quantitative "Impossibility Sandwich" showing that the minimum complexity for usefulness exceeds the maximum complexity for safety, demonstrate that perfect alignment is not an engineering challenge but a mathematical impossibility. This foundational result reframes UAT safety from a problem of "how to achieve perfect control" to one of "how to operate safely in the presence of irreducible uncontrollability," with profound implications for the future of UAT development and governance.