On the Mathematical Impossibility of Safe Universal Approximators
Not on the map. Why?
Abstract (arXiv)
We establish fundamental mathematical limits on universal approximation theorem (UAT) system alignment by proving that catastrophic failures are an inescapable feature of any useful computational system. Our central thesis is that for any universal approximator, the expressive power required for useful computation is inextricably linked to a dense set of instabilities that make perfect, reliable control a mathematical impossibility. We prove this through a three-level argument that leaves no escape routes for any class of universal approximator architecture. i) Combinatorial Necessity: For the vast majority of practical universal approximators (e.g., those using ReLU activations), we prove that the density of catastrophic failure points is directly proportional to the network's expressive power. ii) Topological Necessity: For any theoretical universal approximator, we use singularity theory to prove that the ability to approximate generic functions requires the ability to implement the dense, catastrophic singularities that characterize them. iii) Empirical Necessity: We prove that the universal existence of adversarial examples is empirical evidence that real-world tasks are themselves catastrophic, forcing any successful model to learn and replicate these instabilities. These results, combined with a quantitative "Impossibility Sandwich" showing that the minimum complexity for usefulness exceeds the maximum complexity for safety, demonstrate that perfect alignment is not an engineering challenge but a mathematical impossibility. This foundational result reframes UAT safety from a problem of "how to achieve perfect control" to one of "how to operate safely in the presence of irreducible uncontrollability," with profound implications for the future of UAT development and governance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 3 Jul 2025.
Builds on
12 of 16 references.
References · 12
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 406
- 2019On the equivalence between graph isomorphism testing and function approximation with GNNscited by 172
- 2023Stabilizing Transformer Training by Preventing Attention Entropy Collapsecited by 161
- 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
- 2018Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approachcited by 91
- 2025Scalable-Softmax Is Superior for Attentioncited by 45
- 2019Stabilize Deep ResNet with A Sharp Scaling Factor τcited by 10
- 2024On the Nonlinearity of Layer Normalizationcited by 9
- 2018Bridgeout: stochastic bridge regularization for deep neural networkscited by 4
- 2023On the Training Instability of Shuffling SGD with Batch Normalizationcited by 3
- 2021Dropout in Training Neural Networks: Flatness of Solution and Noise Structurecited by 2
Built on by
Not cited yet.