# A Minimal Control Family of Dynamical Systems for Universal Approximation

arXiv: https://arxiv.org/abs/2312.12903 · HTML: https://mlsanity.com/paper/2312.12903/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [The Reversible Residual Network: Backpropagation Without Storing Activations](https://mlsanity.com/paper/1707.04585/) (2017)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [ResNet with one-neuron hidden layers is a Universal Approximator](https://mlsanity.com/paper/1806.10909/) (2018)
- [Deep Learning via Dynamical Systems: An Approximation Perspective](https://mlsanity.com/paper/1912.10382/) (2019)
- [Neural ODE control for classification, approximation and transport](https://mlsanity.com/paper/2104.05278/) (2021)
- [Approximation Capabilities of Neural ODEs and Invertible Residual Networks](https://mlsanity.com/paper/1907.12998/) (2019)
- [Depth Separation for Neural Networks](https://mlsanity.com/paper/1702.08489/) (2017)
- [PolyNet: A Pursuit of Structural Diversity in Very Deep Networks](https://mlsanity.com/paper/1611.05725/) (2016)
- [Deep neural networks, generic universal interpolation, and controlled ODEs](https://mlsanity.com/paper/1908.07838/) (2019)
- [Achieve the Minimum Width of Neural Networks for Universal Approximation](https://mlsanity.com/paper/2209.11395/) (2022)
- [Interpolation, Approximation and Controllability of Deep Neural Networks](https://mlsanity.com/paper/2309.06015/) (2023)
- [Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation](https://mlsanity.com/paper/2305.18460/) (2023)
- [Vocabulary for Universal Approximation: A Linguistic Perspective of Mapping Compositions](https://mlsanity.com/paper/2305.12205/) (2023)
- [Rethinking ResNets: Improved Stacking Strategies With High Order Schemes](https://mlsanity.com/paper/2103.15244/) (2021)

## Built on by (most cited first, 3 of 3)

- [Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximations](https://mlsanity.com/paper/2605.22557/) (2026)
- [Universal interpolation for deep residual self-attention networks](https://mlsanity.com/paper/2610.01981/) (2026)
- [Deep learning from a control vantage point](https://mlsanity.com/paper/2610.14385/) (2026)

## Abstract (arXiv)

The universal approximation property (UAP) holds a fundamental position in deep learning, as it provides a theoretical foundation for the expressive power of neural networks. It is widely recognized that a composition of linear and nonlinear functions, such as the rectified linear unit (ReLU) activation function, can approximate continuous functions on compact domains. In this paper, we extend this efficacy to a scenario containing dynamical systems with controls. We prove that the control family F1 containing all affine maps and the nonlinear ReLU map is sufficient for generating flow maps that can approximate orientation-preserving (OP) diffeomorphisms on any compact domain. Since F1 contains only one nonlinear function and the UAP does not hold if we remove the nonlinear function, we call F1 a minimal control family for the UAP. On this basis, several mild sufficient conditions, such as affine invariance, are established for the control family and discussed. Our results reveal an underlying connection between the approximation power of neural networks and control systems and could provide theoretical guidance for examining the approximation power of flow-based models.