# Interpolation, Approximation and Controllability of Deep Neural Networks

arXiv: https://arxiv.org/abs/2309.06015 · HTML: https://mlsanity.com/paper/2309.06015/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 42 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 3 of 3)

- [A Mean-Field Optimal Control Formulation of Deep Learning](https://mlsanity.com/paper/1807.01083/) (2018)
- [Universal Approximation Power of Deep Residual Neural Networks via Nonlinear Control Theory](https://mlsanity.com/paper/2007.06007/) (2020)
- [Mean-Field Pontryagin Maximum Principle](https://mlsanity.com/paper/1504.02236/) (2015)

## Built on by (most cited first, 16 of 16)

- [A mathematical perspective on Transformers](https://mlsanity.com/paper/2312.10794/) (2023)
- [Measure-to-measure interpolation using Transformers](https://mlsanity.com/paper/2411.04551/) (2024)
- [Global Convergence in Training Large-Scale Transformers](https://mlsanity.com/paper/2410.23610/) (2024)
- [Cluster-based classification with neural ODEs via control](https://mlsanity.com/paper/2312.13807/) (2023)
- [Constructive approximate transport maps with normalizing flows](https://mlsanity.com/paper/2412.19366/) (2024)
- [A Minimal Control Family of Dynamical Systems for Universal Approximation](https://mlsanity.com/paper/2312.12903/) (2023)
- [Demystifying the Token Dynamics of Deep Selective State Space Models](https://mlsanity.com/paper/2410.03292/) (2024)
- [A unified framework for establishing the universal approximation of transformer-type architectures](https://mlsanity.com/paper/2506.23551/) (2025)
- [Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding](https://mlsanity.com/paper/2512.03058/) (2025)
- [Layer-wise Geometric Approximation Rates for Deep Networks](https://mlsanity.com/paper/2604.20219/) (2026)
- [Autonomous-Flow-Based Generation](https://mlsanity.com/paper/2511.09902/) (2025)
- [Theoretical Analysis of Measure Consistency Regularization for Partially Observed Data](https://mlsanity.com/paper/2602.01437/) (2026)
- [Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximations](https://mlsanity.com/paper/2605.22557/) (2026)
- [Constructive interpolation and generalization rates for neural ODEs: a control perspective](https://mlsanity.com/paper/2606.00469/) (2026)
- [Universal interpolation for deep residual self-attention networks](https://mlsanity.com/paper/2610.01981/) (2026)
- [Deep learning from a control vantage point](https://mlsanity.com/paper/2610.14385/) (2026)

## Abstract (arXiv)

We investigate the expressive power of deep residual neural networks idealized as continuous dynamical systems through control theory. Specifically, we consider two properties that arise from supervised learning, namely universal interpolation - the ability to match arbitrary input and target training samples - and the closely related notion of universal approximation - the ability to approximate input-target functional relationships via flow maps. Under the assumption of affine invariance of the control family, we give a characterisation of universal interpolation, showing that it holds for essentially any architecture with non-linearity. Furthermore, we elucidate the relationship between universal interpolation and universal approximation in the context of general control systems, showing that the two properties cannot be deduced from each other. At the same time, we identify conditions on the control family and the target function that ensures the equivalence of the two notions.