mlsanityData to 7 Oct 2026

Deep Neural Network Approximation of Invariant Functions through Dynamical Systems

arXiv 2208.08707 ↗First posted 18 Aug 2022cs.LG math.DS math.OCcited by 8

Not on the map. Why?

Abstract (arXiv)

We study the approximation of functions which are invariant with respect to certain permutations of the input indices using flow maps of dynamical systems. Such invariant functions includes the much studied translation-invariant ones involving image tasks, but also encompasses many permutation-invariant functions that finds emerging applications in science and engineering. We prove sufficient conditions for universal approximation of these functions by a controlled equivariant dynamical system, which can be viewed as a general abstraction of deep residual networks with symmetry constraints. These results not only imply the universal approximation for a variety of commonly employed neural network architectures for symmetric function approximation, but also guide the design of architectures with approximation guarantees for applications involving new symmetry requirements.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 18 Aug 2022.

Builds on

35 of 56 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  4. 2017Deep Setscited by 1,394
  5. 2016Group Equivariant Convolutional Networkscited by 1,154
  6. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  7. 2017Stable Architectures for Deep Neural Networkscited by 340
  8. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  9. 2015Order Matters: Sequence to sequence for setscited by 236
  10. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  11. 2018Universality of Deep Convolutional Neural Networkscited by 176
  12. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
23 more
  1. 2020Deep Network Approximation for Smooth Functionscited by 163
  2. 2019On the Universality of Invariant Networkscited by 151
  3. 2017Equivariance Through Parameter-Sharingcited by 139
  4. 2018Universal approximations of invariant maps by neural networkscited by 127
  5. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  6. 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 120
  7. 2019Machine learning for molecular simulationcited by 109
  8. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  9. 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 90
  10. 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
  11. 2017Maximum Principle Based Algorithms for Deep Learningcited by 77
  12. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  13. 2021Universal Approximation of Functions on Setscited by 62
  14. 2020Universal Equivariant Multilayer Perceptronscited by 49
  15. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
  16. 2019Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networkscited by 40
  17. 2017Deep Learning for Computational Chemistrycited by 38
  18. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  19. 2019Universal approximations of permutation invariant/equivariant functions by deep neural networkscited by 29
  20. 2020On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysiscited by 26
  21. 2020A Functional Perspective on Learning Symmetric Functions with Neural Networkscited by 16
  22. 2020Approximation in shift-invariant spaces with deep ReLU neural networkscited by 12
  23. 2021DeepSets and their derivative networks for solving symmetric PDEscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 0; 2025: 2; 2026: 1.

8 papers.

  1. 2023A Brief Survey on the Approximation Theory for Sequence Modellingcited by 15
  2. 2022On the Universal Approximation Property of Deep Fully Convolutional Neural Networkscited by 6
  3. 2023On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Networkcited by 6
  4. 2023Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectivescited by 4
  5. 2025A unified framework for establishing the universal approximation of transformer-type architecturescited by 2
  6. 2022On the Approximation and Complexity of Deep Neural Networks to Invariant Functionscited by 0
  7. 2025On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functionscited by 0
  8. 2026Constructive interpolation and generalization rates for neural ODEs: a control perspectivecited by 0