mlsanityData to 7 Oct 2026

Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines

arXiv 2405.13682 ↗First posted 22 May 2024cs.LG math.RT stat.MLcited by 2

Not on the map. Why?

Abstract (arXiv)

We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps, called the joint-equivariant machines, based on the group representation theory. ``Constructive'' here indicates that the distribution of parameters is given in a closed-form expression known as the ridgelet transform. Joint-group-equivariance encompasses a broad class of feature maps that generalize classical group-equivariance. Particularly, fully-connected networks are not group-equivariant but are joint-group-equivariant. Our main theorem also unifies the universal approximation theorems for both shallow and deep networks. Until this study, the universality of deep networks has been shown in a different manner from the universality of shallow networks, but our results discuss them on common ground. Now we can understand the approximation schemes of various learning machines in a unified manner. As applications, we show the constructive universal approximation properties of four examples: depth-n joint-equivariant machine, depth-n fully-connected network, depth-n group-convolutional network, and a new depth-2 network with quadratic forms whose universality has not been known.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 50 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

28 of 50 references.

References · 28

  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  3. 2016Error bounds for approximations with deep ReLU networkscited by 567
  4. 2017Stable Architectures for Deep Neural Networkscited by 340
  5. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  6. 2016Benefits of depth in neural networkscited by 248
  7. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
  8. 2019Universal Approximation with Deep Narrow Networkscited by 198
  9. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  10. 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 120
  11. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  12. 2020Minimum Width for Universal Approximationcited by 92
16 more
  1. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  2. 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 78
  3. 2015Neural Network with Unbounded Activation Functions is Universal Approximatorcited by 63
  4. 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 50
  5. 2018An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networkscited by 25
  6. 2022Achieve the Minimum Width of Neural Networks for Universal Approximationcited by 25
  7. 2023Minimum width for universal approximation using ReLU networks on compact domaincited by 17
  8. 2022Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysiscited by 14
  9. 2023Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximationcited by 13
  10. 2022Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groupscited by 11
  11. 2021Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Spacecited by 7
  12. 2020Phase Transitions in Rate Distortion Theory and Deep Learningcited by 5
  13. 2023Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computationcited by 5
  14. 2023Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networkscited by 5
  15. 2022Limitations on approximation by deep and shallow neural networkscited by 4
  16. 2024A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networkscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Symmetry in Neural Network Parameter Spacescited by 31
  2. 2025Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSscited by 1