# Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines

arXiv: https://arxiv.org/abs/2405.13682 · HTML: https://mlsanity.com/paper/2405.13682/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 50 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 28 of 28)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [Approximating Continuous Functions by ReLU Nets of Minimal Width](https://mlsanity.com/paper/1710.11278/) (2017)
- [On the Expressive Power of Deep Learning: A Tensor Analysis](https://mlsanity.com/paper/1509.05009/) (2015)
- [Minimum Width for Universal Approximation](https://mlsanity.com/paper/2006.08859/) (2020)
- [The phase diagram of approximation rates for deep neural networks](https://mlsanity.com/paper/1906.09477/) (2019)
- [ResNet with one-neuron hidden layers is a Universal Approximator](https://mlsanity.com/paper/1806.10909/) (2018)
- [Neural Network with Unbounded Activation Functions is Universal Approximator](https://mlsanity.com/paper/1505.03654/) (2015)
- [Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces](https://mlsanity.com/paper/2211.14400/) (2022)
- [An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks](https://mlsanity.com/paper/1803.01299/) (2018)
- [Achieve the Minimum Width of Neural Networks for Universal Approximation](https://mlsanity.com/paper/2209.11395/) (2022)
- [Minimum width for universal approximation using ReLU networks on compact domain](https://mlsanity.com/paper/2309.10402/) (2023)
- [Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis](https://mlsanity.com/paper/2203.01631/) (2022)
- [Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation](https://mlsanity.com/paper/2305.18460/) (2023)
- [Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups](https://mlsanity.com/paper/2205.14819/) (2022)
- [Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space](https://mlsanity.com/paper/2106.04770/) (2021)
- [Phase Transitions in Rate Distortion Theory and Deep Learning](https://mlsanity.com/paper/2008.01011/) (2020)
- [Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation](https://mlsanity.com/paper/2301.11936/) (2023)
- [Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks](https://mlsanity.com/paper/2310.03530/) (2023)
- [Limitations on approximation by deep and shallow neural networks](https://mlsanity.com/paper/2212.02223/) (2022)
- [A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks](https://mlsanity.com/paper/2402.15984/) (2024)

## Built on by (most cited first, 2 of 2)

- [Symmetry in Neural Network Parameter Spaces](https://mlsanity.com/paper/2506.13018/) (2025)
- [Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSs](https://mlsanity.com/paper/2509.21895/) (2025)

## Abstract (arXiv)

We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps, called the joint-equivariant machines, based on the group representation theory. ``Constructive'' here indicates that the distribution of parameters is given in a closed-form expression known as the ridgelet transform. Joint-group-equivariance encompasses a broad class of feature maps that generalize classical group-equivariance. Particularly, fully-connected networks are not group-equivariant but are joint-group-equivariant. Our main theorem also unifies the universal approximation theorems for both shallow and deep networks. Until this study, the universality of deep networks has been shown in a different manner from the universality of shallow networks, but our results discuss them on common ground. Now we can understand the approximation schemes of various learning machines in a unified manner. As applications, we show the constructive universal approximation properties of four examples: depth-n joint-equivariant machine, depth-n fully-connected network, depth-n group-convolutional network, and a new depth-2 network with quadratic forms whose universality has not been known.