# A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks

arXiv: https://arxiv.org/abs/2402.15984 · HTML: https://mlsanity.com/paper/2402.15984/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 70 references that are in the map.
- First posted on arXiv 25 Feb 2024.

## Builds on (most cited first, 30 of 31)

- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [A Mean Field View of the Landscape of Two-Layers Neural Networks](https://mlsanity.com/paper/1804.06561/) (2018)
- [On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport](https://mlsanity.com/paper/1805.09545/) (2018)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Norm-Based Capacity Control in Neural Networks](https://mlsanity.com/paper/1503.00036/) (2015)
- [Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry](https://mlsanity.com/paper/1806.03417/) (2018)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [On the Universality of Invariant Networks](https://mlsanity.com/paper/1901.09342/) (2019)
- [Universal approximations of invariant maps by neural networks](https://mlsanity.com/paper/1804.10306/) (2018)
- [Hyperbolic Attention Networks](https://mlsanity.com/paper/1805.09786/) (2018)
- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [How do infinite width bounded norm networks look in function space?](https://mlsanity.com/paper/1902.05040/) (2019)
- [Neural Network with Unbounded Activation Functions is Universal Approximator](https://mlsanity.com/paper/1505.03654/) (2015)
- [Convex Analysis of the Mean Field Langevin Dynamics](https://mlsanity.com/paper/2201.10469/) (2022)
- [Stochastic Particle Gradient Descent for Infinite Ensembles](https://mlsanity.com/paper/1712.05438/) (2017)
- [A representer theorem for deep neural networks](https://mlsanity.com/paper/1802.09210/) (2018)
- [Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamics](https://mlsanity.com/paper/2007.05824/) (2020)
- [Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis](https://mlsanity.com/paper/2203.01631/) (2022)
- [Universal Approximation Theorem for Equivariant Maps by Group CNNs](https://mlsanity.com/paper/2012.13882/) (2020)
- [Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups](https://mlsanity.com/paper/2205.14819/) (2022)
- [Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space](https://mlsanity.com/paper/2106.04770/) (2021)
- [Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation](https://mlsanity.com/paper/2301.11936/) (2023)
- [Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks](https://mlsanity.com/paper/2310.03530/) (2023)
- [Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities](https://mlsanity.com/paper/2310.03696/) (2023)

## Built on by (most cited first, 2 of 2)

- [Molecule Graph Networks with Many-body Equivariant Interactions](https://mlsanity.com/paper/2406.13265/) (2024)
- [Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines](https://mlsanity.com/paper/2405.13682/) (2024)

## Abstract (arXiv)

To investigate neural network parameters, it is easier to study the distribution of parameters than to study the parameters in each neuron. The ridgelet transform is a pseudo-inverse operator that maps a given function f to the parameter distribution γ so that a network mathttNN[γ] reproduces f, i.e. mathttNN[γ]=f. For depth-2 fully-connected networks on a Euclidean space, the ridgelet transform has been discovered up to the closed-form expression, thus we could describe how the parameters are distributed. However, for a variety of modern neural network architectures, the closed-form expression has not been known. In this paper, we explain a systematic method using Fourier expressions to derive ridgelet transforms for a variety of modern networks such as networks on finite fields mathbbFp, group convolutional networks on abstract Hilbert space H, fully-connected networks on noncompact symmetric spaces G/K, and pooling layers, or the d-plane ridgelet transform.