# A Functional Perspective on Learning Symmetric Functions with Neural Networks

arXiv: https://arxiv.org/abs/2008.06952 · HTML: https://mlsanity.com/paper/2008.06952/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2020.

## Built on by (most cited first, 17 of 17)

- [A mathematical perspective on Transformers](https://mlsanity.com/paper/2312.10794/) (2023)
- [Sinkformers: Transformers with Doubly Stochastic Attention](https://mlsanity.com/paper/2110.11773/) (2021)
- [How Smooth Is Attention?](https://mlsanity.com/paper/2312.14820/) (2023)
- [The Exact Sample Complexity Gain from Invariances for Kernel Regression](https://mlsanity.com/paper/2303.14269/) (2023)
- [On the Universality of Graph Neural Networks on Large Random Graphs](https://mlsanity.com/paper/2105.13099/) (2021)
- [Deep Neural Network Approximation of Invariant Functions through Dynamical Systems](https://mlsanity.com/paper/2208.08707/) (2022)
- [Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks](https://mlsanity.com/paper/2404.18769/) (2024)
- [Understanding Catastrophic Forgetting In LoRA via Mean-Field Attention Dynamics](https://mlsanity.com/paper/2402.15415/) (2024)
- [Theoretical guarantees for the advantage of GNNs over NNs in generalizing bandlimited functions on Euclidean cubes](https://mlsanity.com/paper/2206.05904/) (2022)
- [Learning Theory of Distribution Regression with Neural Networks](https://mlsanity.com/paper/2307.03487/) (2023)
- [Softmax as Linear Attention in the Large-Prompt Regime: a Measure-based Perspective](https://mlsanity.com/paper/2512.11784/) (2025)
- [Any-Dimensional Invariant Universality](https://mlsanity.com/paper/2605.23156/) (2026)
- [Anti-symmetric Barron functions and their approximation with sums of determinants](https://mlsanity.com/paper/2303.12856/) (2023)
- [Discretization Invariant Networks for Learning Maps between Neural Fields](https://mlsanity.com/paper/2206.01178/) (2022)
- [Efficient anti-symmetrization of a neural network layer by taming the sign problem](https://mlsanity.com/paper/2205.12250/) (2022)
- [Graph Lineages and Skeletal Graph Products](https://mlsanity.com/paper/2508.00197/) (2025)
- [Dimension-Free Scaling Laws for Size Generalization: An Exponential Separation](https://mlsanity.com/paper/2610.19279/) (2026)

## Abstract (arXiv)

Symmetric functions, which take as input an unordered, fixed-size set, are known to be universally representable by neural networks that enforce permutation invariance. These architectures only give guarantees for fixed input sizes, yet in many practical applications, including point clouds and particle physics, a relevant notion of generalization should include varying the input size. In this work we treat symmetric functions (of any size) as functions over probability measures, and study the learning and representation of neural networks defined on measures. By focusing on shallow architectures, we establish approximation and generalization bounds under different choices of regularization (such as RKHS and variation norms), that capture a hierarchy of functional spaces with increasing degree of non-linear learning. The resulting models can be learned efficiently and enjoy generalization guarantees that extend across input sizes, as we verify empirically.