mlsanityData to 7 Oct 2026

On the Universality of Invariant Networks

arXiv 1901.09342 ↗First posted 27 Jan 2019cs.LG stat.MLcited by 151

Not on the map. Why?

Abstract (arXiv)

Constraining linear layers in neural networks to respect symmetry transformations from a group G is a common design principle for invariant networks that has found many applications in machine learning. In this paper, we consider a fundamental question that has received little attention to date: Can these networks approximate any (continuous) invariant function? We tackle the rather general case where G≤ Sn (an arbitrary subgroup of the symmetric group) that acts on ℝn by permuting coordinates. This setting includes several recent popular invariant networks. We present two main results: First, G-invariant networks are universal if high-order tensors are allowed. Second, there are groups G for which higher-order tensors are unavoidable for obtaining universality. G-invariant networks consisting of only first-order tensors are of special interest due to their practical value. We conclude the paper by proving a necessary condition for the universality of G-invariant networks that incorporate only first-order tensors.

Key facts. Cited by 151 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 34; 2023: 31; 2024: 25; 2025: 24; 2026: 8.

151 papers.

Sort:

  1. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 342
  2. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  3. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
  4. 2022Sign and Basis Invariant Networks for Spectral Graph Representation Learningcited by 176
  5. 2022How Powerful are Spectral Graph Neural Networkscited by 169
  6. 2021Nested Graph Neural Networkscited by 136
  7. 2023Representational Strengths and Limitations of Transformerscited by 132
  8. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  9. 2021Weisfeiler and Leman go Machine Learning: The Story so farcited by 121
  10. 2022Understanding and Extending Subgraph GNNs by Rethinking Their Symmetriescited by 112
  11. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 106
  12. 2021Breaking the Limits of Message Passing Graph Neural Networkscited by 101
38 more
  1. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 90
  2. 2024PirateNets: Physics-informed Deep Learning with Residual Adaptive Networkscited by 70
  3. 2021Understanding Pooling in Graph Neural Networkscited by 69
  4. 2021Reconstruction for Powerful Graph Representationscited by 65
  5. 2022Theory of Graph Neural Networks: Representation and Learningcited by 60
  6. 2022Theory for Equivariant Quantum Neural Networkscited by 60
  7. 2022Boosting the Cycle Counting Power of Graph Neural Networks with I2-GNNscited by 55
  8. 2023How does over-squashing affect the power of GNNs?cited by 53
  9. 2022Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networkscited by 52
  10. 2021Graph Neural Networks with Local Graph Parameterscited by 51
  11. 2022Shortest Path Networks for Graph Property Predictioncited by 46
  12. 2021Boost then Convolve: Gradient Boosting Meets Graph Neural Networkscited by 42
  13. 2021Autobahn: Automorphism-based Graph Neural Netscited by 39
  14. 2022On Representing Linear Programs by Graph Neural Networkscited by 39
  15. 2022Low Dimensional Invariant Embeddings for Universal Geometric Learningcited by 38
  16. 2022SpeqNets: Sparsity-aware Permutation-equivariant Graph Networkscited by 37
  17. 2021Provably Strict Generalisation Benefit for Invariance in Kernel Methodscited by 29
  18. 2021Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlatorscited by 28
  19. 2023The Exact Sample Complexity Gain from Invariances for Kernel Regressioncited by 27
  20. 2023The Expressive Power of Graph Neural Networks: A Surveycited by 27
  21. 2021On the Universality of Graph Neural Networks on Large Random Graphscited by 26
  22. 2023Learning Probabilistic Symmetrization for Architecture Agnostic Equivariancecited by 25
  23. 2024A Canonicalization Perspective on Invariant and Equivariant Learningcited by 25
  24. 2023Explainable Equivariant Neural Networks for Particle Physics: PELICANcited by 22
  25. 2024A Survey on Hypergraph Neural Networks: An In-Depth and Step-By-Step Guidecited by 22
  26. 2022What is an equivariant neural network?cited by 21
  27. 2023Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embeddingcited by 21
  28. 2024The Empirical Impact of Neural Parameter Symmetries, or Lack Thereofcited by 21
  29. 2024Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscited by 21
  30. 2021Multiresolution Equivariant Graph Variational Autoencodercited by 20
  31. 2022Permutation Invariant Representations with Applications to Graph Deep Learningcited by 20
  32. 2022A Practical, Progressively-Expressive GNNcited by 20
  33. 2023A General Theory of Correct, Incorrect, and Extrinsic Equivariancecited by 18
  34. 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 17
  35. 2025Equivariance Everywhere All At Once: A Recipe for Graph Foundation Modelscited by 17
  36. 2021Graph Neural Networks: Methods, Applications, and Opportunitiescited by 16
  37. 2024Homomorphism Counts for Graph Neural Networks: All About That Basiscited by 16
  38. 2021Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformerscited by 15

50 most cited of 151 All 151 (JSON)