mlsanityData to 14 Oct 2026

Low Dimensional Invariant Embeddings for Universal Geometric Learning

arXiv 2205.02956 ↗First posted 5 May 2022cs.LG math.AGcited by 38

Not on the map. Why?

Abstract (arXiv)

This paper studies separating invariants: mappings on D dimensional domains which are invariant to an appropriate group action, and which separate orbits. The motivation for this study comes from the usefulness of separating invariants in proving universality of equivariant neural network architectures. We observe that in several cases the cardinality of separating invariants proposed in the machine learning literature is much larger than the dimension D. As a result, the theoretical universal constructions based on these separating invariants is unrealistically large. Our goal in this paper is to resolve this issue. We show that when a continuous family of semi-algebraic separating invariants is available, separation can be obtained by randomly selecting 2D+1 of these invariants. We apply this methodology to obtain an efficient scheme for computing separating invariants for several classical group actions which have been studied in the invariant learning literature. Examples include matrix multiplication actions on point clouds by permutations, rotations, and various other linear groups. Often the requirement of invariant separation is relaxed and only generic separation is required. In this case, we show that only D+1 invariants are required. More importantly, generic invariants are often significantly easier to compute, as we illustrate by discussing generic and full separation for weighted graphs. Finally we outline an approach for proving that separating invariants can be constructed also when the random parameters have finite precision.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Builds on 33 of its 77 references that are in the map. First posted on arXiv 5 May 2022.

Builds on

33 of 77 references.

References · 33

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  3. 2017Deep Setscited by 1,421
  4. 2021E(n) Equivariant Graph Neural Networkscited by 965
  5. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  6. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  7. 2019Provably Powerful Graph Networkscited by 389
  8. 2021Vector Neurons: A General Framework for SO(3)-Equivariant Networkscited by 305
  9. 2019Intrinsic dimension of data representations in deep neural networkscited by 282
  10. 2020Fast Differentiable Sorting and Rankingcited by 197
  11. 2019Universal Invariant and Equivariant Graph Neural Networkscited by 183
  12. 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 171
21 more
  1. 2021Frame Averaging for Invariant and Equivariant Network Designcited by 168
  2. 2019On the Universality of Invariant Networkscited by 152
  3. 2018Universal approximations of invariant maps by neural networkscited by 127
  4. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  5. 2021Weisfeiler and Leman go Machine Learning: The Story so farcited by 122
  6. 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 121
  7. 2019On the Limitations of Representing Functions on Setscited by 90
  8. 2020On the Universality of Rotation Equivariant Point Cloud Networkscited by 80
  9. 2019Deep Set Prediction Networkscited by 71
  10. 2015Provable approximation properties for deep neural networkscited by 60
  11. 2019FSPool: Learning Set Representations with Featurewise Sort Poolingcited by 53
  12. 2019On Universal Equivariant Set Networkscited by 45
  13. 2019Universal approximations of permutation invariant/equivariant functions by deep neural networkscited by 29
  14. 2022Permutation Invariant Representations with Applications to Graph Deep Learningcited by 20
  15. 2021ZZ-Net: A Universal Rotation Equivariant Architecture for 2D Point Cloudscited by 15
  16. 2021Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN Designcited by 14
  17. 2022A Simple and Universal Rotation Equivariant Point-cloud Networkcited by 11
  18. 2016Exact Recovery with Symmetries for Procrustes Matchingcited by 10
  19. 2019Linearly Converging Quasi Branch and Bound Algorithms for Global Rigid Registrationcited by 6
  20. 2021A simple equivariant machine learning method for dynamics based on scalarscited by 6
  21. 2017Exact Recovery with Symmetries for the Doubly-Stochastic Relaxationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 8; 2024: 11; 2025: 11; 2026: 6.

38 papers.

Sort:

By area:

  1. 2023Neural Injective Functions for Multisets, Measures and Graphs via a Finite Witness Theoremcited by 34
  2. 2024Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscited by 22
  3. 2024Equivariance via Minimal Frame Averaging for More Symmetries and Efficiencycited by 19
  4. 2022Group-invariant max filteringcited by 18
  5. 2023Expressive Sign Equivariant Networks for Spectral Geometric Learningcited by 15
  6. 2022Machine learning and invariant theorycited by 13
  7. 2024On the Completeness of Invariant Geometric Deep Learning Modelscited by 11
  8. 2023Towards fully covariant machine learningcited by 10
  9. 2023Complete Neural Networks for Complete Euclidean Graphscited by 10
  10. 2023Universal Representation of Permutation-Invariant Functions on Vectors and Tensorscited by 10
  11. 2024A Characterization Theorem for Equivariant Networks with Point-wise Activationscited by 9
  12. 2024FSW-GNN: A Bi-Lipschitz WL-Equivalent Graph Neural Networkcited by 8
26 more
  1. 2025Fourier Sliced-Wasserstein Embedding for Multisets and Measurescited by 8
  2. 2024On the Expressive Power of Sparse Geometric MPNNscited by 7
  3. 2023Polynomial Width is Sufficient for Set Representation with High-dimensional Featurescited by 6
  4. 2024On the H"older Stability of Multiset and Graph Neural Networkscited by 6
  5. 2025Learning with Exact Invariances in Polynomial Timecited by 6
  6. 2023Neural approximation of Wasserstein distance via a universal architecture for symmetric and factorwise group invariant functionscited by 4
  7. 2023Learning Symmetrization for Equivariance with Orbit Distance Minimizationcited by 4
  8. 2024Sequential Signal Mixing Aggregation for Message Passing Graph Neural Networkscited by 3
  9. 2024Stability of sorting based embeddingscited by 3
  10. 2025Bi-Lipschitz Ansatz for Anti-Symmetric Functionscited by 3
  11. 2025On Universality of Deep Equivariant Networkscited by 3
  12. 2026Efficient Learning and Symmetry Discovery under Exact Invariancescited by 3
  13. 2024Subequivariant Reinforcement Learning Framework for Coordinated Motion Controlcited by 2
  14. 2025Quantitative Bounds for Sorting-Based Permutation-Invariant Embeddingscited by 2
  15. 2025Equivariant Eikonal Neural Networks: Grid-Free, Scalable Travel-Time Prediction on Homogeneous Spacescited by 1
  16. 2025On the (Non) Injectivity of Piecewise Linear Janossy Poolingcited by 1
  17. 2025Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoningcited by 1
  18. 2025Toward bilipshiz geometric modelscited by 1
  19. 2026Embedding Dimension Lower Bounds for Universality of Deep Sets and Janossy Poolingcited by 1
  20. 2026Invariant Shape Analysis of Surfaces with Spherical Topologycited by 1
  21. 2024Fourier Sliced-Wasserstein Embedding for Multisets and Measurescited by 0
  22. 2025Reconstructing Rational Functions on Finite Abelian Groups with Higher Autocorrelationscited by 0
  23. 2025If you can distinguish, you can express: Galois theory, Stone--Weierstrass, machine learning, and linguisticscited by 0
  24. 2026Generalized Reduction to the Isotropy for Flexible Equivariant Neural Fieldscited by 0
  25. 2026Group-invariant Coresets for Data-efficient Active Learningcited by 0
  26. 2026Finite Probes Suffice: Identifiability and Universality for Weight-Space Learningcited by 0