# Invariant polynomials and machine learning

arXiv: https://arxiv.org/abs/2104.12733 · HTML: https://mlsanity.com/paper/2104.12733/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 23 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 5 of 5)

- [Universal approximations of invariant maps by neural networks](https://mlsanity.com/paper/1804.10306/) (2018)
- [Lorentz Group Equivariant Neural Network for Particle Physics](https://mlsanity.com/paper/2006.04780/) (2020)
- [The universal approximation theorem for complex-valued neural networks](https://mlsanity.com/paper/2012.03351/) (2020)
- [A closer look at the approximation capabilities of neural networks](https://mlsanity.com/paper/2002.06505/) (2020)
- [Compromise-free Bayesian neural networks](https://mlsanity.com/paper/2004.12211/) (2020)

## Built on by (most cited first, 4 of 4)

- [Scalars are universal: Equivariant machine learning, structured like classical physics](https://mlsanity.com/paper/2106.06610/) (2021)
- [Dimensionless machine learning: Imposing exact units equivariance](https://mlsanity.com/paper/2204.00887/) (2022)
- [Machine learning and invariant theory](https://mlsanity.com/paper/2209.14991/) (2022)
- [Learning Polynomial Problems with SL(2,ℝ) Equivariance](https://mlsanity.com/paper/2312.02146/) (2023)

## Abstract (arXiv)

We present an application of invariant polynomials in machine learning. Using the methods developed in previous work, we obtain two types of generators of the Lorentz- and permutation-invariant polynomials in particle momenta; minimal algebra generators and Hironaka decompositions. We discuss and prove some approximation theorems to make use of these invariant generators in machine learning algorithms in general and in neural networks specifically. By implementing these generators in neural networks applied to regression tasks, we test the improvements in performance under a wide range of hyperparameter choices and find a reduction of the loss on training data and a significant reduction of the loss on validation data. For a different approach on quantifying the performance of these neural networks, we treat the problem from a Bayesian inference perspective and employ nested sampling techniques to perform model comparison. Beyond a certain network size, we find that networks utilising Hironaka decompositions perform the best.