# Bispectral Neural Networks

arXiv: https://arxiv.org/abs/2209.03416 · HTML: https://mlsanity.com/paper/2209.03416/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 61 references that are in the map.
- First posted on arXiv 7 Sep 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Excessive Invariance Causes Adversarial Vulnerability](https://mlsanity.com/paper/1811.00401/) (2018)
- [PyTorch Metric Learning](https://mlsanity.com/paper/2008.09164/) (2020)
- [Covariant Compositional Networks For Learning Graphs](https://mlsanity.com/paper/1801.02144/) (2018)
- [Learning the Irreducible Representations of Commutative Lie Groups](https://mlsanity.com/paper/1402.4437/) (2014)
- [Symmetry-Based Representations for Artificial and Biological General Intelligence](https://mlsanity.com/paper/2203.09250/) (2022)
- [Disentangling images with Lie group transformations and sparse coding](https://mlsanity.com/paper/2012.12071/) (2020)
- [Transformational Sparse Coding](https://mlsanity.com/paper/1712.03257/) (2017)

## Built on by (most cited first, 18 of 18)

- [Pre-trained Large Language Models Use Fourier Features to Compute Addition](https://mlsanity.com/paper/2406.03445/) (2024)
- [Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks](https://mlsanity.com/paper/2312.08550/) (2023)
- [On genuine invariance learning without weight-tying](https://mlsanity.com/paper/2308.03904/) (2023)
- [On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory](https://mlsanity.com/paper/2412.11521/) (2024)
- [A polar prediction model for learning to represent visual transformations](https://mlsanity.com/paper/2303.03432/) (2023)
- [Learning Lie Group Symmetry Transformations with Neural Networks](https://mlsanity.com/paper/2307.01583/) (2023)
- [The Selective G-Bispectrum and its Inversion: Applications to G-Invariant Networks](https://mlsanity.com/paper/2407.07655/) (2024)
- [A General Framework for Robust G-Invariance in G-Equivariant Networks](https://mlsanity.com/paper/2310.18564/) (2023)
- [Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensors](https://mlsanity.com/paper/2412.04594/) (2024)
- [Learning Equivariant Functions via Quadratic Forms](https://mlsanity.com/paper/2509.22184/) (2025)
- [Selective Disk Bispectrum: A Complete and Rotation Invariant Image Descriptor](https://mlsanity.com/paper/2511.19706/) (2025)
- [A Differentiable Measure of Algebraic Complexity: Provably Exact Discovery of Group Structures](https://mlsanity.com/paper/2511.23152/) (2025)
- [Exact Symmetry as Algebra: Group-Structured Tensors that Enforce Physical Selection Rules](https://mlsanity.com/paper/2605.20440/) (2026)
- [Learning Linear Groups in Neural Networks](https://mlsanity.com/paper/2305.18552/) (2023)
- [Bispectral OT: Dataset Comparison using Symmetry-Aware Optimal Transport](https://mlsanity.com/paper/2509.20678/) (2025)
- [Interpretable Discovery of One-parameter Subgroups: A Modular Framework for Elliptical, Hyperbolic, and Parabolic Symmetries](https://mlsanity.com/paper/2509.22219/) (2025)
- [Spectral Discovery of Continuous Symmetries via Generalized Fourier Transforms](https://mlsanity.com/paper/2603.07299/) (2026)
- [bispectrum: Selective G-Bispectra Made Practical](https://mlsanity.com/paper/2605.07270/) (2026)

## Abstract (arXiv)

We present a neural network architecture, Bispectral Neural Networks (BNNs) for learning representations that are invariant to the actions of compact commutative groups on the space over which a signal is defined. The model incorporates the ansatz of the bispectrum, an analytically defined group invariant that is complete -- that is, it preserves all signal structure while removing only the variation due to group actions. Here, we demonstrate that BNNs are able to simultaneously learn groups, their irreducible representations, and corresponding equivariant and complete-invariant maps purely from the symmetries implicit in data. Further, we demonstrate that the completeness property endows these networks with strong invariance-based adversarial robustness. This work establishes Bispectral Neural Networks as a powerful computational primitive for robust invariant representation learning