# Noether Networks: Meta-Learning Useful Conserved Quantities

arXiv: https://arxiv.org/abs/2112.03321 · HTML: https://mlsanity.com/paper/2112.03321/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 86 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 30 of 47)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Stochastic Video Generation with a Learned Prior](https://mlsanity.com/paper/1802.07687/) (2018)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [Action-Conditional Video Prediction using Deep Networks in Atari Games](https://mlsanity.com/paper/1507.08750/) (2015)
- [DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning](https://mlsanity.com/paper/2006.08381/) (2020)
- [One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning](https://mlsanity.com/paper/1802.01557/) (2018)
- [Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction](https://mlsanity.com/paper/2007.04439/) (2020)
- [A Compositional Object-Based Approach to Learning Physical Dynamics](https://mlsanity.com/paper/1612.00341/) (2016)
- [Broaden Your Views for Self-Supervised Video Learning](https://mlsanity.com/paper/2103.16559/) (2021)
- [Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints](https://mlsanity.com/paper/2010.13581/) (2020)

## Built on by (most cited first, 26 of 26)

- [Generative Adversarial Symmetry Discovery](https://mlsanity.com/paper/2302.00236/) (2023)
- [Relaxing Equivariance Constraints with Non-stationary Continuous Filters](https://mlsanity.com/paper/2204.07178/) (2022)
- [A posteriori learning for quasi-geostrophic turbulence parametrization](https://mlsanity.com/paper/2204.03911/) (2022)
- [Position: Categorical Deep Learning is an Algebraic Theory of All Architectures](https://mlsanity.com/paper/2402.15332/) (2024)
- [Neural Algorithmic Reasoning with Causal Regularisation](https://mlsanity.com/paper/2302.10258/) (2023)
- [Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures](https://mlsanity.com/paper/2407.09468/) (2024)
- [Unsupervised Learning of Equivariant Structure from Sequences](https://mlsanity.com/paper/2210.05972/) (2022)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities](https://mlsanity.com/paper/2210.00272/) (2022)
- [Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations](https://mlsanity.com/paper/2411.09734/) (2024)
- [Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networks](https://mlsanity.com/paper/2304.12865/) (2023)
- [Noether's razor: Learning Conserved Quantities](https://mlsanity.com/paper/2410.08087/) (2024)
- [Tensor learning with orthogonal, Lorentz, and symplectic symmetries](https://mlsanity.com/paper/2406.01552/) (2024)
- [Deformable Surface Reconstruction via Riemannian Metric Preservation](https://mlsanity.com/paper/2212.11596/) (2022)
- [Metriplector: From Field Theory to Neural Architecture](https://mlsanity.com/paper/2603.29496/) (2026)
- [Pseudo-Hamiltonian system identification](https://mlsanity.com/paper/2305.06920/) (2023)
- [Understanding Learning through the Lens of Dynamical Invariants](https://mlsanity.com/paper/2401.10428/) (2024)
- [SymmetryLens: Unsupervised Symmetry Learning via Locality and Density Preservation](https://mlsanity.com/paper/2410.05232/) (2024)
- [Discover physical concepts and equations with machine learning](https://mlsanity.com/paper/2412.12161/) (2024)
- [From Adam to Adam-Like Lagrangians: Second-Order Nonlocal Dynamics](https://mlsanity.com/paper/2602.09101/) (2026)
- [Machine Learning for the identification of phase-transitions in interacting agent-based systems: a Desai-Zwanzig example](https://mlsanity.com/paper/2310.19039/) (2023)
- [Analysis of the Identifying Regulation with Adversarial Surrogates Algorithm](https://mlsanity.com/paper/2405.02953/) (2024)
- [Functional Risk Minimization](https://mlsanity.com/paper/2412.21149/) (2024)
- [Blind Recovery of Latent Domains via Unsupervised Symmetry Discovery](https://mlsanity.com/paper/2606.17782/) (2026)
- [When Do Conservation Laws Survive Learned Representations? Certified Horizons for Latent World Models](https://mlsanity.com/paper/2606.24945/) (2026)
- [LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systems](https://mlsanity.com/paper/2608.01582/) (2026)

## Abstract (arXiv)

Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often exploit symmetries in the prediction problem, such as convolutional networks relying on translation equivariance. Automatically discovering these useful symmetries holds the potential to greatly improve the performance of ML systems, but still remains a challenge. In this work, we focus on sequential prediction problems and take inspiration from Noether's theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities. We propose Noether Networks: a new type of architecture where a meta-learned conservation loss is optimized inside the prediction function. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential problems.