mlsanityData to 7 Oct 2026

Noether Networks: Meta-Learning Useful Conserved Quantities

arXiv 2112.03321 ↗First posted 6 Dec 2021cs.LG cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often exploit symmetries in the prediction problem, such as convolutional networks relying on translation equivariance. Automatically discovering these useful symmetries holds the potential to greatly improve the performance of ML systems, but still remains a challenge. In this work, we focus on sequential prediction problems and take inspiration from Noether's theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities. We propose Noether Networks: a new type of architecture where a meta-learned conservation loss is optimized inside the prediction function. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential problems.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 47 of its 86 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

47 of 86 references.

References · 47

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  7. 2021E(n) Equivariant Graph Neural Networkscited by 951
  8. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  9. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  10. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  11. 2019Hamiltonian Neural Networkscited by 702
  12. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
35 more
  1. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 536
  2. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 521
  3. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  4. 2020Lagrangian Neural Networkscited by 388
  5. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  6. 2018Neural Relational Inference for Interacting Systemscited by 339
  7. 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 335
  8. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  9. 2018Stochastic Video Generation with a Learned Priorcited by 244
  10. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 205
  11. 2019Symplectic ODE-Net: Learning Hamiltonian Dynamics with Controlcited by 183
  12. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  13. 2020DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningcited by 159
  14. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  15. 2020Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Predictioncited by 137
  16. 2016A Compositional Object-Based Approach to Learning Physical Dynamicscited by 119
  17. 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
  18. 2020Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraintscited by 102
  19. 2020AI Poincar'e: Machine Learning Conservation Laws from Trajectoriescited by 77
  20. 2020Meta-Learning Symmetries by Reparameterizationcited by 75
  21. 2021Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systemscited by 66
  22. 2020Tent: Fully Test-time Adaptation by Entropy Minimizationcited by 59
  23. 2019Test-Time Training with Self-Supervision for Generalization under Distribution Shiftscited by 55
  24. 2018Can recurrent neural networks warp time?cited by 53
  25. 2020Learning Invariances in Neural Networkscited by 52
  26. 2014Unsupervised Learning of Spatiotemporally Coherent Metricscited by 45
  27. 2019Graph Element Networks: adaptive, structured computation and memorycited by 41
  28. 2018Meta-Learning Update Rules for Unsupervised Representation Learningcited by 37
  29. 2020Discovering Symmetry Invariants and Conserved Quantities by Interpreting Siamese Neural Networkscited by 33
  30. 2015Slow and steady feature analysis: higher order temporal coherence in videocited by 28
  31. 2020The Surprising Effectiveness of Linear Models for Visual Foresight in Object Pile Manipulationcited by 25
  32. 2020Symbolic Pregression: Discovering Physical Laws from Distorted Videocited by 21
  33. 2021Noether: The More Things Change, the More Stay the Samecited by 19
  34. 2020Teaching with Commentariescited by 16
  35. 2020Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction timecited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 5; 2024: 10; 2025: 1; 2026: 5.

26 papers.

Sort:

By area:

  1. 2022Relaxing Equivariance Constraints with Non-stationary Continuous Filterscited by 41
  2. 2023Generative Adversarial Symmetry Discoverycited by 40
  3. 2022A posteriori learning for quasi-geostrophic turbulence parametrizationcited by 36
  4. 2024Position: Categorical Deep Learning is an Algebraic Theory of All Architecturescited by 32
  5. 2023Neural Algorithmic Reasoning with Causal Regularisationcited by 30
  6. 2024Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structurescited by 20
  7. 2022Unsupervised Learning of Equivariant Structure from Sequencescited by 15
  8. 2022FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantitiescited by 12
  9. 2025Interpretable Machine Learning in Physics: A Reviewcited by 11
  10. 2023Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networkscited by 6
  11. 2024Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equationscited by 6
  12. 2024Noether's razor: Learning Conserved Quantitiescited by 4
14 more
  1. 2024Tensor learning with orthogonal, Lorentz, and symplectic symmetriescited by 3
  2. 2022Deformable Surface Reconstruction via Riemannian Metric Preservationcited by 2
  3. 2023Pseudo-Hamiltonian system identificationcited by 1
  4. 2024Understanding Learning through the Lens of Dynamical Invariantscited by 1
  5. 2024SymmetryLens: Unsupervised Symmetry Learning via Locality and Density Preservationcited by 1
  6. 2024Discover physical concepts and equations with machine learningcited by 1
  7. 2026From Adam to Adam-Like Lagrangians: Second-Order Nonlocal Dynamicscited by 1
  8. 2026Metriplector: From Field Theory to Neural Architecturecited by 1
  9. 2023Machine Learning for the identification of phase-transitions in interacting agent-based systems: a Desai-Zwanzig examplecited by 0
  10. 2024Analysis of the Identifying Regulation with Adversarial Surrogates Algorithmcited by 0
  11. 2024Functional Risk Minimizationcited by 0
  12. 2026Blind Recovery of Latent Domains via Unsupervised Symmetry Discoverycited by 0
  13. 2026When Do Conservation Laws Survive Learned Representations? Certified Horizons for Latent World Modelscited by 0
  14. 2026LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systemscited by 0