mlsanityData to 7 Oct 2026

Revisiting Orbital Minimization Method for Neural Operator Decomposition

arXiv 2510.21952 ↗First posted 24 Oct 2025cs.LG cs.NA math.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

Spectral decomposition of linear operators plays a central role in many areas of machine learning and scientific computing. Recent work has explored training neural networks to approximate eigenfunctions of such operators, enabling scalable approaches to representation learning, dynamical systems, and partial differential equations (PDEs). In this paper, we revisit a classical optimization framework from the computational physics literature known as the orbital minimization method (OMM), originally proposed in the 1990s for solving eigenvalue problems in computational chemistry. We provide a simple linear-algebraic proof of the consistency of the OMM objective, and reveal connections between this method and several ideas that have appeared independently across different domains. Our primary goal is to justify its broader applicability in modern learning pipelines. We adapt this framework to train neural networks to decompose positive semidefinite operators, and demonstrate its practical advantages across a range of benchmark tasks. Our results highlight how revisiting classical numerical methods through the lens of modern theory and computation can provide not only a principled approach for deploying neural networks in numerical simulation, but also effective and scalable tools for machine learning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 57 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

32 of 57 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  7. 2017Large Batch Training of Convolutional Networkscited by 529
  8. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  9. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  10. 2020Combining Label Propagation and Simple Models Out-performs Graph Neural Networkscited by 199
  11. 2019Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networkscited by 195
  12. 2017VAMPnets: Deep learning of molecular kineticscited by 135
20 more
  1. 2021Solo-learn: A Library of Self-supervised Methods for Visual Representation Learningcited by 116
  2. 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
  3. 2018The Laplacian in RL: Learning Representations with Efficient Approximationscited by 83
  4. 2020EigenGame: PCA as a Nash Equilibriumcited by 39
  5. 2018Spectral Inference Networks: Unifying Deep and Spectral Learningcited by 34
  6. 2022NeuralEF: Deconstructing Kernels by Deep Neural Networkscited by 26
  7. 2021Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawingcited by 22
  8. 2022Electronic excited states in deep variational Monte Carlocited by 21
  9. 2018An Efficient Approach to Informative Feature Extraction from Multimodal Datacited by 20
  10. 2023Learning invariant representations of time-homogeneous stochastic dynamical systemscited by 17
  11. 2022Neural Eigenfunctions Are Structured Representation Learnerscited by 16
  12. 2023Accurate Computation of Quantum Excited States with Neural Networkscited by 16
  13. 2024Neural Conditional Probability for Uncertainty Quantificationcited by 14
  14. 2021EigenGame Unloaded: When playing games is better than optimizingcited by 13
  15. 2019Exponentially convergent stochastic k-PCA without variance reductioncited by 12
  16. 2024Operator SVD with Neural Networks via Nested Low-Rank Approximationcited by 12
  17. 2023Neural Feature Learning in Function Spacecited by 11
  18. 2023Proper Laplacian Representation Learningcited by 9
  19. 2023Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learningcited by 6
  20. 2025Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systemscited by 5

Built on by

Not cited yet.