mlsanityData to 14 Oct 2026

NeuralEF: Deconstructing Kernels by Deep Neural Networks

arXiv 2205.00165 ↗First posted 30 Apr 2022cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Learning the principal eigenfunctions of an integral operator defined by a kernel and a data distribution is at the core of many machine learning problems. Traditional nonparametric solutions based on the Nystr"om formula suffer from scalability issues. Recent work has resorted to a parametric approach, i.e., training neural networks to approximate the eigenfunctions. However, the existing method relies on an expensive orthogonalization step and is difficult to implement. We show that these problems can be fixed by using a new series of objective functions that generalizes the EigenGame~citepgemp2020eigengame to function space. We test our method on a variety of supervised and unsupervised learning problems and show it provides accurate approximations to the eigenfunctions of polynomial, radial basis, neural network Gaussian process, and neural tangent kernels. Finally, we demonstrate our method can scale up linearised Laplace approximation of deep neural networks to modern image classification datasets through approximating the Gauss-Newton matrix. Code is available at urlhttps://github.com/thudzj/neuraleigenfunction.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 30 Apr 2022.

Builds on

25 of 49 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  8. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  9. 2017Deep Neural Networks as Gaussian Processescited by 622
  10. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 573
  11. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  12. 2015Deep Kernel Learningcited by 445
13 more
  1. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  2. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  3. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
  4. 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 167
  5. 2019Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernelscited by 161
  6. 2019Neural Tangents: Fast and Easy Infinite Neural Networks in Pythoncited by 149
  7. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 138
  8. 2016Orthogonal Random Featurescited by 91
  9. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  10. 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 80
  11. 2020EigenGame: PCA as a Nash Equilibriumcited by 39
  12. 2018Spectral Inference Networks: Unifying Deep and Spectral Learningcited by 35
  13. 2018Quadrature-based features for kernel approximationcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 5; 2024: 1; 2025: 9; 2026: 7.

27 papers.

Sort:

By area:

  1. 2023Improved Operator Learning by Orthogonal Attentioncited by 61
  2. 2022Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functionscited by 32
  3. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 28
  4. 2022Neural Eigenfunctions Are Structured Representation Learnerscited by 16
  5. 2024Operator SVD with Neural Networks via Nested Low-Rank Approximationcited by 13
  6. 2022A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspacescited by 7
  7. 2023Harnessing small projectors and multiple views for efficient vision pretrainingcited by 5
  8. 2025Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systemscited by 5
  9. 2023Learning Neural Eigenfunctions for Unsupervised Semantic Segmentationcited by 4
  10. 2025Demystifying Spectral Feature Learning for Instrumental Variable Regressioncited by 4
  11. 2025STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problemcited by 3
  12. 2022Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernelcited by 2
15 more
  1. 2023Representations and Exploration for Deep Reinforcement Learning using Singular Value Decompositioncited by 2
  2. 2025Generalizable Spectral Embedding with an Application to UMAPcited by 1
  3. 2025Accelerating Eigenvalue Dataset Generation via Chebyshev Subspace Filtercited by 1
  4. 2023Contrastive Learning as Kernel Approximationcited by 0
  5. 2025Compact Vision Transformer by Reduction of Kernel Complexitycited by 0
  6. 2025Revisiting Orbital Minimization Method for Neural Operator Decompositioncited by 0
  7. 2025Eigenfunction Extraction for Ordered Representation Learningcited by 0
  8. 2025Learning-Guided Integration Contours Construction for Fast Large-Scale Generalized Eigensolverscited by 0
  9. 2026NeuraLSP: A Neural Spectral Preconditioner for Accelerating PDE Solverscited by 0
  10. 2026From Complex Dynamics to DynFormer: Rethinking Transformers for PDEscited by 0
  11. 2026Minimax Rates and Spectral Distillation for Tree Ensemblescited by 0
  12. 2026Learning Orthonormal Bases for Function Spacescited by 0
  13. 2026Score the Algebra, Not the Span: Dimension Reduction for Transfer Operator Models of Dynamical Systemscited by 0
  14. 2026Revisiting Spectral Representations in Generative Diffusion Modelscited by 0
  15. 2026Low-Fidelity FDM Spectral Guidance for Neural Eigenvalue Solverscited by 0