# Machine learning in spectral domain

arXiv: https://arxiv.org/abs/2005.14436 · HTML: https://mlsanity.com/paper/2005.14436/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2020.

## Built on by (most cited first, 20 of 20)

- [Spectrum-guided Multi-granularity Referring Video Object Segmentation](https://mlsanity.com/paper/2307.13537/) (2023)
- [Emergence of hierarchical modes from deep learning](https://mlsanity.com/paper/2208.09859/) (2022)
- [How a student becomes a teacher: learning and forgetting through Spectral methods](https://mlsanity.com/paper/2310.12612/) (2023)
- [Stable Attractors for Neural networks classification via Ordinary Differential Equations (SA-nODE)](https://mlsanity.com/paper/2311.10387/) (2023)
- [Complex Recurrent Spectral Network](https://mlsanity.com/paper/2312.07296/) (2023)
- [Recurrent Spectral Network (RSN): shaping the basin of attraction of a discrete map to reach automated classification](https://mlsanity.com/paper/2202.04497/) (2022)
- [Estimating Global Input Relevance and Enforcing Sparse Representations with a Scalable Spectral Neural Network Approach](https://mlsanity.com/paper/2406.01183/) (2024)
- [Deterministic versus stochastic dynamical classifiers: opposing random adversarial attacks with noise](https://mlsanity.com/paper/2409.13470/) (2024)
- [On the training of sparse and dense deep neural networks: less parameters, same performance](https://mlsanity.com/paper/2106.09021/) (2021)
- [Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems](https://mlsanity.com/paper/2309.05337/) (2023)
- [Extraction of nonlinearity in neural networks with Koopman operator](https://mlsanity.com/paper/2402.11740/) (2024)
- [Spectral Architecture Search for Neural Network Models](https://mlsanity.com/paper/2504.00885/) (2025)
- [Spectral Higher-Order Neural Networks](https://mlsanity.com/paper/2603.28420/) (2026)
- [Dual-path convolutional neural network using micro-FTIR imaging to predict breast cancer subtypes and biomarkers levels: estrogen receptor, progesterone receptor, HER2 and Ki67](https://mlsanity.com/paper/2310.15099/) (2023)
- [Engineered Ordinary Differential Equations as Classification Algorithm (EODECA): thorough characterization and testing](https://mlsanity.com/paper/2312.14681/) (2023)
- [Towards Symmetric Low-Rank Adapters](https://mlsanity.com/paper/2504.03719/) (2025)
- [Neural Operators for Biomedical Spherical Heterogeneity](https://mlsanity.com/paper/2601.03561/) (2026)
- [Factorized Neural Operators Decompose Dynamic and Persistent Responses](https://mlsanity.com/paper/2606.16900/) (2026)
- [Spectral Higher-Order Neural Networks Have Sharp Expressivity Bounds](https://mlsanity.com/paper/2607.19042/) (2026)
- [Learning-Based Reconstruction of Optical Properties in Bilayered Media from Single-distance Time-Resolved Reflectance Measurements](https://mlsanity.com/paper/2609.19786/) (2026)

## Abstract (arXiv)

Deep neural networks are usually trained in the space of the nodes, by adjusting the weights of existing links via suitable optimization protocols. We here propose a radically new approach which anchors the learning process to reciprocal space. Specifically, the training acts on the spectral domain and seeks to modify the eigenvalues and eigenvectors of transfer operators in direct space. The proposed method is ductile and can be tailored to return either linear or non-linear classifiers. Adjusting the eigenvalues, when freezing the eigenvectors entries, yields performances which are superior to those attained with standard methods it restricted to a operate with an identical number of free parameters. Tuning the eigenvalues correspond in fact to performing a global training of the neural network, a procedure which promotes (resp. inhibits) collective modes on which an effective information processing relies. This is at variance with the usual approach to learning which implements instead a local modulation of the weights associated to pairwise links. Interestingly, spectral learning limited to the eigenvalues returns a distribution of the predicted weights which is close to that obtained when training the neural network in direct space, with no restrictions on the parameters to be tuned. Based on the above, it is surmised that spectral learning bound to the eigenvalues could be also employed for pre-training of deep neural networks, in conjunction with conventional machine-learning schemes. Changing the eigenvectors to a different non-orthogonal basis alters the topology of the network in direct space and thus allows to export the spectral learning strategy to other frameworks, as e.g. reservoir computing.