# Estimating Global Input Relevance and Enforcing Sparse Representations with a Scalable Spectral Neural Network Approach

arXiv: https://arxiv.org/abs/2406.01183 · HTML: https://mlsanity.com/paper/2406.01183/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 46 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Builds on (most cited first, 13 of 13)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Graph neural networks for materials science and chemistry](https://mlsanity.com/paper/2208.09481/) (2022)
- [Machine learning in spectral domain](https://mlsanity.com/paper/2005.14436/) (2020)
- [SGD and Weight Decay Secretly Minimize the Rank of Your Neural Network](https://mlsanity.com/paper/2206.05794/) (2022)
- [How a student becomes a teacher: learning and forgetting through Spectral methods](https://mlsanity.com/paper/2310.12612/) (2023)
- [Complex Recurrent Spectral Network](https://mlsanity.com/paper/2312.07296/) (2023)
- [Recurrent Spectral Network (RSN): shaping the basin of attraction of a discrete map to reach automated classification](https://mlsanity.com/paper/2202.04497/) (2022)

## Built on by (most cited first, 4 of 4)

- [Deterministic versus stochastic dynamical classifiers: opposing random adversarial attacks with noise](https://mlsanity.com/paper/2409.13470/) (2024)
- [Spectral Higher-Order Neural Networks](https://mlsanity.com/paper/2603.28420/) (2026)
- [Spectral Higher-Order Neural Networks Have Sharp Expressivity Bounds](https://mlsanity.com/paper/2607.19042/) (2026)
- [Learning-Based Reconstruction of Optical Properties in Bilayered Media from Single-distance Time-Resolved Reflectance Measurements](https://mlsanity.com/paper/2609.19786/) (2026)

## Abstract (arXiv)

In machine learning practice it is often useful to identify relevant input features. Isolating key input elements, ranked according their respective degree of relevance, can help to elaborate on the process of decision making. Here, we propose a novel method to estimate the relative importance of the input components for a Deep Neural Network. This is achieved by leveraging on a spectral re-parametrization of the optimization process. Eigenvalues associated to input nodes provide in fact a robust proxy to gauge the relevance of the supplied entry features. Notably, the spectral features ranking is performed automatically, as a byproduct of the network training, with no additional processing to be carried out. Moreover, by leveraging on the regularization of the eigenvalues, it is possible to enforce solutions making use of a minimum subset of the input components, increasing the explainability of the model and providing sparse input representations. The technique is compared to the most common methods in the literature and is successfully challenged against both synthetic and real data.